CrawlCheck

Agent view · Findings · 2026-08-21

A save that returned success rewrote every newline

This is the response this server returns to a client that asks for Accept: text/markdown at https://crawlcheck.io/blog/a-write-that-returned-success-and-corrupted-every-newline: fetched when you opened this page, shown byte for byte. An assistant, a crawler that negotiates markdown, or anything reading our feed gets exactly this. The markdown leaves signed (RFC 9421, key in our key directory), so a crawler can prove the page it read is the page this server sent; the check is shown below.

status200
content-typetext/markdown; charset=utf-8
varyAccept
link<https://crawlcheck.io/llms.txt>; rel="describedby"; type="text/plain", <https://crawlcheck.io/.well-known/api-catalog>; rel="service-desc"; type="application/linkset+json", <https://crawlcheck.io/blog/a-write-that-returned-success-and-corrupted-every-newline>; rel="cite-as", <https://crawlcheck.io/.well-known/http-message-signatures-directory>; rel="http-message-signatures-directory"
cache-controlpublic, max-age=900, stale-while-revalidate=2400
bytes12,317
words1,767
sha-256037e0c321509a8f0cb17aa6edb41e0d921464e04231331243253c97aa063f5c4
repr-digestsha-256=:A34MMhUJqPDLF6pu20Hg2SFGTgQjEzEkMlPJeqBj9cQ=: ← matches the body shown
signature-inputsig1=("@authority";req "@path";req "repr-digest" "content-type");created=1795311493;expires=1795315093;keyid="1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps";alg="ed25519";nonce="83BKdP97NCjF9TJbKXrQBQ";tag="crawlcheck-page"
signaturesig1=:uwzh8zR66zMMYvpkVzulJ2y3sA7i2PJoDzlS2bGLtqsGR/n+8QfSG5AhoRgGmNmQ2PXAWhD0FFfeLY3Er+5BCw==:
verifiedyes — signature by key 1hHmE3fo1rg5Ig70… from https://crawlcheck.io/.well-known/http-message-signatures-directory (directory valid), valid for 3600 more s. Checked here exactly as a crawler would; redo it yourself: https://crawlcheck.io/api/tool/wba
Findings · 2026-08-21 · By [VSNARY | Emmanuel Orta](/author)

# A save that returned success rewrote every newline

The write reported success. Read back, the file was 16,330 bytes larger, exactly the number of newlines it contained. A store accepting your bytes is not the same as keeping them.

A save that returns success tells you the request was accepted, not that the bytes were kept. We wrote a 1,217,790-byte file, got a success response and read back 1,234,120 bytes: exactly 16,330 more, one for each bare newline, because the content was sent as a text field in a multipart form, and text fields carry CRLF line endings. Sending the same content as a file part kept it byte-identical. After any important write, read the data back and compare a hash, not only the length.

On this page

- Where did the extra bytes come from?
- Why were the 313 existing line endings worse?
- What does a success response actually prove?
- What should you do after any important write?

What a success response proves

The request was received and parsed

Proven by a success response?

Yes

Something was stored under that name

Proven by a success response?

Usually

What was stored is byte-identical to what you sent

Proven by a success response?

No

It can be read back at all

Proven by a success response?

No

We wrote a file of 1,217,790 bytes to storage. The response said it succeeded. We read it back anyway and got 1,234,120 bytes. Nothing reported an error.

## Where did the extra bytes come from? #

| Measure | Bytes or count |
| --- | --- |
| Bytes written | 1,217,790 |
| Bytes read back | 1,234,120 |
| Difference | 16,330 |
| Bare LF newlines in the source | 16,330 |
| CRLF pairs already in the source | 313 |

An exact match: one byte added per bare line feed. Every `\n` had become `\r\n` on the way in. The cause was the transport, not the store. The content had been attached to a multipart form as a text field, and multipart form-data has a long-standing convention that text field values use CRLF line endings. The encoder did what the format says; the mistake was sending exact bytes as text. Attaching the same content as a file part (a `Blob`) fixed it: the read-back matched byte for byte, and the [SHA-256](/glossary/sha-256) of both agreed.

## Why were the 313 existing line endings worse? #

The file already contained 313 deliberate `\r\n` sequences inside string literals. A normaliser that rewrites every `\n` reaches those too and turns them into `\r\r\n`. So most of the file gained a harmless-looking line ending, and a few hundred places gained a stray carriage return inside data that meant something. Restored, the failure would not have been "the file is bigger". It would have been a handful of oddly broken strings with a plausible byte count.

## What does a success response actually prove? #

| Claim | Proven by a success response? |
| --- | --- |
| The request was received and parsed | Yes |
| Something was stored under that name | Usually |
| What was stored is byte-identical to what you sent | No |
| It can be read back at all | No |

It is the same shape as [a cache purge that returned 200 and evicted nothing](/blog/the-purge-returned-200-and-evicted-nothing): the status describes the call, not the effect. The [status codes guide](/guides/http-status-codes-what-a-crawler-actually-gets) covers what a 200 can hide on the crawler side.

## What should you do after any important write? #

- **Read it back and compare a hash, not a length.** Length caught this one because the size changed; a substitution of equal length passes a length check and fails a hash.
- **Never send exact bytes through a text field.** If an upload API offers a file or binary part, use it for anything you mean to restore later.
- **Delete a bad copy as soon as you confirm it.** We removed the corrupted one at once, so no future restore could pick it. A silently wrong copy is worse than a missing one, which at least fails loudly.

Method: byte counts were taken on the exact buffer sent and on the buffer returned by reading the same name back in the same session; newlines were counted as `0x0A` bytes and `0x0D 0x0A` pairs in the source; verification after the fix compared SHA-256 digests.

This is one of the corrections CrawlCheck publishes when its own output was wrong. [All of them are listed together](/blog?kind=corrections).

CrawlCheck’s dataset when this page loaded: 582,334 sites measured.

Written by

[Emmanuel Orta](/author) · VSNARY

I build CrawlCheck and write everything published here, from measurements the verification layer took. Where a post got something wrong, the correction stays in the post.

[Every post, and where I got it wrong →](/author) · [Follow on LinkedIn →](https://www.linkedin.com/in/emmanuelorta)

Share this finding: [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fa-write-that-returned-success-and-corrupted-every-newline%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial%26utm_campaign%3Dshare)[X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fa-write-that-returned-success-and-corrupted-every-newline%3Futm_source%3Dx%26utm_medium%3Dsocial%26utm_campaign%3Dshare&text=A%20save%20that%20returned%20success%20rewrote%20every%20newline)[Email](mailto:?subject=A%20save%20that%20returned%20success%20rewrote%20every%20newline&body=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fa-write-that-returned-success-and-corrupted-every-newline)[Follow Emmanuel on LinkedIn](https://www.linkedin.com/in/emmanuelorta)

## Questions this post answers

### How can a write that reports success corrupt a file?

If exact bytes are sent as a text field in a multipart form, the line endings can be converted to CRLF on the way in. The write succeeds, but the stored bytes are not the bytes sent. Sending the content as a file part preserves them.

### How do I verify a saved file or backup?

Read it back and compare a hash such as SHA-256 against the original. A size comparison is not enough, because a change of equal length passes it.

### Why did existing CRLF line endings make it worse?

A converter that rewrites every LF also rewrites the LF inside an existing CRLF, producing a stray carriage return. That breaks deliberate strings in a few scattered places while the file still looks plausible.

### Does an HTTP 200 mean a write worked?

It means the request was received and accepted. It does not prove the stored data matches what was sent, or that it can be read back.

## Related findings

[2026-11-06 Our AI accuracy check ran for six days and measured nothing Our daily AI-accuracy check made no successful Gemini call for six days after Google changed key authorisation, and logged clean r](/blog/our-ai-accuracy-check-ran-for-six-days-and-measured-nothing)[2026-09-16 XML sitemap audit: check the file and every URL in it Audit an XML sitemap in two layers: the file (status, type, parse, robots.txt line), then every URL in it (redirects, noindex, can](/guides/xml-sitemap-audit)[2026-08-13 robots.txt returns 200 but no crawler can read it A robots.txt or llms.txt that answers 200 with a challenge or login page reads as no file. Why monitors miss it, and the two-reque](/blog/a-file-that-answered-200-and-could-not-be-read)

The main product

Found this on your own site? **We fix it for $100.**

Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.

[Fix it all — $100](https://buy.stripe.com/4gM00j7OjawhdLEdyR2VG0v)
[What it covers](/pricing#fix) · [scan first, it is free](/)

Free browser extension

Check this on any site you open, **without leaving the page.**

CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.

[Add to Chrome — free](https://chromewebstore.google.com/detail/plgjengobhbdmbgckiiaaflgiepdbcpi?utm_source=crawlcheck.io&utm_medium=post)
[What it does](/extension)

Write for the library

Measured something yourself? **Submit an article.**

Send your finding, the data and the method. A person reads every submission and replies by email; accepted articles are published under your name with a link to your study. Free to submit. Articles are accepted on merit; no one can pay to have one published.

[Submit an article](/submit-finding#send)
[How it works](/submit-finding) · a reply within about a week

Spotted a mistake or a bug on this page? [Report an error](/report-error?k=article). Confirmed errors are fixed and logged on the [corrections page](/corrections).

## Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed. An AI agent with a published signing key can post without the queue: it signs the request and the signature is checked against its own domain ([how](/api/comment/signed)). Those appear at once, marked *verified*.

Send for review

Writing about this? [Facts, live figures and marks](/press) — every number on that page is dated and traceable to a scan.

[All findings](/blog) · [The dataset](/data) · [How the dataset works](/policy)

We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io)

#### How we measure, and our own machine files

CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io).

**Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing.

**Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours.

**Registry:** [MCP](/registry/mcp) · [AI policy](/registry/aipolicy) · [alanjaychevrolet.com](/registry/alanjaychevrolet.com) · [corriere.it](/registry/corriere.it) · [gasbuddy.com](/registry/gasbuddy.com) | **Pages:** [IP ranges](/data/ip-ranges) · [Keys](/keys) | **Glossary:** [Metadata filtering](/glossary/metadata-filtering) · [Content type](/glossary/content-type) · [Referring domain](/glossary/referring-domain) · [hash-prefix lookup](/glossary/hash-prefix-lookup) · [nosnippet](/glossary/nosnippet)

[Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)

Same body as a file: /blog/a-write-that-returned-success-and-corrupted-every-newline.md · as data: /feed.json · what we tell agents site-wide: /llms.txt, /agents.md. To see any other site this way, from your own browser, after its JavaScript runs: the browser extension.