Library · Corrections
Corrections
In one sentence
Findings are short case studies written from real measurements — a robots.txt that answered 200 with a challenge page, a site whose GPTBot traffic came from one address wearing seven crawler names, a homepage that was two percent readable text, a contact email that bounced for one published character — each with the bytes that showed it; a business found with a defect is named only after its owner has been contacted.
Where CrawlCheck's own readings, reports or pipeline were wrong, and the fix.
2026-11-02 · Corrections13 views · 2 min
Our receipt checker would have rejected our own receipts
A flaw in our published offline verifiers would have called genuine CrawlCheck receipts forged after our first routine key change. Our tests caught it before any receipt was affected.
Read the measurements →
2026-11-02 · Corrections4 views · 2 min
For a week, CrawlCheck told AI agents not to pay us
Our own site made the kind of change a hijacker makes first. Until we confirmed it, agents asking to pay crawlcheck.io were stopped at the payment step.
Read the measurements →
2026-11-06 · Corrections3 views · 3 min
Most of our 'referring pages' were forged Referer headers
The owner panel that lists who links to a site was reporting its own IP addresses, parked lookalike domains and a radiology college. None of them had linked to anything. How a referrer is forged, and the rule that now counts one.
Read the measurements →
2026-11-02 · Corrections2 views · 2 min
Three of our own checks passed because we measured nothing
Three CrawlCheck checks passed on sites where there was nothing to measure, a homepage counter would have stopped counting, and a paid branding option did nothing. All fixed.
Read the measurements →
2026-11-13 · Corrections1 view · 2 min
Claude-User uses two published address lists. We checked one
Anthropic publishes a crawler address list and a separate list of outbound addresses. Claude-User can arrive from either. We compared against the first only, and filed real requests as failed.
Read the measurements →
2026-11-06 · Corrections1 view · 3 min
Our crawler counts missed about 45% of hits
Every published crawler hit count before 6 November 2026 is too low. Concurrent writes to one counter dropped nearly half of all hits. Here is the measurement, the cause and the fix.
Read the measurements →
2026-11-06 · Corrections1 view · 3 min
Our AI accuracy check ran for six days and measured nothing
The daily job that asks Gemini what it tells customers about each business got a 403 on every call from 31 October to 6 November 2026. The job reported success. What broke, why nothing alerted, and what a wrong answer looked like when it came back.
Read the measurements →
2026-11-02 · Corrections1 view · 4 min
Lazy-load placeholders: why image checks miss your photos
A lazy-load plugin puts a placeholder in src and the real URL one attribute over. Tools that read src see no photographs. Ours did: 14 placeholders, zero WebP files, on a page of 13 WebP photos.
Read the measurements →
2026-11-02 · Corrections1 view · 2 min
OpenTimestamps proof says pending? Upgrade the receipt
Every timestamp file this site served was the calendar's first receipt, not the finished Bitcoin proof. The anchoring was sound the whole time; the files did not show it.
Read the measurements →
2026-11-02 · Corrections1 view · 2 min
Our audit labelled one page and measured another for weeks
Every scan of a specific URL reported the homepage's figures. The report header showed the right page; the numbers belonged to another.
Read the measurements →
2026-11-02 · Corrections1 view · 2 min
Dead link or blind spot? When a checker's failure is its own
A site's schema pointed at a live profile page. Our check could not reach it and reported it dead. The failure was ours, and the report now says so.
Read the measurements →
2026-11-02 · Corrections1 view · 2 min
Unblocking AI crawlers: what an access repair proves
Twelve synthetic robots.txt cases show that the agent’s name decides which rules apply. They do not show access, citations or referrals, and this note says so.
Read the measurements →
2026-11-19 · Corrections6 min
Who reads llms.txt? Our check called 8 of 10 unverified
Our llms.txt reader check marked 8 of 10 readers 'unverified' and counted two of our own fetches as a confirmed AI assistant. Here is what changed and what the readers were.
Read the measurements →
2026-11-13 · Corrections3 min
14 of 43 crawlers in our catalogue were classed wrong
We counted a block on 43 crawlers against a site. On 13 November 2026 we read each one against its operator's own page. Fourteen were in the wrong class, and nine more were judged by the wrong rule.
Read the measurements →
2026-11-13 · Corrections2 min
Our mirror relayed real crawlers and inflated forgery counts
A copy of this site that we ran ourselves passed crawler requests back to us under the crawler's own name. Our verifier counted them as failed checks. The counts were ours to get right.
Read the measurements →
2026-11-13 · Corrections2 min
Both our SDKs cut a tool list at 300 with no warning
A fingerprint of an MCP server's tools is only useful if it covers all of them. Ours stopped at 300 and did not say so.
Read the measurements →
2026-11-13 · Corrections2 min
Booking links counted as missing sitemap pages
Our index alerts said five small sites had hundreds of new pages Google had never fetched. The sites had 15 to 55 pages each. The rest were booking links.
Read the measurements →
2026-11-02 · Corrections2 min
Unquoted HTML attributes: why we said a viewport was missing
A report scored one of our sites 0 out of 100 on mobile while its viewport tag sat in the page. The attributes were unquoted, which is valid HTML. What changed, and how to check yours.
Read the measurements →
2026-11-02 · Corrections2 min
Why a web server can gain and lose 43 million sites
Netcraft counts tens of millions of OpenResty sites; W3Techs does not list it. Both measure honestly, and the gap is a lesson in reading denominators.
Read the measurements →
2026-11-02 · Corrections2 min
A save that returned success rewrote every newline
The write reported success. Read back, the file was 16,330 bytes larger, exactly the number of newlines it contained. A store accepting your bytes is not the same as keeping them.
Read the measurements →
2026-11-02 · Corrections2 min
Analytics added 650 ms to every request: how we found it
Our own field data said 70% of the homepage's LCP was time to first byte. The server was recording each visit before it started answering. Moving that work after the response cut TTFB to 85 ms.
Read the measurements →
2026-11-02 · Corrections6 min
Is your host blocking ClaudeBot? Group the 403s by IP first
A 57% block rate in our own access logs looked like a hosting provider shutting out AI crawlers. Grouped by source address, almost all of it was one machine wearing seven different crawler names.
Read the measurements →
2026-11-02 · Corrections4 min
Is your auditor grading your firewall? A two-request test
We once published a failing grade for a competitor that measured only its firewall. The random-path test that would have caught it, and the curl commands to run it on your own site.
Read the measurements →
2026-11-02 · Corrections2 min
When an LLM summarises your audit report and gets it wrong
An AI read one of our reports and confidently explained a grade penalty we had not applied. The report left the reason out, so the mistake was ours.
Read the measurements →
2026-11-02 · Corrections2 min
Why caching only successes kept our scans slow
Scans of small business sites ran slow and varied from run to run, because we kept asking a question whose answer was always no. Our first fix made one case worse.
Read the measurements →
2026-11-02 · Corrections2 min
A placeholder check flagged NaN. It was inside a URL
Our check for leaked placeholder text flagged a national newspaper's record three times. All three hits were letters inside one URL slug.
Read the measurements →
2026-11-02 · Corrections3 min
Seven site types, and the report checks we corrected
Some report checks failed well-built sites for things that are not defects, and a few counted one fault twice. We read every graded check against seven very different sites and fixed what was wrong.
Read the measurements →
2026-11-02 · Corrections2 min
Correction: our paywall said sites with findings were clean
For about forty minutes, a locked report told unpaid readers there was nothing to fix, and its spreadsheet export said there were no findings. Hiding findings was intended; claiming there were none was the bug.
Read the measurements →
2026-11-02 · Corrections2 min
A 200 status hid an empty page for weeks
One of our own internal pages returned HTTP 200 while showing a single apology sentence instead of its content. Every check we ran passed, because none of them read the page.
Read the measurements →
2026-11-02 · Corrections2 min
Sites measured is not people: what our own logs showed
The sites-measured figure on our homepage is true, and it is not demand. One week of our own request logs shows who visits a site about crawlers: mostly crawlers.
Read the measurements →
2026-11-02 · Corrections2 min
Correction: the owner's own scans came back locked
For a time the site owner's scans came back gated like a stranger's while the report page showed everything. No score or customer was affected. The lesson is about testing gated content.
Read the measurements →
2026-11-02 · Corrections2 min
Five days of our alerts went to an address with no mailbox
After a mail migration, the address our alerts were sent to had no mailbox behind it. Every message was marked sent and none arrived, for five days.
Read the measurements →
2026-11-02 · Corrections2 min
Our security headers skipped every signed-in session
A new Content-Security-Policy and tighter CORS headers were checked live and called done. Signed-in sessions received neither. The check had only ever looked signed out.
Read the measurements →
2026-11-02 · Corrections2 min
Our extension read a page as 36.5% JavaScript. It was zero.
The CrawlCheck browser extension overstated how much of a page needed JavaScript, for two reasons that both pushed the number up. Version 1.0.2 fixed both.
Read the measurements →
2026-11-02 · Corrections2 min
Eight contradictions in our own copy, rechecked
An outside audit listed eight places where crawlcheck.io contradicted itself. Rechecked against the live site: which were real, which were two truths read as one, and the one still wrong.
Read the measurements →
2026-11-02 · Corrections3 min
Our directory froze for 20 hours and logged no error
For about 20 hours on 13 and 14 October 2026 the index behind our directory of SEO and AI tools did not rebuild, and eight cards opened reports that no longer existed. Nothing reported an error.
Read the measurements →
2026-11-02 · Corrections2 min
An access rule on one path locked our customers out
On 14 October 2026 an access rule meant for staff-only pages also covered customer sign-in, which shared the same address. For an hour, the Sign in link on 247 pages led to a login customers could not pass.
Read the measurements →
2026-11-02 · Corrections2 min
Three sections said “not scored”. Our grade counted them.
For part of 22 October 2026, three new report sections were labelled “measured, not scored” but still counted in the grade. Rescan if your report is from that window.
Read the measurements →
2026-11-02 · Corrections4 min
One 503 is not a blocked site: refusal or hiccup?
A test scan read one 503 from a site's machine files and called the origin blocked. The same files answered 200 to six requests straight after. What a single refusal means, what a repeated one means, and how to tell them apart from your own desk.
Read the measurements →
2026-11-02 · Corrections3 min
AI answer corrections: three clocks, not one update time
An earlier edition of this note read two archived observations as if they timed a correction in AI answers. They do not. This is what they show, and the log that would.
Read the measurements →