Agent view · Findings · 2026-08-18
When an LLM summarises your audit report and gets it wrong
This is the response this server returns to a client that asks for Accept: text/markdown at https://crawlcheck.io/blog/the-grade-that-could-not-explain-itself: fetched when you opened this page, shown byte for byte. An assistant, a crawler that negotiates markdown, or anything reading our feed gets exactly this. The markdown leaves signed (RFC 9421, key in our key directory), so a crawler can prove the page it read is the page this server sent; the check is shown below.
| status | 200 |
|---|---|
| content-type | text/markdown; charset=utf-8 |
| vary | Accept |
| link | <https://crawlcheck.io/llms.txt>; rel="describedby"; type="text/plain", <https://crawlcheck.io/.well-known/api-catalog>; rel="service-desc"; type="application/linkset+json", <https://crawlcheck.io/blog/the-grade-that-could-not-explain-itself>; rel="cite-as", <https://crawlcheck.io/.well-known/http-message-signatures-directory>; rel="http-message-signatures-directory" |
| cache-control | public, max-age=900, stale-while-revalidate=2400 |
| bytes | 11,674 |
| words | 1,635 |
| sha-256 | d61e6f6358358884343720f9cf8a654baf368ed5528962db9a29fa43d909975d |
| repr-digest | sha-256=:1h5vY1g1iIQ0NyD5z4plS682jtVSiWLbmin6Q9kJl10=: ← matches the body shown |
| signature-input | sig1=("@authority";req "@path";req "repr-digest" "content-type");created=1795311487;expires=1795315087;keyid="1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps";alg="ed25519";nonce="UCT6Tx5dstg1zAzJQL2JJg";tag="crawlcheck-page" |
| signature | sig1=:kUYr1wm5O55NzgTk3i88D8EdWBBWOQSi6Oq12rpjIkqUzeQzOOzjeY/zxERC9b04Ud+Pjt+193/q1LTQgWOeAw==: |
| verified | yes — signature by key 1hHmE3fo1rg5Ig70… from https://crawlcheck.io/.well-known/http-message-signatures-directory (directory valid), valid for 3600 more s. Checked here exactly as a crawler would; redo it yourself: https://crawlcheck.io/api/tool/wba |
Findings · 2026-08-18 · By [VSNARY | Emmanuel Orta](/author) # When an LLM summarises your audit report and gets it wrong An AI read one of our reports and confidently explained a grade penalty we had not applied. The report left the reason out, so the mistake was ours. Someone asked a language model to review a CrawlCheck report. It concluded the grade was lowered for things the report admitted it could not measure. That was wrong: the grade was set by one real finding, a homepage served from a cache copy nearly four hours old. But the report never said what set the grade, so the model filled the gap with a plausible guess. Reports now name the finding behind the grade and say that unmeasured sections did not affect it. Test your own reports the same way. On this page - What actually set the grade? - Why was the model's reading still our fault? - What changed? - How do you test your own reports? What a report shows, and what it means for your grade Measured, passed What it means for your grade Counts in full Measured, failed What it means for your grade Lowers the score, and raises a finding if serious enough Not measured What it means for your grade Named in the report, and does not lower the grade Someone ran one of our reports through a language model for a second opinion. The site scored 90 out of 100, sat at the 86th percentile of our corpus, and still graded C. The model concluded the letter was being lowered for categories the report said it could not measure, such as speed data with no field measurements. That would be a bad design: it would punish a site for the tool's own gaps. ## What actually set the grade? # One finding of medium severity: the homepage was being served from a cache copy nearly four hours old. The unmeasured sections had no effect. You can check the cache problem on your own site with our guide to [stale cached files behind the edge](/guides/machine-file-caching-stale-split-and-blocked-behind-the-edge). ## Why was the model's reading still our fault? # The report showed the grade and the list of unmeasured sections side by side, and never said which caused what. Placing them together implied a link. A careful reader, human or machine, followed it. For a product built on legible evidence, that is a defect in the output, not in the reader. | What a report shows | What it means for your grade | | --- | --- | | Measured, passed | Counts in full | | Measured, failed | Lowers the score, and raises a finding if serious enough | | Not measured | Named in the report, and does not lower the grade | ## What changed? # Reports now say which finding set the grade, and state that unmeasured sections did not affect it. The count of scored sections shown on the site is read from the scorer itself rather than typed by hand, so the two cannot drift apart. A check whose input never arrived is now shown differently from one that ran and found nothing. The [report guide](/guides/how-to-read-a-crawlcheck-report) walks through each part. ## How do you test your own reports? # Reports are increasingly read by machines, and a machine does not get the context you would add in a meeting. Hand one of your reports to a model and ask it to explain the headline number. If it invents a plausible cause, your report implied one. Fix it by putting the relationship in the report itself: say which input changed which number. The same applies to [facts on your own website](/blog/the-two-facts-almost-nobody-publishes): if it is not stated, an assistant will guess. CrawlCheck’s dataset when this page loaded: 582,334 sites measured. Written by [Emmanuel Orta](/author) · VSNARY I build CrawlCheck and write everything published here, from measurements the verification layer took. Where a post got something wrong, the correction stays in the post. [Every post, and where I got it wrong →](/author) · [Follow on LinkedIn →](https://www.linkedin.com/in/emmanuelorta) Share this finding: [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fthe-grade-that-could-not-explain-itself%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial%26utm_campaign%3Dshare)[X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fthe-grade-that-could-not-explain-itself%3Futm_source%3Dx%26utm_medium%3Dsocial%26utm_campaign%3Dshare&text=When%20an%20LLM%20summarises%20your%20audit%20report%20and%20gets%20it%20wrong)[Email](mailto:?subject=When%20an%20LLM%20summarises%20your%20audit%20report%20and%20gets%20it%20wrong&body=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fthe-grade-that-could-not-explain-itself)[Follow Emmanuel on LinkedIn](https://www.linkedin.com/in/emmanuelorta) ## Questions this post answers ### What should a score always carry? The input that produced it. If a grade is lowered, the finding that lowered it should be named next to it, so the number can be checked rather than trusted. ### Does an unmeasured section lower a CrawlCheck grade? No. Sections that could not be measured are named in the report and do not lower the grade. ### Why did the language model get the grade wrong? The report placed the grade next to the list of unmeasured sections without saying which caused what, so the model inferred a link that did not exist. ### How can I test whether my reports are clear to machines? Give a report to a language model and ask it to explain the headline number. If its explanation is wrong but plausible, the report is missing the link between the number and its cause. ## Related findings [2026-09-17 Correction: our paywall said sites with findings were clean For about forty minutes our locked reports said “Nothing to fix” for sites that had findings. What went wrong, what changed, and h](/blog/the-paywall-that-said-your-site-was-clean)[2026-09-08 Unquoted HTML attributes: why we said a viewport was missing Minified HTML with unquoted attributes read as having no viewport, canonical or description. All three were there. What was wrong](/blog/the-viewport-was-there-all-along)[2026-08-19 Is your auditor grading your firewall? A two-request test A site scored 15 of 100 because its firewall refused CrawlCheck. How to tell whether an audit measured your website or your WAF,](/blog/the-site-that-scored-15-because-it-refused-us) Where this is used Guides that cite this finding: - [Free AI visibility checkers: how to tell if one is any good](/guides/free-ai-visibility-checkers-what-they-measure) - [What is entity SEO? What machines must resolve to cite you](/guides/what-is-entity-seo) The main product Found this on your own site? **We fix it for $100.** Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after. [Fix it all — $100](https://buy.stripe.com/4gM00j7OjawhdLEdyR2VG0v) [What it covers](/pricing#fix) · [scan first, it is free](/) Free browser extension Check this on any site you open, **without leaving the page.** CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account. [Add to Chrome — free](https://chromewebstore.google.com/detail/plgjengobhbdmbgckiiaaflgiepdbcpi?utm_source=crawlcheck.io&utm_medium=post) [What it does](/extension) Write for the library Measured something yourself? **Submit an article.** Send your finding, the data and the method. A person reads every submission and replies by email; accepted articles are published under your name with a link to your study. Free to submit. Articles are accepted on merit; no one can pay to have one published. [Submit an article](/submit-finding#send) [How it works](/submit-finding) · a reply within about a week Spotted a mistake or a bug on this page? [Report an error](/report-error?k=article). Confirmed errors are fixed and logged on the [corrections page](/corrections). ## Comments Comments are read before they appear. Nothing is published automatically, and no account is needed. An AI agent with a published signing key can post without the queue: it signs the request and the signature is checked against its own domain ([how](/api/comment/signed)). Those appear at once, marked *verified*. Send for review Writing about this? [Facts, live figures and marks](/press) — every number on that page is dated and traceable to a scan. [All findings](/blog) · [The dataset](/data) · [How the dataset works](/policy) We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io) #### How we measure, and our own machine files CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io). **Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing. **Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours. **Pages:** [/](/) · [Listing check](/listing-check) · [SDK](/sdk) · [Certificates](/certificates) · [Website audit](/website-audit) | **Registry:** [lakewoodplumbingpros.com](/registry/lakewoodplumbingpros.com) · [rosstree.net](/registry/rosstree.net) · [Trade: Fencing](/registry/trade/fencing) · [exa.ai](/registry/capabilities/exa.ai) | **Datasets:** [Technologies: Hubspot](/datasets/technologies/hubspot) · [Technologies: Powr](/datasets/technologies/powr) | **Guides:** [HTTP 3 how to check whether your site offers it](/guides/http-3-how-to-check-whether-your-site-offers-it) [Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)
Same body as a file: /blog/the-grade-that-could-not-explain-itself.md · as data: /feed.json · what we tell agents site-wide: /llms.txt, /agents.md. To see any other site this way, from your own browser, after its JavaScript runs: the browser extension.