lab2.crawlcheck.io
Fetched 2026-11-06. Graded D, 73 of 100, 15 findings.
llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable
Is this your site? Claim this record — free, by DNS TXT or a meta tag.
Get these 15 findings fixed
Every defect this row counts, corrected on the site itself and then re-measured from the outside — done is measured, not asserted. One-time, $100 for one local-business site, thirty days of monitoring included.
Get this fixed — $100Claiming is free and changes nothing but the owner mark. What a licence adds is the record behind this entry: re-measured twice a day, finding age on every defect, change receipts between any two dates. Keep it on the daily record.
What the fetch found
| Finding | Severity |
|---|---|
| A named crawler was served materially less text than an ordinary client | 4 of 5 |
| Crawlers are given scripts, links or instructions that visitors never see | 4 of 5 |
| A child sitemap named in the sitemap index does not load | 3 of 5 |
| robots.txt has a group for a misspelt crawler name | 3 of 5 |
| The homepage depends on files this site's own robots.txt disallows | 3 of 5 |
| A sitemap lastmod is not a valid date | 2 of 5 |
| The sitemap lists URLs on another host | 2 of 5 |
| robots.txt keeps Google's image crawler off the site's images | 2 of 5 |
| llms.txt links to pages that do not load | 2 of 5 |
| robots.txt declares a sitemap with a relative path | 2 of 5 |
| robots.txt uses a Noindex line no crawler obeys | 2 of 5 |
| This domain is on the browser HSTS preload list but the header no longer qualifies | 2 of 5 |
Section scores on this fetch
Weakest first. 8 of 21 scored sections at 90 or above.
| Section | Score |
|---|---|
| RDF and linked-data surface | 33 |
| Quotable content | 33 |
| What each agent receives | 38 |
| Image signals | 40 |
| Schema and entity graph | 63 |
| AI and search agents | 67 |
| Machine-file trust chain | 71 |
| Server and hosting | 75 |
| Authorship and trust signals (E-E-A-T proxies) | 75 |
| Error pages | 75 |
| Sitemap coverage | 80 |
| Machine layer | 83 |
| Accessibility structure (from the delivered HTML) | 89 |
| Crawl waste | 100 |
| Speed a crawler sees, and field data where it exists | 100 |
| Payload and render path | 100 |
| Agent instruction surface | 100 |
| Mobile surface | 100 |
| Exposed files and secrets | 100 |
| Internal linking | 100 |
| Naming and case consistency | 100 |
Shelves
| Shelf | State | Definition |
|---|---|---|
| llms.txt | on the shelf | What counts, and who has it |
| agents.md | not found | What counts, and who has it |
| Media kit | not found | What counts, and who has it |
| Entity graph | not found | What counts, and who has it |
| AI access policy | on the shelf | What counts, and who has it |
| Agent-callable | not found | What counts, and who has it |
The full report for this fetch
Machine-readable: /api/registry?host=lab2.crawlcheck.io. This row states what was served to CrawlCheck at that time. It is not an endorsement, and it says nothing this scan did not measure.