Agent view · Findings · 2026-08-19
AI visibility vendors audited: 11 of 23 have llms.txt
This is the response this server returns to a client that asks for Accept: text/markdown at https://crawlcheck.io/blog/we-scanned-the-tools-that-audit-your-site: fetched when you opened this page, shown byte for byte. An assistant, a crawler that negotiates markdown, or anything reading our feed gets exactly this. The markdown leaves signed (RFC 9421, key in our key directory), so a crawler can prove the page it read is the page this server sent; the check is shown below.
| status | 200 |
|---|---|
| content-type | text/markdown; charset=utf-8 |
| vary | Accept |
| link | <https://crawlcheck.io/llms.txt>; rel="describedby"; type="text/plain", <https://crawlcheck.io/.well-known/api-catalog>; rel="service-desc"; type="application/linkset+json", <https://crawlcheck.io/blog/we-scanned-the-tools-that-audit-your-site>; rel="cite-as", <https://crawlcheck.io/.well-known/http-message-signatures-directory>; rel="http-message-signatures-directory" |
| cache-control | public, max-age=900, stale-while-revalidate=2400 |
| bytes | 14,714 |
| words | 2,089 |
| sha-256 | 737346c09055d058f895d5fefd8fe5d5b4ee3b812971d0b1e819c3a8818eafea |
| repr-digest | sha-256=:c3NGwJBV0Fj4ldX+/Y/l1bTuO4EpcdCx6BnDqIGOr+o=: ← matches the body shown |
| signature-input | sig1=("@authority";req "@path";req "repr-digest" "content-type");created=1795311494;expires=1795315094;keyid="1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps";alg="ed25519";nonce="tR4eol2Ay4sw-cYjTJKCVA";tag="crawlcheck-page" |
| signature | sig1=:iQ9KlRt3e5dCKZKUNTbE3KJetgjZ9GCXjxx08V8kFjYSUoY5dY1KHL0Hx2iWHlk95S4Es7MckTJFmmlLzepqDw==: |
| verified | yes — signature by key 1hHmE3fo1rg5Ig70… from https://crawlcheck.io/.well-known/http-message-signatures-directory (directory valid), valid for 3600 more s. Checked here exactly as a crawler would; redo it yourself: https://crawlcheck.io/api/tool/wba |
Exclusive finding · Findings · 2026-08-19 · By [VSNARY | Emmanuel Orta](/author) · 5 views # AI visibility vendors audited: 11 of 23 have llms.txt The companies that grade machine readability, run through the same checks they would point at you. The reading is from a directory that re-scans itself, so today's figures are one click away. On 9 September 2026, the CrawlCheck directory listed 25 SEO and AI-visibility tools and graded 23 of them. Eleven of the 23 published an llms.txt and none published an entity map. Scores ran from 44 to 84, with a median of 78, and two vendors whose edge refused the scan are listed as refused rather than graded. An earlier one-off run of 33 vendor sites in August found the same pattern: roughly a third had an llms.txt, none had an entity map. CrawlCheck exclusive finding Sample the websites of 33 SEO and AI-visibility vendors, run through the same checks a customer gets Measured 19 August 2026, re-measured on a rolling basis What it established The tools that sell AI visibility scored 41 to 92 on their own sites, a third published an llms.txt, none an entity map, and several served a stale machine file. One of [23 findings measured only here](/blog?kind=exclusive). Every figure comes from CrawlCheck’s own readings, dated, with the method at the foot. Have your own? [Submit a finding](/submit-finding). On this page - What has not moved? - How close are the grades? - Open door, little to quote - What the August run found - How to read these figures fairly Every tool in this list sells a judgement about other people's websites. So we pointed CrawlCheck at them — the same crawler-identity, machine-file, cache and render checks a customer gets — and kept the results in [the directory](/directory), which re-scans each site on a rolling basis and keeps a dated ledger of what moved. The figures below are its reading on 9 September 2026; open the directory for today's. **Disclosure:** CrawlCheck publishes this piece and sells a tool in the same category. crawlcheck.io is not in its own corpus, because a scanner cannot grade its own host from the outside view it uses for everyone else; check our files from any client you like. Machine files on the sites of SEO and AI-visibility vendors, directory reading of 9 September 2026. | | Directory, 2026-09-09 | First run, 2026-08-19 | | --- | --- | --- | | Vendors | 25 listed, 23 graded | 33 scanned | | Publish an [llms.txt](/glossary/llms-txt) | 11 of 23 | roughly a third | | Publish an [entity map](/glossary/entity-map) | 0 of 23 | 0 | | Highest score | 84 (Sitebulb) | 92 | | Lowest completed scan | 44 | 41 | | Median | 78 | not published | | Refused CrawlCheck | 2, listed as refused and not graded | 1 | The two columns are different populations. The directory is curated to tools that sell a judgement about other sites; the August run was every vendor domain the crawl touched. Read them as two readings of a similar group, not one group changing over time. **Live, as you read this:** the corpus now holds 582,334 domains across 9,306 scans. The figures in this piece were measured on the date above; this line is not. ## What has not moved? # The entity-map count. Zero of twenty-three, weeks after the first run, on the sites of companies whose product includes telling you to publish [structured data](/glossary/structured-data) about yourself. That is not a lag. If you are weighing whether these files matter, see [does llms.txt work](/guides/does-llms-txt-actually-work) and [what entity SEO is](/guides/what-is-entity-seo). The refusals are now a category rather than an anecdote. A vendor whose edge challenges datacentre traffic is listed as *refused CrawlCheck — not graded*, with no number, because a number would be our [vantage point](/glossary/vantage-point) dressed up as their defect. The first time we met one of these, [we graded the firewall instead of the website](/blog/the-site-that-scored-15-because-it-refused-us). ## How close are the grades? # Close. Twenty of the twenty-three graded sites landed between 62 and 84, a few points apart, on a test each of them would put a customer through. The outliers say more than the middle: a site crawler at 44 with nine findings, and an enterprise SEO platform with zero findings and a 65, which is what a clean scan of a page that says almost nothing looks like. ## Open door, little to quote # Access and substance do not move together, and where they split is the useful part. Two AI-visibility monitors let every crawler in and have almost nothing on the page an [answer engine](/glossary/answer-engine) could repeat: no stated number, no dated claim, no answered question. A site crawler lets crawlers in, but most of what arrives is markup and script rather than words. A single blended score hides both, which is why each directory card shows the evidence behind the number. ## What the August run found # The first run covered 33 vendor sites on 19 August 2026. The best scored 92 of 100 and the worst that completed a scan scored 41. The median site delivered under one visible word in twenty bytes of payload; the rest was markup and script, from companies that sell payload advice. Several served a cached machine file that no longer matched what their own server answered, so a crawler and a human could be told two different things. One site answered every path identically, including one that cannot exist: a firewall speaking, not a website. ## How to read these figures fairly # Each is a snapshot of the day the scan ran. A vendor who fixes their [machine layer](/glossary/machine-layer) tomorrow deserves the better number, and the directory ledger records the change with its date. That is the point of keeping it as a page rather than a spreadsheet: a stale accusation is unfair to the vendor who fixed the thing. The industry that grades machine readability is, on its own machine layer, about average. The average is low. CrawlCheck’s dataset when this page loaded: 582,334 sites measured. Written by [Emmanuel Orta](/author) · VSNARY I build CrawlCheck and write everything published here, from measurements the verification layer took. Where a post got something wrong, the correction stays in the post. [Every post, and where I got it wrong →](/author) · [Follow on LinkedIn →](https://www.linkedin.com/in/emmanuelorta) Share this finding: [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fwe-scanned-the-tools-that-audit-your-site%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial%26utm_campaign%3Dshare)[X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fwe-scanned-the-tools-that-audit-your-site%3Futm_source%3Dx%26utm_medium%3Dsocial%26utm_campaign%3Dshare&text=AI%20visibility%20vendors%20audited%3A%2011%20of%2023%20have%20llms.txt)[Email](mailto:?subject=AI%20visibility%20vendors%20audited%3A%2011%20of%2023%20have%20llms.txt&body=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fwe-scanned-the-tools-that-audit-your-site)[Follow Emmanuel on LinkedIn](https://www.linkedin.com/in/emmanuelorta) ## Questions this post answers ### Do AI-visibility vendors follow their own advice? Mostly not on entity maps, and only about half on llms.txt. In the CrawlCheck directory reading, no graded vendor site published an entity map, and fewer than half published an llms.txt. ### Is it a fair comparison? Each vendor gets the same checks from the same client, no vendor is told in advance, and a site whose edge refuses the scan is listed as refused rather than given a score. The figures are dated snapshots and the directory re-scans them. ### Why is a vendor listed as refused instead of graded? Because when a firewall challenges datacentre traffic, the scan measures the firewall, not the website. Scoring that would present CrawlCheck's vantage point as the vendor's defect. ### Is CrawlCheck itself in the directory? No. A scanner cannot grade its own host from the same outside view it uses for other sites, so crawlcheck.io is left out and its machine files are open for anyone to check directly. ### What does the result say about the checks? That they are not specialist requirements only an expert can meet. They are cheap to implement, and the companies that sell the advice have mostly not implemented all of it. ## Related findings [2026-09-30 NAP consistency: how to check your name, address and phone Five NAP checks, from a missing map point to a directory printing another business name, how to run each by hand, and why a refuse](/guides/nap-consistency-audit-phone-address-and-coordinates)[2026-09-01 Seven site types, and the report checks we corrected We ran seven very different sites through every graded check and found some failing sites for non-defects. Which checks changed, a](/blog/we-read-every-scored-row-against-seven-sites-here-is-what-we-unscored)[2026-10-06 State of AI visibility, October 2026: 139,788 homepages 139,788 homepages measured 3–6 October 2026: 6.3% block an AI crawler, 15.4% publish llms.txt, the median homepage is 3.6% text. L](/blog/state-of-ai-visibility-october-2026-139788-homepages) Where this is used Guides that cite this finding: - [AI visibility tools compared: what each one measures](/guides/ai-visibility-tools-what-each-one-measures) - [Does llms.txt work? What is known and how to test it](/guides/does-llms-txt-actually-work) - [GEO vs AEO vs LLM SEO: the terms, and what is measurable](/guides/geo-aeo-llm-seo-what-the-terms-mean) The main product Found this on your own site? **We fix it for $100.** Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after. [Fix it all — $100](https://buy.stripe.com/4gM00j7OjawhdLEdyR2VG0v) [What it covers](/pricing#fix) · [scan first, it is free](/) Free browser extension Check this on any site you open, **without leaving the page.** CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account. [Add to Chrome — free](https://chromewebstore.google.com/detail/plgjengobhbdmbgckiiaaflgiepdbcpi?utm_source=crawlcheck.io&utm_medium=post) [What it does](/extension) Write for the library Measured something yourself? **Submit an article.** Send your finding, the data and the method. A person reads every submission and replies by email; accepted articles are published under your name with a link to your study. Free to submit. Articles are accepted on merit; no one can pay to have one published. [Submit an article](/submit-finding#send) [How it works](/submit-finding) · a reply within about a week Spotted a mistake or a bug on this page? [Report an error](/report-error?k=article). Confirmed errors are fixed and logged on the [corrections page](/corrections). ## Comments Comments are read before they appear. Nothing is published automatically, and no account is needed. An AI agent with a published signing key can post without the queue: it signs the request and the signature is checked against its own domain ([how](/api/comment/signed)). Those appear at once, marked *verified*. Send for review Writing about this? [Facts, live figures and marks](/press) — every number on that page is dated and traceable to a scan. [All findings](/blog) · [The dataset](/data) · [How the dataset works](/policy) We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io) #### How we measure, and our own machine files CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io). **Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing. **Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours. **Pages:** [Certificates](/certificates) · [Listing check](/listing-check) · [Removal](/removal) | **Registry:** [MCP: Operator](/registry/mcp/operator) · [MCP](/registry/mcp) · [AI policy](/registry/aipolicy) · [woorank.com](/registry/woorank.com) · [agent-exec.448c.org](/registry/capabilities/agent-exec.448c.org) | **Datasets:** [West Virginia](/datasets/state/west-virginia) · [Technologies: Wp rocket](/datasets/technologies/wp-rocket) | **Guides:** [Localbusiness schema for service area businesses](/guides/localbusiness-schema-for-service-area-businesses) | **Glossary:** [AI classifiers: Answer justification](/glossary/ai-classifiers/answer-justification) [Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)
Same body as a file: /blog/we-scanned-the-tools-that-audit-your-site.md · as data: /feed.json · what we tell agents site-wide: /llms.txt, /agents.md. To see any other site this way, from your own browser, after its JavaScript runs: the browser extension.