CrawlCheck
llms.txt audit · free, no signup

llms.txt audit: does yours resolve, parse and match the site it describes?

In one sentence

An llms.txt audit checks whether the file at /llms.txt answers 200 to every crawler identity with a text content type, parses, and cites only pages the site actually declares and serves — a file that answers 200 with a login wall or an HTML shell counts as absent.

Type a domain. The scan fetches /llms.txt on the apex and www hosts as fifteen client identities, records the status, content type, cache age and bytes each one received, and checks the file against the pages the site actually declares.

No signup. Nothing installed. Nothing changed on your site. The same scan as the homepage - about 30 read-only requests - and the same report, opened at the sections that answer this question.

How often the machine files fail
No llms.txt at allNO_LLMS_TXT31.4%A cache serves a stale copySTALE_CACHE_SERVED16.1%Machine files cost more than the pageMACHINE_CHAIN_HEAVY17.6%Machine file stale at the edgeMACHINE_FILE_CACHE_STALE0.6%Share of 4,317 scans in the public dataset that carried the finding. Live; the same figures as /data.

Is your site one of them? Check it free → All 63 rates →

Measured

What the scan measures for this

The file itself

Does /llms.txt answer 200 to every identity, with a text content type, from a cache that is not stale? A file that answers 200 with a login wall or an HTML shell is scored as unreadable, not present.

See this section on a real report: Machine files →

The surfaces around it

llms.txt, agents.md, the entity map and the sitemap are read together. A file that cites pages the sitemap does not declare, or that the site redirects, is a claim the site does not support.

See this section on a real report: Agent surfaces →

Instruction surfaces

What the file tells an agent to do, whether those instructions came from you or from a template, and whether any of them contradict robots.txt.

See this section on a real report: Instructions for agents →

What an agent is served

The page an AI client receives, compared with the page a browser receives. llms.txt cannot compensate for a homepage that ships as a script shell.

See this section on a real report: What each crawler gets →

For agents

What an agent is told about the same site

An agent does not read the report. It asks one endpoint before it reads, cites, connects or transacts, and gets a signed answer built from the same measurement.

1 · See it

The agent view

The answer an agent receives about any domain, rendered for a person: each action’s decision and the reason.

Agent view

2 · Call it

One endpoint

GET /api/v1/resolve?domain= returns the signed answer; preflight returns a decision and a receipt for one action.

QuickstartResolve spec

3 · Check it

Without trusting us

Every answer is Ed25519-signed and logged; it verifies offline with the published key.

Signing keysProof

Workflow

The workflow

  1. Scan the domainThe free scan fetches the file and every surface around it. Nothing is installed and nothing changes on the site.
  2. Open the Sections tab and read Machine files and SurfacesEach row names what was requested, what came back, and the bytes that decide the row.
  3. Draft or correct the fileIf there is no file, the free draft below builds one from the pages the site already publishes. If there is one, the fix list names the lines that fail.
  4. Publish, purge the cache, re-scanDone is measured, not asserted. The second scan is the proof, and on a watched domain it becomes the first point of the record.
Free

Free tools for this

Each one is a single request to the domain you name, answered as text or a download. Put the domain on the end of the URL, or use the form on the tools page.

Reading

Read next

Start from what you are checking

You cannot pay to improve a score. A licence adds the lists behind the counts and the record over time - never a better grade.

Questions about this page

QDoes an llms.txt improve AI visibility on its own?
No. It is one machine surface among several, and the scan weights the homepage a crawler actually receives far above it. A site with a perfect llms.txt and a script-shell homepage still fails the Read stage.
QWhat if my site has no llms.txt?
The scan records it as absent, not as a defect that caps the grade. The free draft tool builds one from your homepage and declared pages; the value of the file is choosing which pages to keep, so cut it down before publishing.
QWhy does the scan fetch it as fifteen identities?
Because a file served differently to GPTBot than to a browser is the defect that matters. Each identity's status, bytes and content type are recorded separately, and any divergence is a finding with the evidence bytes attached.
QIs this the same as the homepage scan?
Yes. It is the same free scan and the same report; this page opens it at the sections that answer the llms.txt question. Nothing here is a separate score.