ai.txt
A proposed file for declaring how AI systems may use a site content, in the same family as llms.txt and Content Signals.
What ai.txt does not show
Informal proposal, not an enforced standard, and adoption is low enough that absence tells you nothing.
How CrawlCheck measures ai.txt
One finding code measures it. Its share of all scans is computed from the live corpus when this page is served, not typed into it.
| Finding code | Share of scans | Guide |
|---|---|---|
NO_LLMS_TXT | 32.4% | Does llms.txt work? What is known and how to test it |
Every code, its rule and its current rate: the dataset page. Scan a site to see which apply to it.
Terms this definition uses
Definitions that name ai.txt
tdmrep.json · TDM exception · security.txt · per-use rights
Related terms in Machine files
The files addressed to machines rather than readers, and what publishing one does and does not buy.
llms.txt · llms-full.txt · agents.md · entity map · entity map parity · machine layer · well-known URI · MCP · agent card · sitemap index · OpenAPI specification · Tool definition · SKILL.md · sitemap origin error · soft 404 on a machine file · tdmrep.json · TDM exception · security.txt · Content Signals · HTTP Message Signatures · changefreq and priority · staging leak · host duplicate
Questions about ai.txt
What is ai.txt?
A proposed file for declaring how AI systems may use a site content, in the same family as llms.txt and Content Signals.
What does ai.txt not show or guarantee?
Informal proposal, not an enforced standard, and adoption is low enough that absence tells you nothing.
How does CrawlCheck measure ai.txt?
Through the finding code NO_LLMS_TXT, each with a stated rule, evidence bytes and a live share of scans on the dataset page.
Which area of the glossary does ai.txt belong to?
Machine files: The files addressed to machines rather than readers, and what publishing one does and does not buy.
← OpenAPI specification · Tool definition →
See it in the full glossary · 772 terms across 19 areas.
Free browser extension
See this on any site you visit, in one click.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.