AI opt-out
Any published instruction asking that a site's content not be used for model training.
What AI opt-out does not show
It is a request rather than a control, and whether it is honoured varies by operator.
How CrawlCheck measures AI opt-out
One finding code measures it. Its share of all scans is computed from the live corpus when this page is served, not typed into it.
| Finding code | Share of scans | Guide |
|---|---|---|
AI_OPTOUT_SET | 2.5% | Is Cloudflare blocking AI crawlers on your site? |
Every code, its rule and its current rate: the dataset page. Scan a site to see which apply to it.
Where AI opt-out comes up on this site
- Is Cloudflare blocking AI crawlers on your site?Across the public dataset, robots.txt allows an answer engine while the edge refuses it on of scans, and an AI opt-out is already set on .
- How to rank in ChatGPT: the part that is in your controlThe opposite trap is an AI opt-out in robots.txt that nobody on the team chose — a copied block or a plugin default.
- How to block AI crawlers like GPTBot and ClaudeBotAn AI opt-out of some kind is set on of scans here.
- Agent readiness checkers: Is It Agent Ready? vs CrawlCheckAn AI opt-out is already set: .
Related terms in Crawlers and access
Who is fetching, whether they are who they claim, and what your rules actually permit.
visitor resolve · published crawler ranges · operator origin · signed agent · agent arrival · agent conduct record · agent traffic index · user-triggered fetch exemption · retrieval crawler · user-triggered fetch · verified crawler · forged crawler identity · unverifiable · robots.txt · user-agent group · Content-Signal · crawl budget · cloaking · challenge page at 200 · uniform refusal · nonexistent-path control · blocked render resource · off-host redirect · homepage refused · FCrDNS · Crawler trap · Conditional request · ASN blocking · operator feed · residential proxy · Google-Extended · Google-Agent · GPTBot vs OAI-SearchBot · ClaudeBot vs Claude-User · PerplexityBot vs Perplexity-User · CCBot · Bytespider · agentic traffic · crawler classification · user agent · datacenter IP · vantage point · rate limit · delivery comparison · honeypot
Questions about AI opt-out
What is AI opt-out?
Any published instruction asking that a site's content not be used for model training.
What does AI opt-out not show or guarantee?
Any published instruction asking that a site's content not be used for model training. It is a request rather than a control, and whether it is honoured varies by operator.
How does CrawlCheck measure AI opt-out?
Through the finding code AI_OPTOUT_SET, each with a stated rule, evidence bytes and a live share of scans on the dataset page.
Which area of the glossary does AI opt-out belong to?
Crawlers and access: Who is fetching, whether they are who they claim, and what your rules actually permit.
← user-agent group · Content-Signal →
See it in the full glossary · 772 terms across 19 areas.
Free browser extension
See this on any site you visit, in one click.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.