Content Signals
A robots.txt extension proposed in 2025 in which a site states, per path, whether its content may be used for search, for AI input at answer time, and for AI training, as three separate signals.
What Content Signals means
A robots.txt extension proposed in 2025 in which a site states, per path, whether its content may be used for search, for AI input at answer time, and for AI training, as three separate signals. It is a declaration crawlers may read; nothing enforces it, and a crawler that ignores robots.txt ignores this too.
What Content Signals can and cannot support
| It can support | It cannot support |
|---|---|
| A robots.txt extension proposed in 2025 in which a site states, per path, whether its content may be used for search, for AI input at answer time, and for AI training, as three separate signals. | It is a declaration crawlers may read; nothing enforces it, and a crawler that ignores robots.txt ignores this too. |
Where Content Signals comes up on this site
- Is Cloudflare blocking AI crawlers on your site? What changes on 15 September 2026Enterprise Bot Management customers can declare how bots may use content after crawling it, through a use parameter extending the Content Signals vocabulary in robots.txt.
- How to block AI crawlers, and what each block costs youCloudflare's content-use controls express three levels — Immediate , meaning no storage or reuse; Reference , meaning indexing, excerpts and links back; and Full , meaning summaries or reproduction — with a use parameter extending the Content Signals…
Terms this definition uses
Definitions that name Content Signals
Related terms in Machine files
The files addressed to machines rather than readers, and what publishing one does and does not buy.
llms.txt · llms-full.txt · agents.md · entity map · entity map parity · machine layer · well-known URI · MCP · agent card · sitemap index · OpenAPI specification · ai.txt · Tool definition · SKILL.md · sitemap origin error · soft 404 on a machine file · tdmrep.json · TDM exception · security.txt · HTTP Message Signatures · changefreq and priority · staging leak · host duplicate
Questions about Content Signals
What is Content Signals?
A robots.txt extension proposed in 2025 in which a site states, per path, whether its content may be used for search, for AI input at answer time, and for AI training, as three separate signals.
What does Content Signals not show or guarantee?
It is a declaration crawlers may read; nothing enforces it, and a crawler that ignores robots.txt ignores this too.
Which area of the glossary does Content Signals belong to?
Machine files: The files addressed to machines rather than readers, and what publishing one does and does not buy.
← security.txt · HTTP Message Signatures →
See it in the full glossary · 668 terms across 19 areas.