CrawlCheck

AI crawlers · SEO and research crawlers

Cloudflare-AI-Search-External

Cloudflare-AI-Search-External is the tool or developer fetch run by Cloudflare: Cloudflare AI Search External. Verify the HTTP Message Signature against the operator’s public key directory: https://search.ai.cloudflare.com/external/.well-known/http-message-signatures-directory.

User-agent tokenCloudflare-AI-Search-External
OperatorCloudflare
What it doesCloudflare AI Search External
Kindtool or developer fetch
Can a request be verified?Yes — web bot auth
Verification sourcehttps://search.ai.cloudflare.com/external/.well-known/http-message-signatures-directory
Ready-made robots.txtBlock training and corpus crawlers · Block every AI crawler · Block SEO and ad crawlers · server rules · this page as JSON
In a CrawlCheck scanRead from robots.txt; reported, never scored

Allow or block it in robots.txt

User-agent: Cloudflare-AI-Search-External
Disallow: /
User-agent: Cloudflare-AI-Search-External
Allow: /

Add a group that names it: "User-agent: Cloudflare-AI-Search-External" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names Cloudflare-AI-Search-External, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.

What blocking it changes

No. It is a third-party tool's crawler. Blocking it changes what that tool knows about your site, not anyone's AI answers.

Other agents from Cloudflare

See what Cloudflare-AI-Search-External and every other named agent receive from your own site: run a free scan. The full catalogue is on AI crawlers and as JSON at /ai-crawlers.json.

Questions about Cloudflare-AI-Search-External

QHow do I block Cloudflare-AI-Search-External in robots.txt?
Add a group that names it: "User-agent: Cloudflare-AI-Search-External" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names Cloudflare-AI-Search-External, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.
QHow can I tell if a request from Cloudflare-AI-Search-External is real?
Verify the HTTP Message Signature against the operator’s public key directory: https://search.ai.cloudflare.com/external/.well-known/http-message-signatures-directory.
QDoes blocking Cloudflare-AI-Search-External affect whether my site appears in AI answers?
No. It is a third-party tool's crawler. Blocking it changes what that tool knows about your site, not anyone's AI answers.
QDoes CrawlCheck test my site as Cloudflare-AI-Search-External?
Not as a request. CrawlCheck reads the rules your robots.txt sets for Cloudflare-AI-Search-External and reports them. Rules for it are reported and never scored, because refusing it is a policy choice.