AI crawlers · Answer engines
Claude-User
Claude-User is the user-triggered fetch run by Anthropic: Claude live fetch. Its operator publishes the IP ranges it crawls from (https://claude.com/crawling/bots.json), so a request carrying the name can be checked against that list.
| User-agent token | Claude-User |
| Operator | Anthropic |
| What it does | Claude live fetch |
| Kind | user-triggered fetch |
| Can a request be verified? | Yes — published ip feed |
| Verification source | https://claude.com/crawling/bots.json |
| In a CrawlCheck scan | Read from robots.txt; a refusal is scored |
Allow or block it in robots.txt
User-agent: Claude-User Disallow: /
User-agent: Claude-User Allow: /
Add a group that names it: "User-agent: Claude-User" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names Claude-User, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.
What blocking it changes
Yes. It is how Anthropic reaches a page to answer a question at the moment a person asks, so refusing it stops that engine reading the site.
What it actually did on crawlcheck.io
| Verified requests | 97 |
| Requests using its name from outside its operator's ranges | 54 (35.8% of claims) |
| robots.txt fetched | 14 times |
| Fetches of paths robots.txt disallows | 1 |
| Observed | 2026-08-12 to 2026-10-29 |
Measured on this site’s own traffic, verified against the operator’s published ranges. Every agent’s record: agent conduct.
Other agents from Anthropic
Claude-SearchBot— Claude search indexClaudeBot— Anthropic traininganthropic-ai— Anthropic legacy agentClaude-Web— Anthropic legacy agent
See what Claude-User and every other named agent receive from your own site: run a free scan. The full catalogue is on AI crawlers and as JSON at /ai-crawlers.json.