published crawler ranges
The list of IP address ranges an operator publishes for a crawler, so that a request claiming the crawler's name can be checked against where it came from. OpenAI, Anthropic, Google, Microsoft, Apple and Perplexity publish such lists; many operators do not.
What published crawler ranges does not show
It can prove a request came from the operator's own network; it cannot prove anything about an agent whose operator publishes nothing, which is a limit of the evidence rather than a judgement.
Where published crawler ranges comes up on this site
- How much AI crawler traffic is forged? We check the IPsGoogle’s hostname rule for Googlebot passes rented Google Cloud servers ; check the address against the published crawler ranges as well.
Related terms in Crawlers and access
Who is fetching, whether they are who they claim, and what your rules actually permit.
visitor resolve · operator origin · signed agent · agent arrival · agent conduct record · agent traffic index · user-triggered fetch exemption · retrieval crawler · user-triggered fetch · verified crawler · forged crawler identity · unverifiable · robots.txt · user-agent group · AI opt-out · Content-Signal · crawl budget · cloaking · challenge page at 200 · uniform refusal · nonexistent-path control · blocked render resource · off-host redirect · homepage refused · FCrDNS · Crawler trap · Conditional request · ASN blocking · operator feed · residential proxy · Google-Extended · Google-Agent · GPTBot vs OAI-SearchBot · ClaudeBot vs Claude-User · PerplexityBot vs Perplexity-User · CCBot · Bytespider · agentic traffic · crawler classification · user agent · datacenter IP · vantage point · rate limit · delivery comparison · honeypot
Questions about published crawler ranges
What is published crawler ranges?
The list of IP address ranges an operator publishes for a crawler, so that a request claiming the crawler's name can be checked against where it came from. OpenAI, Anthropic, Google, Microsoft, Apple and Perplexity publish such lists; many operators do not.
What does published crawler ranges not show or guarantee?
OpenAI, Anthropic, Google, Microsoft, Apple and Perplexity publish such lists; many operators do not. It can prove a request came from the operator's own network; it cannot prove anything about an agent whose operator publishes nothing, which is a limit of the evidence rather than a judgement.
Which area of the glossary does published crawler ranges belong to?
Crawlers and access: Who is fetching, whether they are who they claim, and what your rules actually permit.
← visitor resolve · operator origin →
See it in the full glossary · 792 terms across 19 areas.
Free browser extension
See this on any site you visit, in one click.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.