CrawlCheck
Data · crawler addresses

When AI and search operators change the addresses their crawlers use.

Operators publish the IP ranges their crawlers come from, so a site can tell the real crawler from an impostor. They change those lists without notice. CrawlCheck re-reads 18 published feeds every day and keeps every change, so a block list, a firewall rule or a log audit can be checked against the list as it stood on the day.

Compare your crawler allowlist

Choose one operator feed and paste the CIDR entries for that feed, one per line. The comparison runs in your browser. Prefix splits and merges can change the entries while covering the same addresses. Review firewall changes against the official source.


The feeds

OperatorFeedPrefixes nowChanges seenLast change
OpenAI · GPTBotopenai.com/gptbot.json180—
OpenAI · OAI-SearchBotopenai.com/searchbot.json390—
OpenAI · ChatGPT-Useropenai.com/chatgpt-user.json23722026-11-06
Anthropic · ClaudeBot, Claude-User, Claude-SearchBotclaude.com/crawling/bots.json140—
Perplexity · PerplexityBotwww.perplexity.ai/perplexitybot.json1312026-10-28
Perplexity · Perplexity-Userwww.perplexity.ai/perplexity-user.json1012026-10-28
Google · common crawlersdevelopers.google.com/static/crawling/ipranges/common-crawlers.json3170—
Google · user-triggered fetchers (Google-owned)developers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json5040—
Google · user-triggered agentsdevelopers.google.com/static/crawling/ipranges/user-triggered-agents.json200—
Google · special-case crawlersdevelopers.google.com/static/crawling/ipranges/special-crawlers.json2720—
Microsoft · Bingbotwww.bing.com/toolbox/bingbot.json2912026-11-05
Apple · Applebotsearch.developer.apple.com/applebot.json240—
DuckDuckGo · DuckDuckBotduckduckgo.com/duckduckbot.json4860—
Mistral · MistralAI-Usermistral.ai/mistralai-user-ips.json40—
Ahrefs · AhrefsBotapi.ahrefs.com/v3/public/crawler-ips10,3500—
Amazon · Amazonbotdeveloper.amazon.com/amazonbot/ip-addresses/1,2920—
Common Crawl · CCBotindex.commoncrawl.org/ccbot.json50—
Amazon · Amzn-SearchBotdeveloper.amazon.com/amazonbot/searchbot-ip-addresses/8160—

Not tracked on purpose: Google’s user-triggered-fetchers.json lists Google Cloud addresses that any customer can run code from, so a match there does not show the request came from Google. Google’s general goog.json covers Google services that are not crawlers.

Changes

Traffic evidence reports two hourly counts. Operator-authenticated hours identify the organization. Crawler-authenticated hours also verify the crawler name carried in the signed request. Coverage shows sampled feed checks and hours with authenticated traffic from still-listed ranges. Each evidence link includes a closeout summary, updated while its observation window is open.

DayOperatorPrefixesAdded / removed
2026-11-06OpenAI · ChatGPT-User235 → 237+2 / −0
2026-11-05Microsoft · Bingbot28 → 29+1 / −0
2026-10-28Perplexity · PerplexityBot8 → 13+9 / −4 · Traffic evidence
2026-10-28Perplexity · Perplexity-User4 → 10+8 / −2 · Traffic evidence
2026-10-27OpenAI · ChatGPT-User234 → 235+1 / −0

The full record, with the prefixes added and removed, is at /data/ip-ranges.json (CC BY 4.0). Verify a log line against today’s lists with the free crawler check.