CrawlCheck

AI crawlers · Answer engines

OAI-SearchBot

OAI-SearchBot is the answer-engine crawler run by OpenAI: ChatGPT search index. Its operator publishes the IP ranges it crawls from (https://openai.com/searchbot.json), so a request carrying the name can be checked against that list.

User-agent tokenOAI-SearchBot
OperatorOpenAI
What it doesChatGPT search index
Kindanswer-engine crawler
Can a request be verified?Yes — published ip feed
Verification sourcehttps://openai.com/searchbot.json
In a CrawlCheck scanSent as a request on every scan; a refusal is scored

Allow or block it in robots.txt

User-agent: OAI-SearchBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /

Add a group that names it: "User-agent: OAI-SearchBot" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names OAI-SearchBot, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.

What blocking it changes

Yes. It is how OpenAI reaches a page to answer a question or to build the index its answers read, so refusing it stops that engine reading the site.

What it actually did on crawlcheck.io

Verified requests3,625
Requests using its name from outside its operator's ranges92 (2.5% of claims)
robots.txt fetched221 times
Fetches of paths robots.txt disallows0
Observed2026-08-13 to 2026-10-30

Measured on this site’s own traffic, verified against the operator’s published ranges. Every agent’s record: agent conduct.

Other agents from OpenAI

See what OAI-SearchBot and every other named agent receive from your own site: run a free scan. The full catalogue is on AI crawlers and as JSON at /ai-crawlers.json.

Questions about OAI-SearchBot

QHow do I block OAI-SearchBot in robots.txt?
Add a group that names it: "User-agent: OAI-SearchBot" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names OAI-SearchBot, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.
QHow can I tell if a request from OAI-SearchBot is real?
Its operator publishes the IP ranges it crawls from (https://openai.com/searchbot.json), so a request carrying the name can be checked against that list.
QDoes blocking OAI-SearchBot affect whether my site appears in AI answers?
Yes. It is how OpenAI reaches a page to answer a question or to build the index its answers read, so refusing it stops that engine reading the site.
QDoes CrawlCheck test my site as OAI-SearchBot?
Yes. Every CrawlCheck scan fetches the homepage as OAI-SearchBot and compares the response with what a browser receives. A refusal of it counts toward the grade, because it costs the site visibility.