AI crawlers · Search indexes
YandexBot
YandexBot is the search index crawler run by Yandex: indexes pages for Yandex search. Its operator documents how to verify it: reverse DNS to yandex.ru, yandex.net or yandex.com, then forward DNS back to the same address. A reverse DNS lookup on the source address must return the operator's domain, and a forward lookup on that name must return the same address.
| User-agent token | YandexBot |
| Operator | Yandex |
| What it does | indexes pages for Yandex search |
| Kind | search index crawler |
| Can a request be verified? | Yes — reverse dns |
| Verification source | reverse DNS to yandex.ru, yandex.net or yandex.com, then forward DNS back to the same address |
| In a CrawlCheck scan | Read from robots.txt; a refusal is scored |
Allow or block it in robots.txt
User-agent: YandexBot Disallow: /
User-agent: YandexBot Allow: /
Add a group that names it: "User-agent: YandexBot" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names YandexBot, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.
What blocking it changes
Indirectly. It feeds indexes pages for Yandex search. A refusal removes pages from that index, and most AI answer surfaces still draw on search indexes.
What it actually did on crawlcheck.io
| Verified requests | 10,323 |
| Requests using its name from outside its operator's ranges | 19 (0.2% of claims) |
| robots.txt fetched | 616 times |
| Fetches of paths robots.txt disallows | 0 |
| Observed | 2026-08-19 to 2026-11-12 |
Measured on this site’s own traffic, verified against the operator’s published ranges. Every agent’s record: agent conduct.
Other agents from Yandex
Yandex— Yandex (umbrella token)
See what YandexBot and every other named agent receive from your own site: run a free scan. The full catalogue is on AI crawlers and as JSON at /ai-crawlers.json.