CrawlCheck

AI crawlers · Search indexes

PetalBot

PetalBot is the search index crawler run by Huawei: Huawei Petal. Its operator documents how to verify it: reverse DNS to petalsearch.com, then forward DNS back to the same address. A reverse DNS lookup on the source address must return the operator's domain, and a forward lookup on that name must return the same address.

User-agent tokenPetalBot
OperatorHuawei
What it doesHuawei Petal
Kindsearch index crawler
Can a request be verified?Yes — reverse dns
Verification sourcereverse DNS to petalsearch.com, then forward DNS back to the same address
In a CrawlCheck scanRead from robots.txt; a refusal is scored

Allow or block it in robots.txt

User-agent: PetalBot
Disallow: /
User-agent: PetalBot
Allow: /

Add a group that names it: "User-agent: PetalBot" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names PetalBot, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.

What blocking it changes

Indirectly. It feeds Huawei Petal. A refusal removes pages from that index, and most AI answer surfaces still draw on search indexes.

Other agents from Huawei

See what PetalBot and every other named agent receive from your own site: run a free scan. The full catalogue is on AI crawlers and as JSON at /ai-crawlers.json.

Questions about PetalBot

QHow do I block PetalBot in robots.txt?
Add a group that names it: "User-agent: PetalBot" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names PetalBot, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.
QHow can I tell if a request from PetalBot is real?
Its operator documents how to verify it: reverse DNS to petalsearch.com, then forward DNS back to the same address. A reverse DNS lookup on the source address must return the operator's domain, and a forward lookup on that name must return the same address.
QDoes blocking PetalBot affect whether my site appears in AI answers?
Indirectly. It feeds Huawei Petal. A refusal removes pages from that index, and most AI answer surfaces still draw on search indexes.
QDoes CrawlCheck test my site as PetalBot?
Not as a request. CrawlCheck reads the rules your robots.txt sets for PetalBot and reports them. A refusal of it counts toward the grade, because it costs the site visibility.