CrawlCheck

AI crawlers · Training crawlers

Webzio-Extended

Webzio-Extended is the training crawler run by Webz.io: Webz.io extended. Its operator publishes no IP ranges or DNS names for it, so a request carrying this name cannot be verified. Treat the name in a log line as a claim.

User-agent tokenWebzio-Extended
OperatorWebz.io
What it doesWebz.io extended
Kindtraining crawler
Can a request be verified?No — user agent string only
In a CrawlCheck scanRead from robots.txt; reported, never scored

Allow or block it in robots.txt

User-agent: Webzio-Extended
Disallow: /
User-agent: Webzio-Extended
Allow: /

Add a group that names it: "User-agent: Webzio-Extended" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names Webzio-Extended, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.

What blocking it changes

No. It collects pages for training. Refusing it is a policy choice about training, separate from whether answer engines can reach, read and quote the site.

Other agents from Webz.io

  • omgili — Webz.io omgili
  • Omgilibot — collects pages for Webz.io's web data feeds, which are sold for AI training

See what Webzio-Extended and every other named agent receive from your own site: run a free scan. The full catalogue is on AI crawlers and as JSON at /ai-crawlers.json.

Questions about Webzio-Extended

QHow do I block Webzio-Extended in robots.txt?
Add a group that names it: "User-agent: Webzio-Extended" followed by "Disallow: /". A crawler follows only the most specific group that names it, so once a group names Webzio-Extended, rules under "User-agent: *" no longer apply to it; repeat in that group any Disallow lines you still want.
QHow can I tell if a request from Webzio-Extended is real?
Its operator publishes no IP ranges or DNS names for it, so a request carrying this name cannot be verified. Treat the name in a log line as a claim.
QDoes blocking Webzio-Extended affect whether my site appears in AI answers?
No. It collects pages for training. Refusing it is a policy choice about training, separate from whether answer engines can reach, read and quote the site.
QDoes CrawlCheck test my site as Webzio-Extended?
Not as a request. CrawlCheck reads the rules your robots.txt sets for Webzio-Extended and reports them. Rules for it are reported and never scored, because refusing it is a policy choice.