Findings · 2026-09-12 · By VSNARY | Emmanuel Orta · 1 view

1,582 requests arrived under a name Google says it never sends

Google-Extended is a robots.txt control token. Google's own documentation says it has no HTTP user-agent string. Over seven days, 1,582 requests reached our zones carrying it anyway.

Between 5 and 11 September 2026 we read Cloudflare's own bot-verification verdict on every request that reached sixteen zones on one account. Not a guess from a user-agent string, and not an outside-in probe: the verdict the network that served the request already recorded. Two counts per crawler name, over the same requests, in the same window - how many Cloudflare verified, and how many carried the same name and were not verified.

Eleven names came back with a rate worth publishing. Five came back having never been verified once. One of those five should not exist as a user-agent at all.

The name that is not a crawler #

Google-Extended is the robots.txt token that governs whether content Google has already crawled may be used to train and ground Gemini. It is a policy control, not a fetcher. Google's crawler documentation is explicit about it: Google-Extended does not have a separate HTTP request user agent string, crawling is done with existing Google user agent strings, and the robots.txt token is used in a control capacity.

So there is nothing to verify, because nothing is supposed to arrive. Over the window, 1,582 requests reached these zones carrying Google-Extended in the user-agent. Whatever sent them, it was not Google using a documented identity, because Google documents that this identity is never sent.

That is a different statement from the one our page was making. A rate of 0% verified reads as an accusation about an operator's fleet. The true statement here is narrower and stronger: a name that its owner says never appears on the wire appeared 1,582 times.

What the other four mean, which is not the same thing #

PerplexityBot arrived 4,893 times and was verified zero times. That is not evidence that 4,893 requests were forged. Cloudflare removed Perplexity from its Verified Bots programme in August 2025, naming PerplexityBot and Perplexity-User as the identifiers. Once a crawler is outside the programme there is no verification path at all, so every request under that name lands unverified by construction - a fact about the reference data, not about the requests.

CCBot and Claude-SearchBot are in the same position from where we stand: never verified once, across sixteen zones and seven days. We have no external evidence about either, and we are not going to invent any. What we can say is that we have never seen a verification succeed for those names here, which is exactly the sentence the page now prints instead of a percentage.

The rates that are real #

For the eleven names where verification demonstrably works, the spread is wide and it does not line up with how people talk about these crawlers.

Even Googlebot, the most verifiable crawler on the internet, sits at 51.5%. Roughly half the requests arriving at these zones under Googlebot's name are not Googlebot by Cloudflare's reckoning. That is the number most site owners would be most surprised by, and it is the one we very nearly failed to publish at all.

Our own figures were publishing a false zero #

Before today, the public figures counted the two sides differently: unverified requests for every crawler we list, verified requests only for the crawlers Cloudflare files as AI.

Dividing one into the other produced this: 19 of 30 rows read 0% verified, and Googlebot was one of them. Cloudflare files Googlebot as a search engine crawler, not as AI, so not one of its 5,284 verified requests was counted. The same went for bingbot, AhrefsBot, SemrushBot, MJ12bot, Baiduspider and YandexBot - every one of them verified thousands of times, every one of them published at zero.

Nobody was reading the endpoint yet, which is the only reason this is a note rather than an apology. It is still the worst kind of defect we ship: a number that looks measured, on a page whose entire proposition is that the numbers are measured. We have done this to ourselves before, in the other direction, and counted our own traffic as forged.

A rate needs a verification path to exist #

Counting both sides the same way was the easy half. The harder half is that 0% says two opposite things and cannot tell you which. Either the crawler is verifiable and not one request passed, or there is no verification path for that name and nothing could ever pass. Printed as a percentage, a reader takes the first reading every time.

So a name we have never seen verified, across sixteen zones and seven days, no longer gets a percentage. Those five rows now carry the counts and the reason, alongside the operator’s own published statement where one exists. We do not know that Cloudflare cannot verify CCBot; we know we have not seen it happen. That distinction is the same one every unscored row on a report rests on.

What this does not claim #

These are sixteen zones on one account, measured because we operate them, not a sample of the web. Unverified is not forged: a legitimate crawler behind a proxy, or one whose ranges Cloudflare has not adopted, lands in the same bucket as an impostor. The unverified side is read from the most common user-agent strings per zone per day, so a rare impostor on a busy zone can fall outside the list.

What survives all of that is narrow and worth saying anyway. Half the traffic arriving as Googlebot is not Googlebot. And 1,582 requests introduced themselves with a name that the company that owns it says it never sends. If you want to see the current figures rather than these, they are on the crawler reference, recomputed from the same records.

Related findings

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works