Exclusive finding · Findings · 2026-10-29 · By VSNARY | Emmanuel Orta

Google's Googlebot check passes 198 of 200 rented servers

Google's verification rule accepts reverse DNS under googleusercontent.com, the domain every Google Cloud customer machine gets. In a 200-address sample, 198 passed, and two open-source verifiers accepted a rented server as Googlebot.

Disclosure: CrawlCheck publishes this piece, and its own verifier is one of those tested below.

The most-copied instruction for telling a real Googlebot from an impostor comes from Google's own page, Verifying Google crawlers and fetchers. Take the address from your log, look up its reverse DNS name, check that the name is under googlebot.com, google.com or googleusercontent.com, then look the name up forwards and confirm it returns the same address. Google says this is enough for most needs. It is enough for Google's two crawler categories. It is not enough for the third name on that list, and the third name is the one anyone can rent.

What googleusercontent.com actually is #

googleusercontent.com is the domain Google assigns to content and machines run by its customers. Every virtual machine on Google Cloud with a public IPv4 address gets a reverse DNS record of the form d.c.b.a.bc.googleusercontent.com, set by Google, and that name resolves forwards to the same address. The two-step check therefore passes for the machine by construction. Nobody has to spoof a DNS record; Google publishes the record for them.

Google's page does say which reverse name each of its crawler types uses: common crawlers such as Googlebot appear as crawl-*.googlebot.com or geo-crawl-*.geo.googlebot.com, special-case crawlers as rate-limited-proxy-*.google.com, and user-triggered fetchers as *.gae.googleusercontent.com or google-proxy-*.google.com. The gae.googleusercontent.com entry is Google App Engine, where customers run their own code. A verifier that keeps the per-type mask is safe. A verifier that keeps only the three-domain rule, which is how the rule is usually copied into code and into guides (ours included, until today), is not.

How many addresses pass #

Google publishes its customer address ranges at gstatic.com/ipranges/cloud.json. The copy dated 29 October 2026 13:07 UTC lists 1,040 IPv4 prefixes covering 19,369,344 addresses, plus 95 IPv6 prefixes. We drew 200 addresses at random from the IPv4 ranges, weighted by prefix size so every address had the same chance, and ran Google's two lookups on each one over DNS over HTTPS.

99% of the sample passes. Scaled to the published ranges, that is roughly 19 million addresses a site would accept as “Google” under the three-domain rule, available by the hour to anyone with a card. The 198 all had names of the same shape, d.c.b.a.bc.googleusercontent.com; none was a crawl- or geo-crawl- name, which is what Googlebot's own addresses carry.

Who accepts the rented machine #

A rule only matters where it runs, so we tested the verifiers we could find in the open. We took two Googlebot-verification packages published on npm, ran each in its default configuration, and asked each the same question: is 34.123.45.67, a Google Cloud customer address presenting the standard Googlebot user agent, really Googlebot?

Package A's failure is the rule copied faithfully. Package B's is a different mistake with the same result: treating every file on Google's page as a Googlebot list, when one of the five is explicitly a list of customer egress. We have not named the two packages in this post. The reproduction below is enough for any maintainer to check their own.

Why it matters #

Sites treat verified Googlebot differently from everyone else. It is allowed through rate limits, bot-fight rules and paywalls; it is served the full page where visitors get a challenge; it is the one identity almost no firewall rule blocks. A scraper that reverse-resolves to googleusercontent.com inherits all of that from any verifier built on the three-domain rule. The user-triggered fetchers, which Google says ignore robots.txt, inherit it from any verifier that treats the customer range file as Googlebot.

Forged crawler traffic is not hypothetical; we measured how much of one site's claimed crawler traffic came from outside the published ranges in How much AI crawler traffic is forged?. What was missing was a check on the check itself.

The fix #

1. Match the address against Google's common-crawlers.json first (for Googlebot, GoogleOther and Google-InspectionTool) and special-crawlers.json for the special-case crawlers. Refresh daily; Google rebuilt all five files on 29 October 2026.
2. If you fall back to reverse DNS, require the per-type name, not the three-domain list: Googlebot is crawl-*.googlebot.com or geo-crawl-*.geo.googlebot.com, nothing else.
3. Never treat googleusercontent.com as Google-owned. It is the customer domain.
4. Never treat user-triggered-fetchers.json as a crawler list. It is customer egress. user-triggered-fetchers-google.json and user-triggered-agents.json are Google's own fetchers and agents, which is still not Googlebot: they act for a person and ignore robots.txt.

What we changed #

Until 29 October 2026, CrawlCheck's own Googlebot check accepted the same three names when it fell back to DNS, so a rented machine could pass on the rare occasions Google's range files could not be read. Rare is not never. It now accepts googlebot.com and google.com only, and the same day we corrected six passages in our own guides that taught the three-domain rule. A real Googlebot address still verifies; a Google Cloud customer address claiming Googlebot now reads spoofed. You can run the same check on any address with the free Googlebot checker, which also names Google's crawler type and whether it follows robots.txt.

Reproduce it #

$ host 34.123.45.67
67.45.123.34.in-addr.arpa domain name pointer 67.45.123.34.bc.googleusercontent.com.
$ host 67.45.123.34.bc.googleusercontent.com
67.45.123.34.bc.googleusercontent.com has address 34.123.45.67

$ host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.

The first pair passes Google's two-step rule and is a customer machine. The second is Googlebot. The only thing that separates them is the name, and the rule as commonly written does not look at it. The same range-first approach for Bingbot and the AI crawlers is in how to verify Googlebot, Bingbot and AI crawlers by IP address.

Method #

Sample: 200 IPv4 addresses drawn with a fixed seed from the 1,040 prefixes in cloud.json (creationTime 2026-10-29T13:07:07), each prefix weighted by its size. Lookups: PTR, then A, over Cloudflare's DNS-over-HTTPS resolver, on 29 October 2026 between 19:30 and 19:40 MDT; 0 lookup errors. A pass is a PTR name ending in googleusercontent.com whose A record includes the sampled address. Verifier tests: each package installed from npm at its current version and run with no options, given the IP and the Googlebot user-agent string; Package A's DNS calls were answered by the same resolver. Nothing here required a Google account, a Google Cloud account or any request to a third party's site.

Questions this post answers

Is reverse DNS to googleusercontent.com proof a request is from Google?

No. googleusercontent.com is the domain Google assigns to its customers' machines. Any Google Cloud virtual machine gets a reverse name under it that resolves back to the same address, so it passes the two-step DNS check while being an ordinary rented server.

What reverse DNS name does the real Googlebot use?

Googlebot reverse-resolves to crawl-*.googlebot.com or geo-crawl-*.geo.googlebot.com, and that name resolves forward to the same address. A Googlebot user agent from any other name should not be trusted as Googlebot.

Is user-triggered-fetchers.json a list of Googlebot addresses?

No. Google describes that file as fetches made on behalf of Google Cloud customers. Treating it as a Googlebot range list lets a customer's application be verified as Googlebot. Google's own fetchers and agents are in separate files, and they act for a person rather than crawling.

What is the safest way to verify Googlebot?

Match the address against Google's common-crawlers.json range file first, refreshed daily. If you fall back to DNS, require a googlebot.com name for Googlebot, confirm the forward lookup returns the same address, and never accept googleusercontent.com.

Why does a false Googlebot verification matter?

Sites let verified Googlebot through rate limits, bot rules and paywalls. A scraper that passes as Googlebot inherits all of that access, so a verifier that accepts customer machines opens the door it was built to guard.

Can I reproduce this test myself?

Yes. Pick an address from Google Cloud's published customer ranges, run a reverse lookup, then a forward lookup on the name it returns. If the name ends in googleusercontent.com and resolves back, a three-domain verifier would accept it as Google.

Related findings

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works