CrawlCheck

Findings · 2026-10-08 · By · 0 views

Who signs their agent requests? Web Bot Auth in October 2026

User agents can be typed by anyone and IP ranges do not fit agents that run in shared browsers. Signed requests do. Which operators document signing today, the identity each one signs as, and which large operators still rely on address ranges.

As of 8 October 2026, operators documenting Web Bot Auth signatures include OpenAI (ChatGPT’s cloud browser signs as https://chatgpt.com), Google (experimenting with https://agent.bot.goog for Google-Agent), Cloudflare (Browser Rendering) and the first signed-agent cohort Cloudflare announced in August 2025: ChatGPT agent, Block’s Goose, Browserbase and Anchor Browser. Anthropic and Perplexity document published IP ranges for their crawlers and fetchers but no request signing.

A user agent is a claim anyone can type: on this site 86% of Perplexity-User claims were forged. Address ranges fix that for crawlers, which run from their operator’s own machines. They fit agents badly, because an agent working for a person often runs in a shared cloud browser whose addresses also serve other customers. Web Bot Auth is the answer the industry is converging on: the agent signs each request, names the domain that holds its keys, and the site checks the signature against those keys.

How a signed agent request is verifiedAgent requestSignature-Agent · SignatureKey directory/.well-known/http-message-…Your site verifieskey matches · not expiredWEB BOT AUTH IN ONE LINEA valid signature proves the request came from whoever holds the key in that directory. It says nothing about what the agent will do next.
The site fetches the public keys from the domain named in Signature-Agent and checks the signature. No shared secret, no IP list.

Who documents signing today #

Each row below comes from the operator’s own documentation or announcement, read on 8 October 2026.

OperatorAgentSigns asSource
OpenAIChatGPT’s cloud browser (ChatGPT Work)https://chatgpt.com; keys at chatgpt.com/.well-known/http-message-signatures-directoryOpenAI Help Center, agent allowlisting
GoogleGoogle-AgentExperimenting with https://agent.bot.goog; also publishes user-triggered-agents.json rangesGoogle crawler documentation
CloudflareBrowser RenderingSigned since Web Bot Auth launchedCloudflare, “The age of agents”, 28 Aug 2025
Block, Browserbase, Anchor BrowserGoose, and two remote-browser platformsFirst cohort of Cloudflare’s signed-agent category, with ChatGPT agentSame announcement
CrawlCheckOur own crawlerEvery request signed; directory at /.well-known/http-message-signatures-directoryThis site

Who documents ranges instead #

OperatorWhat it documentsSigning documented?
AnthropicClaudeBot, Claude-User and Claude-SearchBot by user agent; ranges at claude.com/crawling/bots.jsonNo
PerplexityPerplexityBot and Perplexity-User by user agent; ranges at perplexity.com/perplexitybot.json and perplexity-user.jsonNo

Neither is a weakness for the crawlers, which run from those ranges and verify cleanly: ClaudeBot and PerplexityBot are forged in under 2% of claims here. It matters for agents. As assistants move from fetching a page to operating a browser for a person, a range file stops describing where the traffic comes from, and a signature is the only identity that travels with the request.

What a signature proves, and what it does not #

  • It proves the request came from whoever holds the private key published in the named directory, within the signature’s time window.
  • It does not prove what the agent will do with the page, whether it will respect your robots.txt, or that the person behind it is who they say.
  • It replaces the user agent as the thing to allow or block on, for operators that sign.

If you allow a signed agent, allow it by the identity in Signature-Agent after verifying the signature, never by the user agent string next to it. The Web Bot Auth guide shows the verification step by step.

Method #

Operators were included only where their own documentation or announcement states that they sign, and with the identity they state. Anthropic’s and Perplexity’s crawler pages were read for any mention of Web Bot Auth, HTTP message signatures or Signature-Agent and contain none. This table will be re-read and dated when an operator changes its documentation.

Every figure above came out of CrawlCheck.

Point it at your own domain and see the same measurements, free.

Scan a domain — free

The main product

Found this on your own site? We fix it for $749.

Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.

Questions this post answers

Which AI agents use Web Bot Auth?

As of 8 October 2026: ChatGPT’s cloud browser (signs as https://chatgpt.com), Google-Agent (Google is experimenting with https://agent.bot.goog), Cloudflare Browser Rendering, and the agents in Cloudflare’s first signed-agent cohort: ChatGPT agent, Block’s Goose, Browserbase and Anchor Browser.

Does Claude or Perplexity sign its requests?

Not according to their documentation as read on 8 October 2026. Anthropic and Perplexity identify their crawlers and fetchers by user agent and publish IP ranges (claude.com/crawling/bots.json; perplexity.com/perplexitybot.json and perplexity-user.json).

What does a Web Bot Auth signature prove?

That the request came from whoever holds the private key published in the directory named by the Signature-Agent header, within the signature’s validity window. It does not prove how the agent will behave.

Related findings

How anything measured in this article was measured15client identitiesone second, one address5machine filesapex and www114named agentsresolved from robots.txt24sections scoredreach, read, quoteHow anything measured here was measured15 client identities5 machine files114 named agents24 sections scoredone second, one addressapex and wwwresolved from robots.txtreach, read, quote
No account, nothing installed, and the same sequence on every domain — which is what makes one scan comparable to another. Run it on your own site.

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works