Google-Agent

A user-triggered fetcher Google added to its documentation on 20 March 2026 for agents that run on Google infrastructure and browse on a person's behalf, Project Mariner among them. It carries a Chrome-like user-agent string with the token inside it and publishes its own range file, user-triggered-agents.json, separate from the other fetcher feeds. Google classes it as a user proxy, so it generally ignores robots.txt. It is the concrete case where the caveat on user-triggered fetch stops being theoretical: a rule addressed to it is a request the fetcher is documented not to read.

Terms this definition uses

user-triggered fetch · token · robots.txt

Crawlers and access

Who is fetching, whether they are who they claim, and what your rules actually permit.

retrieval crawler · user-triggered fetch · verified crawler · forged crawler identity · unverifiable · robots.txt · user-agent group · AI opt-out · Content-Signal · crawl budget · cloaking · challenge page at 200 · uniform refusal · nonexistent-path control · blocked render resource · off-host redirect · homepage refused · FCrDNS · Crawler trap · Conditional request · ASN blocking · operator feed · residential proxy · Google-Extended · GPTBot vs OAI-SearchBot · ClaudeBot vs Claude-User · PerplexityBot vs Perplexity-User · CCBot · Bytespider · agentic traffic · crawler classification

← Google-Extended · GPTBot vs OAI-SearchBot →

See it in the full glossary · 579 terms across 19 areas. Scan a site to see which of these apply to it.