[Glossary](/glossary) · [Crawlers and access](/glossary/area/crawlers-and-access)

# user agent

The string a client sends naming itself, in the User-Agent request header. It is a claim, not an identity: anyone can send any string, so treating a crawler's name as proof of the crawler is the mistake behind most forged-crawler traffic.

## What user agent means

The name is what a site filters on; the source address is what verifies it.

## Where user agent comes up on this site

- [Google-Extended has no user agent. 1,582 requests used it](/blog/a-name-google-says-it-never-sends)Google's crawler documentation is explicit about it: Google-Extended does not have a separate HTTP request user agent string, crawling is done with existing Google user agent strings, and the robots.txt token is used in a control capacity.
- [How to rank in ChatGPT: the part that is in your control](/guides/how-to-rank-in-chatgpt)Check it: request your homepage once with a browser user agent and once with GPTBot's, and compare status and size.
- [How to block AI crawlers like GPTBot and ClaudeBot](/guides/how-to-block-ai-crawlers)The AI crawler list has every user agent and its purpose. robots.txt a request honoured by operators that choose to Network rule a refusal enforced on verified identity, not the name Content-use signal a statement of intent what may be done after the fetch…
- [HTTP status codes for crawlers: what a 200 can hide](/guides/http-status-codes-what-a-crawler-actually-gets)Set ua to your own browser's user-agent string first (search "what is my user agent" and paste it).
- [How an AI agent should verify a website before it acts](/guides/how-an-ai-agent-verifies-a-website-before-it-acts)When the domain sends its own agents, signed requests under Web Bot Auth (built on RFC 9421 HTTP Message Signatures ) identify them better than a user agent string anyone can type.

## Definitions that name user agent

[visitor resolve](/glossary/visitor-resolve) · [operator origin](/glossary/operator-origin) · [signed agent](/glossary/signed-agent)

## Related terms in Crawlers and access

Who is fetching, whether they are who they claim, and what your rules actually permit.

[visitor resolve](/glossary/visitor-resolve) · [published crawler ranges](/glossary/published-crawler-ranges) · [operator origin](/glossary/operator-origin) · [signed agent](/glossary/signed-agent) · [agent arrival](/glossary/agent-arrival) · [agent conduct record](/glossary/agent-conduct-record) · [agent traffic index](/glossary/agent-traffic-index) · [user-triggered fetch exemption](/glossary/user-triggered-fetch-exemption) · [retrieval crawler](/glossary/retrieval-crawler) · [user-triggered fetch](/glossary/user-triggered-fetch) · [verified crawler](/glossary/verified-crawler) · [forged crawler identity](/glossary/forged-crawler-identity) · [unverifiable](/glossary/unverifiable) · [robots.txt](/glossary/robots-txt) · [user-agent group](/glossary/user-agent-group) · [AI opt-out](/glossary/ai-opt-out) · [Content-Signal](/glossary/content-signal) · [crawl budget](/glossary/crawl-budget) · [cloaking](/glossary/cloaking) · [challenge page at 200](/glossary/challenge-page-at-200) · [uniform refusal](/glossary/uniform-refusal) · [nonexistent-path control](/glossary/nonexistent-path-control) · [blocked render resource](/glossary/blocked-render-resource) · [off-host redirect](/glossary/off-host-redirect) · [homepage refused](/glossary/homepage-refused) · [FCrDNS](/glossary/fcrdns) · [Crawler trap](/glossary/crawler-trap) · [Conditional request](/glossary/conditional-request) · [ASN blocking](/glossary/asn-blocking) · [operator feed](/glossary/operator-feed) · [residential proxy](/glossary/residential-proxy) · [Google-Extended](/glossary/google-extended) · [Google-Agent](/glossary/google-agent) · [GPTBot vs OAI-SearchBot](/glossary/gptbot-oai-searchbot) · [ClaudeBot vs Claude-User](/glossary/claudebot-claude-user) · [PerplexityBot vs Perplexity-User](/glossary/perplexitybot-perplexity-user) · [CCBot](/glossary/ccbot) · [Bytespider](/glossary/bytespider) · [agentic traffic](/glossary/agentic-traffic) · [crawler classification](/glossary/crawler-classification) · [datacenter IP](/glossary/datacenter-ip) · [vantage point](/glossary/vantage-point) · [rate limit](/glossary/rate-limit) · [delivery comparison](/glossary/delivery-comparison) · [honeypot](/glossary/honeypot)

## Questions about user agent

### What is user agent?

The string a client sends naming itself, in the User-Agent request header. It is a claim, not an identity: anyone can send any string, so treating a crawler's name as proof of the crawler is the mistake behind most forged-crawler traffic.

### What does user agent not show or guarantee?

It is a claim, not an identity: anyone can send any string, so treating a crawler's name as proof of the crawler is the mistake behind most forged-crawler traffic.

### Which area of the glossary does user agent belong to?

Crawlers and access: Who is fetching, whether they are who they claim, and what your rules actually permit.

← [crawler classification](/glossary/crawler-classification) · [datacenter IP](/glossary/datacenter-ip) →

[See it in the full glossary](/glossary/user-agent) · 792 terms across 19 areas.

Free browser extension

See this on any site you visit, **in one click.**

CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.

[Add to Chrome — free](https://chromewebstore.google.com/detail/plgjengobhbdmbgckiiaaflgiepdbcpi?utm_source=crawlcheck.io&utm_medium=glossary)
[What it does](/extension)

Written by

[Emmanuel Orta](/author) · VSNARY

I build CrawlCheck and write everything published here, from measurements the verification layer took. Where a post got something wrong, the correction stays in the post.

[Every post, and where I got it wrong →](/author) · [Follow on LinkedIn →](https://www.linkedin.com/in/emmanuelorta)

We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io)

#### How we measure, and our own machine files

CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io).

**Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing.

**Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours.

**Pages:** [IP ranges](/data/ip-ranges) · [Keys](/keys) · [/](/) · [Listing check](/listing-check) | **Glossary:** [Exact-match retrieval](/glossary/exact-match-retrieval) · [Informational query](/glossary/informational-query) · [Focus indicator](/glossary/focus-indicator) · [content-change clock](/glossary/content-change-clock) · [Encrypted Client Hello](/glossary/encrypted-client-hello) | **Registry:** [badmintonsoldiers.club](/registry/badmintonsoldiers.club) · [deodental.co.uk](/registry/deodental.co.uk) · [icustomer.ai](/registry/icustomer.ai)

[Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)
