CrawlCheck
Agent conduct

What each agent actually did, from requests it really sent.

Every request to this site that names a tracked agent is checked against the address ranges its operator publishes. Only verified requests count toward an agent’s record; a request using its name from anywhere else is counted against the impersonator. One property’s census, signed at /api/v1/conduct.

194,897

verified requests

1,524

forged claims, counted against the forger

14

agents with a clean robots.txt record

6

agents that fetched a disallowed path

For sites and agents

How the record is used

1 · A site

Is this visitor really it?

Visitor Resolve answers for one request, from the same published ranges: verified, spoofed or unverifiable.

Visitor Resolve/api/v1/visitor

2 · An operator

How your crawler is seen

Each agent has its own signed record: verified requests, forged claims, robots.txt and its busiest minute here.

The full recordCrawler catalogue

3 · Everyone

Across many sites

The monthly Agent Traffic Index counts which agents reached which reporting sites, from the same kind of verification.

Traffic indexAll open data

Right now

Active today

Verified requests per UTC day, counted from 18 Oct 2026.

Agent of the day (2026-11-21): Amazonbot (Amazon), 2,227 verified requests.

#AgentToday (2026-11-22)YesterdayGrade
1AhrefsBot
Ahrefs
162147D
2YandexBot
Yandex
23485A
3PerplexityBot
Perplexity
23441D
4OAI-SearchBot
OpenAI
20140A
5GPTBot
OpenAI
2063B
6Bingbot
Microsoft
19228D
7Applebot
Apple
1490D
8Googlebot
Google
10110A
9Amazonbot
Amazon
42,227D
10DuckDuckBot
DuckDuckGo
24A
11ChatGPT-User
OpenAI
110A
12Baiduspider
Baidu
10B
13Claude-User
Anthropic
021A
14SeznamBot
Seznam
020A
15GoogleOther
Google
010A
Verified agents

The record, agent by agent

A verdict needs 50 verified requests. The only rule judged is robots.txt; the busiest minute is shown, not judged, because this site publishes no crawl rate.

GradeAgentVerified requestsForged claimsrobots.txtBusiest minuteConduct
DAmazonbot
Amazon · training
37,134119disallowed paths fetched34/minviolations observed
DPerplexityBot
Perplexity · indexing
35,925101disallowed paths fetched120/minviolations observed
AYandexBot
Yandex · indexing
29,92220respected26/minno violations observed
DAhrefsBot
Ahrefs · tool
15,0703disallowed paths fetched64/minviolations observed
DClaudeBot
Anthropic · training
13,375124disallowed paths fetched69/minviolations observed
DApplebot
Apple · indexing
12,68384disallowed paths fetched8/minviolations observed
AGooglebot
Google · indexing
12,437266respected13/minno violations observed
DBingbot
Microsoft · indexing
11,27452disallowed paths fetched12/minviolations observed
BGPTBot
OpenAI · training
10,547184robots txt not fetched120/minno violations observed
AOAI-SearchBot
OpenAI · indexing
9,105111respected119/minno violations observed
AChatGPT-User
OpenAI · live fetch
1,77890not applicable13/minno violations observed
AClaude-User
Anthropic · live fetch
1,60755not applicable7/minno violations observed
ASeznamBot
Seznam · indexing
1,1560respected3/minno violations observed
BBaiduspider
Baidu · indexing
69145robots txt not fetched5/minno violations observed
ACCBot
Common Crawl · training
6369respected7/minno violations observed
AGoogle-CWS
Google · store listing fetch
4780not applicable3/minno violations observed
AGoogle-InspectionTool
Google
3615respected7/minno violations observed
AGoogle-Agent
Google · live fetch
2810not applicable17/minno violations observed
ADuckDuckBot
DuckDuckGo · indexing
2030respected2/minno violations observed
AGoogleOther
Google · training
193134respected5/minno violations observed
—Perplexity-User
Perplexity · live fetch
952insufficient data1/mininsufficient data
—Google-NotebookLM
Google
80insufficient data4/mininsufficient data
—Google-Read-Aloud
70insufficient data3/mininsufficient data
—Yeti
Naver · indexing
60insufficient data1/mininsufficient data
—AdsBot-Google
Google · tool
50insufficient data2/mininsufficient data
—Google-Safety
40insufficient data2/mininsufficient data
—Google-Site-Verification
Google · ownership check
20insufficient datanot yetinsufficient data
—Claude-SearchBot
Anthropic · indexing
058insufficient datanot yetinsufficient data
—MistralAI-User
Mistral · live fetch
00insufficient datanot yetinsufficient data
—FeedFetcher-Google
Google · feed fetch
01insufficient datanot yetinsufficient data
—Google-CloudVertexBot
Google · tool
03insufficient datanot yetinsufficient data
—Storebot-Google
Google · indexing
08insufficient datanot yetinsufficient data

Verified = source address inside the operator’s published ranges. Forged claims = requests that used the agent’s name from outside them; they are counted against the impersonator, never the agent. Disallowed paths are /r/, /backup/, /admin-old/. Busiest minute = all of an operator’s addresses combined; rate and revalidation counters started 7 October 2026. Generated 2026-11-22 01:06 UTC.

Not assessed

Seen, but not verifiable here

No address-range file this record reads, so nothing they did can be separated from an impersonator using their name.

AgentClaimed requestsWhy
Meta-ExternalAgent126,727no range file this record reads: cannot be told from an impersonator
Bytespider20,109no range file this record reads: cannot be told from an impersonator
Google-Extended49no range file this record reads: cannot be told from an impersonator
Diffbot2no range file this record reads: cannot be told from an impersonator
SemrushBot30,353no range file this record reads: cannot be told from an impersonator
DataForSeoBot16,743no range file this record reads: cannot be told from an impersonator
anthropic-ai4no range file this record reads: cannot be told from an impersonator
Applebot-Extended2no range file this record reads: cannot be told from an impersonator
Timpibot4no range file this record reads: cannot be told from an impersonator
YouBot2,524no range file this record reads: cannot be told from an impersonator
Scrapy18no range file this record reads: cannot be told from an impersonator
cohere-ai62no range file this record reads: cannot be told from an impersonator
FacebookBot1no range file this record reads: cannot be told from an impersonator
Method

Where the counts come from

Every request to this site that names a tracked agent is checked against the operator’s published ranges and counted by path. Other sites add what they found through Visitor Resolve (the site_checks column in the record). The monthly population view is the Agent Traffic Index.

Check it yourself

Every number here comes from the signed JSON at /api/v1/conduct; each copy carries its own sha256 and an Ed25519 signature by key 1hHmE3fo1rg5…. Verify it with no call to us:

node crawlcheck-verify.mjs --online conduct.json

The verifier: /crawlcheck-verify.mjs, zero dependencies.

Editions

One sealed edition per ISO week, never edited. Feed: Atom · all instruments: /api/v1/instruments.