Which agents really reached the sites that report, and how many were impersonators.
Every agent request a reporting site receives is checked against the address ranges the agent’s operator publishes, at that site’s own origin. Verified requests count for the agent; requests using its name from anywhere else are forged claims. No reporting site is named.
Definition
- Verified
- The request came from the operator's published network ranges, or carried a valid Web Bot Auth signature.
- Forged claim
- The user agent named a known crawler but the request did not come from that operator.
- Unverifiable
- The operator publishes no way to check its traffic.
- Reporting site
- A site whose edge or reporter sends its crawler log to the index. An agent seen on fewer than 5 sites is not published.
What this does not prove
- A handful of reporting sites is a panel, not the web: shares describe these sites.
- Forged shares depend on who targets these sites; they are not a global spoofing rate.
- A month-to-date edition changes until the month closes; cite the closed edition.
- Fetches say a crawler came, not what it did with the page.
Check it yourself
The numbers on this page come from the signed JSON at /api/v1/traffic-index, rebuilt every few minutes; each copy carries its own sha256 and an Ed25519 signature by key 1hHmE3fo1rg5…. For a copy that never changes, cite a weekly edition. Verify either with no call to us:
node crawlcheck-verify.mjs --online traffic-index.json
The verifier: /crawlcheck-verify.mjs, zero dependencies.
Editions
One sealed edition per ISO week: the signed reading as first taken that week, never edited. Feed: Atom · all instruments: /api/v1/instruments.
Final edition: the month is over and these counts no longer change. Signed 2026-10-11 23:34 UTC. Other months: 2026-10 · 2026-09 · 2026-08
Only 0 sites reported in 2026-08: below the 5-site floor, so totals are shown and no per-agent rate is published.
Use the numbers
The signed edition is /api/v1/traffic-index/2026-08 (kind crawlcheck-traffic-index). Save it and run node crawlcheck-verify.mjs --online edition.json with the public verifier: the signature and the published key are checked on your machine. Licence: CC BY 4.0.
Cite as: CrawlCheck (2026). Agent Traffic Index, 2026-08. https://crawlcheck.io/traffic-index?month=2026-08
Add your site to the index
A site on Cloudflare reports with three lines in its Worker: the snippet in /spec/visitor-resolve checks each agent request with Visitor Resolve and sends one arrival per hit after the response is out. The site is proven by the CF-Worker header Cloudflare sets, so no key is needed, and the site is never named here. Each new site lifts more agents over the 5-site floor. Per-agent behaviour on one property is in Agent conduct.