CrawlCheck
Assurance

How far CrawlCheck can be relied on, with the denominators.

What an agent depends on when it asks before acting: the service answering in time, a receipt it can check without trusting us, observers it can tell apart from us, coverage counted honestly, and evidence of who actually uses it. Every number here is read from the running system when the page loads; the same document is /api/v1/assurance.

0250,000500,000attemptedattempted: 437,213437,213measuredmeasured: 301,851301,851full scansfull scans: 3,8723,872registry-listedregistry-listed: 758758
Denominator: 437,213 domains CrawlCheck has tried to measure · Scale: domains, linear, 0–500,000 · Window: as of 2026-10-16 09:04 UTC, cumulative

Definition

Attempted
A domain CrawlCheck tried to read, from the census, the registry, a lookup or a scan.
Measured
A domain with a stored, signed record of what was served.
Full scan
The on-demand deep read: every section, every finding with its evidence.
Registry-listed
A domain with a record in the public registry, claimed or not.

What this does not prove

  • Measured means read and recorded, not right: every finding carries its evidence so it can be checked.
  • Coverage counts what CrawlCheck was asked about or seeded with, not a sample of the web.
  • A full scan is a moment; the record says when it was taken.
  • The keys, observers and eligibility checks further down are listed as configured; the status page says whether they are answering now.

Check it yourself

The numbers on this page come from the signed JSON at /api/v1/instruments/assurance/now.json, rebuilt every few minutes; each copy carries its own sha256 and an Ed25519 signature by key 1hHmE3fo1rg5…. For a copy that never changes, cite a weekly edition. Verify either with no call to us:

node crawlcheck-verify.mjs --online assurance.json

The verifier: /crawlcheck-verify.mjs, zero dependencies.

Editions

One sealed edition per ISO week: the signed reading as first taken that week, never edited. Feed: Atom · all instruments: /api/v1/instruments.

degraded

service now (status)

2

signing keys, by purpose

1 of 11

observers not run by us

0

decisions to verified AI agents, 30 days

1 · Availability

Targets and what was measured

Last 30 days. /api/slo

ObjectiveTargetMeasuredSamplesp50 / p95Result
Resolve answers without a server error99.9%100%12,210210 / 327 msmet
met at 95% confidence
Resolve answers for a measured domain within 300 ms in the Worker95%90.3%12,210210 / 327 msmissed
missed at 95% confidence
Preflight decisions without a server error99.9%100%2,442219 / 344 msmet
not settled: the target is inside the 95% interval, so more samples are needed to call it either way
Preflight decisions within 300 ms in the Worker95%75.4%2,442219 / 344 msmissed
missed at 95% confidence
A first answer for a never-measured domain within 2 seconds in the Worker95%80%2101076 / 4370 msmissed
missed at 95% confidence

A missed objective stays published until the window rolls past it. Open incidents: 0; the signed incident log is at /api/status/log.

2 · Before acting

A valid signature is not permission

Six separate answers; act only when all hold.

CheckFails when
integrity_valida byte of the receipt changed after signing
signature_validthe Ed25519 signature does not verify over the digest
issuer_trustedthe key is not in CrawlCheck’s published directory: a perfect signature from anyone else’s key is refused
freshthe receipt is past expires_at (each action has its own validity; transact is the shortest)
scope_validthe receipt names a different domain or action from the one you are about to take
decision_permitsthe decision is block or unsupported; warn without your policy accepting it; require_confirmation without a human’s yes
curl -s -X POST https://crawlcheck.io/api/v1/preflight -H 'content-type: application/json' \
  -d '{"domain":"example.com","action":"read"}' -o receipt.json
curl -sO https://crawlcheck.io/crawlcheck-verify.mjs
node crawlcheck-verify.mjs receipt.json --act example.com:read      # exit 0 only when eligible_to_act

The SDK’s guard() makes the same checks in your code; the quickstart puts it inside the function that performs the action, so the model cannot route around it. Keys: requests and documents are signed with separate keys.

3 · Independence

Who runs each observer and witness

10 of 11 are run by CrawlCheck.

Different networks are not different operators. An observer we run from another cloud shows whether a result depends on where it was measured from; only an operator outside CrawlCheck can show it does not depend on us. Each row says which it is.

ObserverOperatorNetworkVerifiesCode · credentials · deployLast readingLimitation
crawlcheck-worker
active
CrawlCheck (VSNARY) (us)CrawlCheck primary networkevery primary measurementours · ours (Worker secret) · ours—this is CrawlCheck itself: it corroborates nothing
gha-azure-1
active
CrawlCheck (VSNARY) (us)GitHub-hosted runners on Microsoft Azure; not any crawler operator's network, so it corroborates a vantage question, never an operator's own crawlthe homepage under each identity, from a second networkours · a new key per run, bound to our workflow by a GitHub token · ours (repository workflow)2026-10-16 06:28a different network and a third party (GitHub) attesting which workflow ran; not a different operator
ext:*
active
people running the CrawlCheck extension (version 1.0.4 and later) with sharing switched onthe visitor's own connection (home, office or mobile), recorded as country and network name only; the address is never storedwhat a crawler is served from a visitor's own connectionours (the extension) · one key per install, made on the device · installed by each person—the vantage is the visitor's; the code is ours; an install key proves one install, not one person
ind:cli-2ns-x7r1zy
unknown
CrawlCheck (VSNARY), PHP client test from a Google Cloud container (us)this machine's networkreadings of queued URLs under the open observer protocolthe open protocol client · an Ed25519 key made on the observer's machine · whoever runs it—run by CrawlCheck from another network: network diversity, not an independent operator
ind:gcp-session-1
unknown
CrawlCheck (VSNARY), run from a Claude cloud session (us)Google Cloud egress (AS396982), not Cloudflare, not Azurereadings of queued URLs under the open observer protocolthe open protocol client · an Ed25519 key made on the observer's machine · whoever runs it—run by CrawlCheck from another network: network diversity, not an independent operator
ind:gcp-tv-1
unknown
CrawlCheck (VSNARY), test vantage run from a Google Cloud container (us)Google Cloud (AS396982), Columbus USreadings of queued URLs under the open observer protocolthe open protocol client · an Ed25519 key made on the observer's machine · whoever runs it—run by CrawlCheck from another network: network diversity, not an independent operator
ind:google-llc-iowa-e94e42
quarantined
CrawlCheck (VSNARY) - installer test from a Google Cloud container (us)AS396982 Google LLC, Council Bluffs, Iowa, USreadings of queued URLs under the open observer protocolthe open protocol client · an Ed25519 key made on the observer's machine · whoever runs it—run by CrawlCheck from another network: network diversity, not an independent operator
ind:installtest-gcp
quarantined
CrawlCheck (VSNARY) installer test (us)AS396982 Google LLC, Columbus USreadings of queued URLs under the open observer protocolthe open protocol client · an Ed25519 key made on the observer's machine · whoever runs it2026-10-08 20:40run by CrawlCheck from another network: network diversity, not an independent operator
ind:refcheck-gcp-
quarantined
CrawlCheck (VSNARY) reference check (us)Google Cloud containerreadings of queued URLs under the open observer protocolthe open protocol client · an Ed25519 key made on the observer's machine · whoever runs it2026-10-08 21:13run by CrawlCheck from another network: network diversity, not an independent operator
ind:vsnary-cloudways-1
active
VSNARY (operator of crawlcheck.io) (us)Cloudways managed hosting: the WordPress server of treeservicedenverllc.comreadings of queued URLs under the open observer protocolthe open protocol client · an Ed25519 key made on the observer's machine · whoever runs it2026-10-16 08:28run by CrawlCheck from another network: network diversity, not an independent operator
log-witness
active
GitHub (identity token) running CrawlCheck's workflow (us)GitHub-hosted runnerseach signed head of the Resolve log, co-signed with a GitHub-issued token naming the workflow that saw itours · GitHub's OIDC token for that run · ours (repository workflow)—GitHub attests which workflow ran and when; it does not recompute the log. A split view is caught by clients comparing heads (gossip)

Anyone can run an observer under the open observer protocol; an observer run by someone else is the one thing that turns network diversity into independence.

4 · Coverage

What each number counts

DomainsCount
attempted by the census437,213
with at least one reading (“sites measured”)301,851
full scans, graded3,872
in the public registry (full scan, publishable, not opted out)758

attempted = every domain the census tried; measured = domains with at least one reading (most are the census's lighter reading); full scans = graded reports; registry = full scans that can be published and have not opted out. Resolve answers for any domain; a never-measured one gets a first, partial answer marked as such.

MCP serversCount
records in the official MCP Registry, imported43,267
with a remote endpoint27,645
probed this cycle (handshake only, never tools/call)5,875
completed the handshake and listed tools2,734
completed the handshake with no tools102
asked for credentials2,045
errored, unreachable or not MCP986
tool lists scanned for poisoning2,734
flagged by the scan2
tool lists changed since first seen522

indexed is not connected, and connected is not safe: a clean tool scan means no known poisoning pattern was found, not that none exists. Cycle started 2026-10-13.

5 · Adoption

Who actually asks

Our own traffic is counted apart, never as adoption.

Last 30 daysCount
signed decisions to verified AI agents (operator ranges)0
signed decisions to other callers706
signed decisions to us (tests, probes, our sites)109
receipt checks by sites we do not run31
receipt checks by our own sites3

Calls to the agent surfaces (MCP, Resolve, preflight, Visitor Resolve, outcome) 2026-10-09 to 2026-10-15: 23,956 from outside, 9,223 ours. Outside, by caller: node 8,524 · other 4,128 · bot:other 3,950 · mcp-client 2,350 · browser 2,005 · bot:Amazonbot 686 · python 608 · http-client 599 · sdk-python 346 · no-ua 171 · perplexity 118 · bot:GolemreachTrustBot 88 · bot:Faceabot 84 · bot:BrickBlueBot 60 · bot:InvokeRankBot 34 · bot:agent-tools.cloud-crawle 29 · bot:TalandorBot 24 · bot:Maps-bot 20 · bot:WellknownBot 20 · bot:MJ12bot 20 · bot:YandexBot 18 · bot:PluginboardBot 16 · claude 11 · openai 9 · bot:Reflectionbot 6 · bot:ShapBot 4 · bot:ThePluginStoreBot 4 · browser-app 3 · bot:DataForSeoBot 3 · bot:SemrushBot 3 · google-ai 3 · bot:archive.org_bot 2 · bot:AhrefsBot 2 · bot:Applebot 2 · bot:CapabilityIndexBot 2 · bot:ExaSearchBot 1 · bot:ExtOpsBot 1 · bot:research-bot 1 · bot:AnchorTerminalBot 1.

ours = our key, owner session, own sites, probes or self-tagged tests; other = everyone else and may still include an untagged test of ours; verified_agent = a caller naming an AI agent from inside that operator's published ranges, which we cannot pad. Caller classes are read from the user agent and recorded from 2026-10-10. The day-by-day series is /api/v1/adoption.

Funnel, last 30 days

StageCountOfShare
decisions asked for by callers that are not us
verified agents plus other callers; other may still include an untagged test of ours
706——
of those, from an AI agent verified by its operator's ranges07060%
distinct agent names asking (self-declared)
counted from 10 Oct 2026; an agent that names itself the same way is one
6——
receipts checked by a site we do not run
a presented receipt is only visible when the site checks it, so presented and checked are one stage
317064.4%
distinct sites checking receipts2——
agent names asking on 2 or more days4666.7%
sites checking on 2 or more days22100%

Our own traffic (our key, owner session, own sites, self-tagged tests) is never counted. Receipt checks recorded before 10 Oct 2026 were classed by host alone, so they are listed apart as host_only_receipts and excluded from every stage and from the goal. Set apart: 10 receipt checks classed by host only.

6 · Operations

Witness, seals, drills and succession

Read from the running system; each row links to its source.

WhatLastMeasuredSource
Outside witness co-signs the log3 h agobatch 584 of 591 (7 behind) · hourly at :51 UTC on GitHub Actions, which can run latewitness
Daily seal after the day closes2026-10-157 of 7 days sealed · median 78 min after close, worst 1,278 · 7 in a Bitcoin blockproof
Revocation drill12 h agopassed in 2,294 ms · 11 of 11 recent runs passedself-test
Restore test (rebuild from archive)4 days agogreen · 20 of 20 identical, 0 mismatched · weekly, Mondays 03:17 MDT (09:17 UTC), and on demand; GitHub Actions, proven by GitHub's OIDC tokentest

daily: the incident switch on selftest.crawlcheck.io is flipped, Resolve must report the incident, a transact preflight must return block, and the domain must appear on the signed revocation list, then it is cleared. the oldest archived records rebuilt from the archive alone (the 90-day projections ignored) and compared byte for byte with what was served. delay = when the seal was written after the day closed; a seal reaches Bitcoin when a calendar's attestation is mined, usually within hours.

Succession

If the operator stops, this continues:

  • every signed answer, receipt and evidence bundle verifies offline with crawlcheck-verify.mjs and the published keys, with no call to CrawlCheck
  • each sealed day stays provable through its OpenTimestamps proof and Bitcoin block, which no one at CrawlCheck can alter
  • the daily signed snapshot and the census are public (CC BY 4.0) and can be mirrored by anyone
  • the transparency log's witnessed heads are held by every client that kept them

This stops:

  • new measurements and new signed answers
  • revocations and incident switches
  • the observer network's scheduling

no successor operator has been named yet; until one is, the guarantees above are the ones that survive the operator.

Generated 2026-10-16 09:04 UTC.