For developers and vendors · API, specs, keys

What can your code ask about any domain?

In one sentence

CrawlCheck for developers is one documented API — every operation in an OpenAPI 3.1 file with a worked example — that returns a signed, offline-verifiable answer about any domain with no key, and metered measurement lanes, typed SDKs, published signing keys and a conformance suite for anyone building on or implementing the specifications.

One documented API: 274 operations across 238 paths in an OpenAPI 3.1 file, each with a worked example. A signed answer per domain, typed SDKs on npm and PyPI, an MCP server in the official registry, and keys with published quotas.

GET https://crawlcheck.io/api/v1/resolve?domain=example.com
POST https://crawlcheck.io/api/v1/preflight

The signed answer and the preflight decision need no key. A key adds the measurement lanes — scans, batch, machine records with the bytes behind every finding, OpenTelemetry and PROV exports, webhooks and a CI gate — metered on a quota that is published with the plan. Every response verifies offline against the published signing keys.

Paying never changes a reading. Quickstart · openapi.json · Conformance

Signed answer · a public site, measured daily · observed 12 days agodenverpost.comreadwarncitewarnconnectnone declaredtransactneeds a human8 of 9 AI-crawler identities were served the page; robots.txt allows 73 of 114 tested agentsCrawl policydeclared · 73 of 114 agents allowedDeliveryobserved · 14 identities, 1 refusedMachine filesobserved · 5 read, bodies hashedFindings8 open · 4 seriousEd25519 · verifies offlineOpen the signed answer →

Three paths down

Docs. Specs. Keys.

API

Every operation, with an example.

The reference is generated from the live service: 274 operations, each with a worked example and the gate it sits behind. Quickstart gets an agent to its first signed decision in a minute.

API reference · Quickstart · SDKs

Specs

The contracts behind the answer.

Resolve is the protocol; the receipt header, the staple, Resolve over DNS, the tool lockfile and the observer protocol are the specifications a vendor implements against. Each has a versioned document and a conformance suite.

Resolve · Receipt header · Observer protocol

Keys

Published, rotated, verifiable.

Signing keys are published with their rotation record; one dependency-free script verifies any answer, receipt or export offline. A licence key is created from the pricing page and shown once; quotas and terms are public.

Signing keys · API terms · Plans and quotas

1 · The core contract

9 operations an agent needs, versioned and kept.

They stay at /api/v1 and keep answering for at least 180 days after any /api/v2 is announced. Stable fields change only in a new major version; everything they return verifies offline.

For	Call	What it gives you
ask	`GET /api/v1/resolve`	one signed answer about a domain: what it permits, serves and declares, and a decision per action
ask	`POST /api/v1/resolve`	the same for 1-100 domains
decide	`POST /api/v1/preflight`	a signed decision for one action, with a receipt to keep
receipt	`GET /api/v1/decisions/{sha256}`	for a site: check the receipt an agent presents in the CrawlCheck-Receipt header
revocation	`GET /api/v1/revocations`	the signed list of domains whose connect and transact decisions are revoked now
visitor	`GET /api/v1/visitor`	for a site: is this request really from the agent it names
visitor	`POST /api/v1/visitor`	the same for 1-100 requests
mcp	`GET /api/v1/mcp/scan`	before connecting: the tools a remote MCP server lists, scanned for poisoning
mcp	`POST /api/v1/mcp/lock/check`	before each session: compare a server's tools with the lockfile you kept

Marked x-core in /openapi.json · The list as JSON · Compatibility policy

2 · The specifications

9 documents a vendor implements against.

Each is a versioned document; this service is the reference implementation.

Resolve protocol
The signed answer about a domain: endpoints, signature, offline verification and versioning.
Receipt header
How an agent that asked before acting proves it to the site it acts on.
Staple
A site serves its own signed answer, so an agent can check it with no extra request.
Resolve over DNS
Every domain's signed answer as a TXT record.
MCP tool lockfile
Pins the tools an operator approved on an MCP server; the client refuses a change.
Observer protocol
Run an independent observer: enrol a key and sign what your network received.
Guard acceptance suite
The signed test cases every guard must pass before an agent acts.
Service level agreement
Availability and latency targets for the paid plans.
Errors, security and privacy considerations
What can fail, and how every answer and signed document handles security and privacy.

3 · SDKs

Ask, verify and guard in one call.

TypeScript and JavaScript

npm i @crawlcheck/sdk

Version 1.5.2 on npm; the same tarball is served at /sdk.

Python

pip install crawlcheck

Version 1.5.3 on PyPI, no dependencies.

MCP

claude mcp add --transport http crawlcheck https://crawlcheck.io/mcp

Listed in the official MCP Registry as io.github.emmanuelorta/crawlcheck.

4 · Keys and conformance

Nothing here has to be trusted.

Signing keys
Published with their rotation record.
The verifier
One file, no dependencies; checks any answer, receipt or export offline.
Conformance suite
The test vectors an implementation must pass.
The rulebook
Every finding code and what it means.

curl -o bundle.json "https://crawlcheck.io/api/bundle?id=<id>"
curl -O https://crawlcheck.io/crawlcheck-verify.mjs
node crawlcheck-verify.mjs bundle.json

SHA-256 of the verifier served today: b22547e95463d03e1c4177ae6de8ba23c1316629766f0138375edf12d9d77183. Check a record in the browser →

5 · Quotas and terms

Published with the plan.

Plan	Scans a month	Keyed lookups a month
Developer	2,000	100,000
Growth	20,000	1,000,000
Watch	500	10,000
Agency	5,000	50,000
Network	20,000	250,000

The signed answer, the preflight decision and the free scan are not metered. Plans → · API terms → · Compatibility policy →

The API, by lane

104 of 274 operations, mapped to 5 lanes.

The lane map is the guided subset; the operation counts and the full reference are generated from the OpenAPI file the service publishes.

Lane	Operations	For example
Decide	57 operations · 46 with no key	`GET /api/v1/resolve` `POST /api/v1/preflight` `GET /api/v1/resolve-prefix/{prefix}`
Measure	6 operations · 3 with no key	`GET /sandbox` `POST /api/v1/scan` `POST /api/scan`
Read	23 operations · 9 with no key	`GET /api/v1/reports/{id}` `GET /r/{id}.json` `GET /r/{id}.raw.json`
Data	5 operations · 5 with no key	`GET /api/public/counts` `GET /api/corpus/state` `GET /api/telemetry`
Tools	13 operations · 9 with no key	`POST /api/tool/verify` `GET /api/tool/robots` `GET /api/tool/llms`

Every operation, with a worked example →

Docs

Everything else, in one list.

Machine-readable

/openapi.json — OpenAPI 3.1
/.well-known/api-catalog
MCP server card
/.well-known/crawlcheck.json — the agent manifest, signed
/llms.txt
SDKs on npm and PyPI, byte-identical to the tarballs served here

Questions about this page

QDo I need a key?

Not for the signed answer, the preflight decision or the free scan. A key adds the measurement lanes — scans, batch, machine records, exports, webhooks, the CI gate — on a quota published with the plan.

QWhich SDKs exist?

Typed clients on npm and PyPI, byte-identical to the tarballs served from this site, with the offline verifier bundled. An MCP server is listed in the official MCP Registry, and plain HTTP works everywhere else.

QHow do I verify a response offline?

Every answer, receipt and export is Ed25519-signed with a published key. One dependency-free script, crawlcheck-verify.mjs, checks any of them on your own machine; evidence bundles also carry the day’s sealed root and the Bitcoin timestamp.

QWhere are the specifications?

Resolve is the protocol; the receipt header, the staple, Resolve over DNS, the tool lockfile and the observer protocol each have a versioned document and a conformance suite you can run against your own implementation.

QWhat are the quotas?

Developer 2,000 scans a month, Studio 1,500, Agency 5,000, Growth and Network 20,000. The signed answer and the preflight decision are not metered. Quotas and terms are public and change only with notice.