Ask before acting, and check the answer yourself.

One package for JavaScript and one for Python, no dependencies in either. An agent gets a signed decision before it reads, cites, connects or transacts, uses a site’s own stapled verdict when one is served, and verifies every answer, receipt and log entry on its own machine.

npm i @crawlcheck/sdk

pip install crawlcheck

From npm (npmjs.com/package/@crawlcheck/sdk), published from this repository by trusted publishing. The same tarball, byte for byte, is served here: npm i https://crawlcheck.io/sdk/crawlcheck-sdk-1.0.9.tgz

80573 bytes · sha256 27ad64db1ea854e5eab8caca2b74dae00d9313e384547a1cefdd4f93f9ef5740 · npm integrity sha512-efnohTscjIQptiZuzd3t2F/Jfkw0Cc//Rnvq8tUv1l6eVFQhTEKrN5UnN3siYgIwlrOjVq+SccMUkfKPoserqw== · verifier c96356e45adb8e56 · types generated for 129 documented paths · versions served here: index.json (npm holds every release)

Check before acting

import { CrawlCheck } from "@crawlcheck/sdk";
const cc = new CrawlCheck();
const { proceed, receipt } = await cc.guard("example.com", "read", { agent: "MyAgentBot" });
// proceed is true on allow; warn proceeds only with { allowWarn: true }; require_confirmation only if confirm(receipt) resolves true.
// The receipt is a signed crawlcheck-decision; guard() verified it here and checked its key against the published directory.

Every verification returns integrity_valid, issuer_trusted and accepted. Act on accepted: it needs the signing key to be one CrawlCheck publishes, so a document someone signed with their own key is verified offline but never accepted. The client methods (verifyDocument, verifyOffline, verifyReceiptOffline, guard) check the published key directory for you.

preflight(domain, action, { agent, template, policy }) returns the signed receipt without deciding for you; verifyDocument(doc) verifies any signed resolve answer or decision receipt offline. resolve(domain) returns the full signed ResolveV1 answer.

Read a record

import { CrawlCheck } from "@crawlcheck/sdk";
const cc = new CrawlCheck();                       // { key: "cc_…" } for licence-gated fields
const mr = await cc.machineRecord({ domain: "example.com" });
mr.findings;            // gated exactly like the website: the top finding without a licence
mr.score_ledger;        // what each row and finding does to the grade

Every request and response type is generated from https://crawlcheck.io/openapi.json. Any documented route is callable with types checked against the contract:

await cc.get("/api/explain", { domain: "example.com", code: "NO_LLMS_TXT" });
await cc.get("/api/explain", { website: "x" });   // compile error: not a parameter of this operation

Use the site's staple

A site can serve its own signed verdict in the CrawlCheck-Staple header (and at /.well-known/crawlcheck-staple.json). resolveFor uses it when it verifies, with no request to CrawlCheck, and falls back to a signed lookup when it is missing, expired, for another domain or tampered:

const res = await fetch("https://shop.example.com/");
const v = await client.resolveFor("shop.example.com", { response: res });
v.source;              // "staple" or "network"
v.decisions.transact;  // allow | warn | require_confirmation | block | unsupported

stapleCheck(record, { host, publishedKids }) does the offline check on its own. Spec: https://crawlcheck.io/spec/staple

Do not trust the API: check the evidence yourself

const v = await cc.verifyOffline(mr.subject.report_id);   // downloads the bundle, verifies it HERE
v.checks;   // manifest hash, Ed25519 signature, key id, record digest, Merkle path to the day's root,
            // OpenTimestamps proof (Bitcoin), sealed roots over fetches, decisions and facts

ok: true passed, ok: false failed, ok: null could not run and why says why. A null is never a pass.

Remediation receipts (before, declared fix, deployment, after, verdict, signed):

import { verifyReceipt } from "@crawlcheck/sdk";
const rc = await cc.receipt("rc1:…");
const r = await verifyReceipt(rc);                                  // hash, key id, signature, order
const ours = (await cc.publishedKeyIds()).includes(rc.signature.kid);  // the one check offline cannot do
const accept = r.verified && ours;

Prove the verifier

const suite = await cc.conformance();   // 19 real bundles and receipts, valid and tampered, with expected outputs
suite.ok;                              // every fixture gave exactly its expected checks and verdict

npm test in the installed package runs the conformance suite and live checks against crawlcheck.io.

Catch a split view

Every Resolve answer names its leaf in CrawlCheck's transparency log and, once merged, carries the signed tree head of its batch. The client keeps every head it sees in client.log (plain JSON) and checks each new one against its neighbours: two heads for one batch, or a head that does not chain to the one before, is a **fork**, kept with both signed heads as proof. guard() refuses to proceed while the client holds one.

import fs from "node:fs";
import { CrawlCheck, newLogStore } from "@crawlcheck/sdk";
const store = fs.existsSync("cc-log.json") ? JSON.parse(fs.readFileSync("cc-log.json", "utf8")) : newLogStore();
const client = new CrawlCheck({ logStore: store });
await client.resolve("example.com");                 // the answer's head is checked against the store
const r = await client.logCheck();                   // current head + GitHub's witness co-signature + the chain down to your heads
console.log(r.summary);                              // "consistent: ..." or "SPLIT VIEW: ..."
fs.writeFileSync("cc-log.json", JSON.stringify(client.log));

logCheck verifies the outside witness: a GitHub Actions run re-checks the chain and co-signs the newest head with a GitHub OIDC token (RS256, GitHub's key) whose audience is the head's digest. A head the witness's chain does not contain shows up as a fork. Heads newer than the witnessed one are listed as unwitnessed. Clients can also swap heads directly: logHeads(store) on one side, client.logGossip(heads) on the other. Forks are reported to /api/v1/log/gossip (pass { report: false } to skip).

Errors

A refused request throws CrawlCheckError with status, body and locked (true for "this needs a licence for the domain").

Requires Node 20+ or any current browser (WebCrypto Ed25519).

Python · crawlcheck 1.0.9

The same calls from Python 3.8+, with no dependencies: resolve and batch resolve, the preflight policy engine and guard, resolve_for and staple_check for a site’s stapled verdict, log_check and split-view detection, the MCP tool lockfile, outcomes, and offline verification of resolve answers and decision receipts (pure-Python Ed25519, canonical JSON identical to the server’s).

pip install crawlcheck

From PyPI (pypi.org/project/crawlcheck), published from this repository by trusted publishing. The same wheel is served here: pip install https://crawlcheck.io/sdk/crawlcheck-1.0.9-py3-none-any.whl

from crawlcheck import CrawlCheck

cc = CrawlCheck()
receipt = cc.preflight("example.com", action="cite", agent="GPTBot", template="safe_citation")
print(receipt["decision"])            # allow | warn | require_confirmation | block | unsupported
assert cc.verify(receipt)["verified"]  # checked locally against the published key

ok, receipt = cc.guard("shop.example", action="transact", confirm=lambda r: ask_a_human(r))

Command line: python -m crawlcheck preflight example.com cite GPTBot · python -m crawlcheck verify receipt.json

23420 bytes · sha256 909367d1128e560cd58587a036b316303efc35887799743c30026c762f8c20fb · built reproducibly by sdk-python/build_wheel.py