SDK

@crawlcheck/sdk 1.0.6

A typed client generated from the API contract, and the offline verifier, in one package with no dependencies.

npm i @crawlcheck/sdk

From npm (npmjs.com/package/@crawlcheck/sdk), published from this repository by trusted publishing. The tarball below is also served here: npm i https://crawlcheck.io/sdk/crawlcheck-sdk-1.0.6.tgz

65949 bytes · sha256 17252e9062be2ff9ae56d44a9e7c94df9303fef0aba7c04e3a9385189b5f9fe1 · npm integrity sha512-LHFDJaq86WfoavDTwmujxMkCw14fTYPeX+g9sVFz2+9zmsTyVbp4JuaE1ltUGL87nHlsHYJqlx40t2cpYtFUTQ== · verifier ab60cebb8a0c800a · typed from 146 documented paths · versions: index.json

A typed client for the CrawlCheck API and the offline verifier for its evidence, in one package with no dependencies.

npm i @crawlcheck/sdk

Check before acting

import { CrawlCheck } from "@crawlcheck/sdk";
const cc = new CrawlCheck();
const { proceed, receipt } = await cc.guard("example.com", "read", { agent: "MyAgentBot" });
// proceed is true on allow; warn proceeds only with { allowWarn: true }; require_confirmation only if confirm(receipt) resolves true.
// The receipt is a signed crawlcheck-decision; guard() verified it here and checked its key against the published directory.

Every verification returns integrity_valid, issuer_trusted and accepted. Act on accepted: it needs the signing key to be one CrawlCheck publishes, so a document someone signed with their own key is verified offline but never accepted. The client methods (verifyDocument, verifyOffline, verifyReceiptOffline, guard) check the published key directory for you.

preflight(domain, action, { agent, template, policy }) returns the signed receipt without deciding for you; verifyDocument(doc) verifies any signed resolve answer or decision receipt offline. resolve(domain) returns the full signed ResolveV1 answer.

Read a record

import { CrawlCheck } from "@crawlcheck/sdk";
const cc = new CrawlCheck();                       // { key: "cc_…" } for licence-gated fields
const mr = await cc.machineRecord({ domain: "example.com" });
mr.findings;            // gated exactly like the website: the top finding without a licence
mr.score_ledger;        // what each row and finding does to the grade

Every request and response type is generated from https://crawlcheck.io/openapi.json. Any documented route is callable with types checked against the contract:

await cc.get("/api/explain", { domain: "example.com", code: "NO_LLMS_TXT" });
await cc.get("/api/explain", { website: "x" });   // compile error: not a parameter of this operation

Do not trust the API: check the evidence yourself

const v = await cc.verifyOffline(mr.subject.report_id);   // downloads the bundle, verifies it HERE
v.checks;   // manifest hash, Ed25519 signature, key id, record digest, Merkle path to the day's root,
            // OpenTimestamps proof (Bitcoin), sealed roots over fetches, decisions and facts

ok: true passed, ok: false failed, ok: null could not run and why says why. A null is never a pass.

Remediation receipts (before, declared fix, deployment, after, verdict, signed):

import { verifyReceipt } from "@crawlcheck/sdk";
const rc = await cc.receipt("rc1:…");
const r = await verifyReceipt(rc);                                  // hash, key id, signature, order
const ours = (await cc.publishedKeyIds()).includes(rc.signature.kid);  // the one check offline cannot do
const accept = r.verified && ours;

Prove the verifier

const suite = await cc.conformance();   // 19 real bundles and receipts, valid and tampered, with expected outputs
suite.ok;                              // every fixture gave exactly its expected checks and verdict

npm test in the installed package runs the conformance suite and live checks against crawlcheck.io.

Errors

A refused request throws CrawlCheckError with status, body and locked (true for "this needs a licence for the domain").

Requires Node 20+ or any current browser (WebCrypto Ed25519).

Python

The same calls from Python 3.8+, with no dependencies: resolve, batch resolve, the preflight policy engine, outcomes, and offline verification of resolve answers and decision receipts (pure-Python Ed25519, canonical JSON identical to the server’s).

pip install crawlcheck

From PyPI (pypi.org/project/crawlcheck), published from this repository by trusted publishing. The same wheel is served here: pip install https://crawlcheck.io/sdk/crawlcheck-1.0.5-py3-none-any.whl

from crawlcheck import CrawlCheck

cc = CrawlCheck()
receipt = cc.preflight("example.com", action="cite", agent="GPTBot", template="safe_citation")
print(receipt["decision"])            # allow | warn | require_confirmation | block | unsupported
assert cc.verify(receipt)["verified"]  # checked locally against the published key

ok, receipt = cc.guard("shop.example", action="transact", confirm=lambda r: ask_a_human(r))

Command line: python -m crawlcheck preflight example.com cite GPTBot · python -m crawlcheck verify receipt.json

17742 bytes · sha256 04b1eebde35227d0b37382cfcbfd42d7c5f6a5be070597ac163003e5949b5d3d · built reproducibly by sdk-python/build_wheel.py