Conformance
Test any CrawlCheck verifier
19 real bundles and receipts of crawlcheck.io’s own scans, each with the exact output a conforming verifier must give. The negative fixtures change one thing and name the one check that must fail. Expected outputs were written from each change, not by running the verifier.
node run.mjs — run.mjs downloads the verifier and every fixture, runs them and exits 1 on any disagreement. Index with every file’s SHA-256: index.json. Built 2026-10-01T14:39:35.511Z against verifier 46206b177cc941e1.
| Fixture | Group | What it is | Expected | Must fail |
|---|---|---|---|---|
valid-v3-anchored | valid | A manifest-v3 bundle of a Bitcoin-anchored day, with the subject and every leaf: every check runs and passes. | verified offline · accepted | |
valid-v2-legacy | legacy | A manifest-v2 bundle (before sealed roots, 2026-09-30): the three root checks cannot run and say why; nothing fails. | verified offline · accepted, partial | |
valid-pre-manifest | legacy | A record from before evidence manifests (2026-09-20): manifest, key and signature checks cannot run; the record digest still folds to its day's root. | verified offline · reject | |
incomplete-anonymous | incomplete | The same record as downloaded without a licence: the subject and the leaves are withheld, so those checks cannot run - never a pass. | verified offline · accepted, partial | |
stale-unsealed | incomplete | A record whose day has not been sealed yet: the seal and timestamp checks cannot run. | verified offline · accepted, partial | |
tampered-manifest-bytes | tampered | One space appended to the manifest body: it no longer hashes to the digest that was signed. | not verified · reject | manifest_hash |
tampered-signature | tampered | One character of the signature changed. | not verified · reject | signature |
wrong-key | tampered | The bundle carries a different public key under the original key id. | not verified · reject | key_id signature |
tampered-subject | tampered | One character of the sealed subject changed: it no longer hashes to the record digest. | not verified · reject | subject_hash subject_binds |
contradictory-report-id | tampered | The bundle says it is a different report than the one the sealed subject names. | not verified · reject | subject_binds |
tampered-merkle-path | tampered | One sibling hash on the path to the day's root changed. | not verified · reject | merkle_path |
timestamp-of-another-day | tampered | The .ots proof of another day is attached: it timestamps a different root. | not verified · reject | ots_commits_root |
tampered-lineage-leaf | tampered | One business fact's count changed. | not verified · reject | lineage_root |
duplicated-experience-leaf | tampered | The fetch whose leaf sorts last is listed twice. The Merkle root is IDENTICAL (an odd node pairs with itself), so only the leaf count can catch it; the body also no longer matches its signed digest. | not verified · reject | manifest_hash experience_root |
extra-decision-leaf | tampered | A decision the scan never made is added to the leaves. | not verified · reject | decision_root |
missing-experience | tampered | One fetch removed from the manifest body (the body then no longer matches its signed digest either). | not verified · reject | manifest_hash experience_root |
receipt-valid | receipt | A remediation receipt signed with the observer's published key. | verified offline · accepted | |
receipt-tampered | receipt | The verdict of a signed receipt changed. | not verified · reject | receipt_hash signature |
receipt-foreign-key | receipt | A doctored receipt re-signed with someone else's key. Every check a verifier can run offline passes; its key id is not the published one. A conforming CONSUMER must reject it by comparing key ids with the directory. the receipt's key id RBj_N824fPZv7FrIuMSNHFizOevhnUneNWoQNqz2pDs is not in https://crawlcheck.io/.well-known/http-message-signatures-directory | verified offline · reject |
Two more suites
The bundles above test integrity. These test the two questions an agent asks next: may it act on a decision receipt, and was the receipt signed by the right key at that time. Each case states the exact expected answer; every one of our verifiers (the script above, @crawlcheck/sdk, crawlcheck for Python) runs them on every change.
| Suite | Cases | What it settles | Files |
|---|---|---|---|
guard-acceptance | 64 | may an agent act on this decision receipt? Every guard must refuse expired, future, over-lifetime, other-domain, other-action, replayed, wrong-operation, tampered, foreign-key and wrong-purpose receipts, and the redirects that leave the receipt's domain | cases.json · runner |
key-policy | 26 | which key may sign which document when: the purpose a document needs at its signing time, keys before their start, retired keys inside and outside their window (a rotated decision key included), compromised keys, the bound exception during a switch, foreign keys | cases.json |
a conforming verifier returns exactly the expected value (true / false / null) for every listed check and the expected overall verdict; null means the check cannot run and must say why, never pass.
Verified is not permission. verified is the offline result: every check that could run passed. It is never permission to act. Rely on a document only when a verifier given the published key directory (published_kids, or the live directory) returns accepted: true; accepted_strict also requires that no check was left unable to run. Act only when, in addition, the decision allows the action and the record is fresh enough for it. Each fixture states the four values (integrity_valid, issuer_trusted, accepted, accepted_strict) under expected.with_key_directory in the index, and run.mjs checks them.