CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

91

sealed days

16,147

reports under this root

#974,471

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-11-03

Confirmed in Bitcoin block #974,471.

Merkle root
39a9d35f62e155fe3588b6bb7afd98e1462c31f86fc6f4e4ee13ddf490cec66b
Reports covered
16147
Sealed
2026-11-04T01:17:44.518Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #974,471

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d 39a9d35f62e155fe3588b6bb7afd98e1462c31f86fc6f4e4ee13ddf490cec66b crawlcheck-2026-11-03.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

91 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 16147 digests sealed under this root

All 16,147 digests as plain text, one per line; the first 20:

0000b87772f9a59fed29427d5d4249d5804ec5aff306757ef5e6748f65258c3c
0001decc0b7d5a2732ee892ed5895452ecf3d58d85c900c93f8627382bd91329
0002c9b6228404837b404956de3bb50ef675b4ca7b5a1da9a8bb6c986efe31a5
0007116d9350ed4e8cb130fc5118d0ecc126316cb666676418b07f1d56cf0727
0007cd5a5a43aec18dbea9ecf05b33b112dbd6069e3d50ea2089ba81c3b75abe
000ceae153256b404e401e498a0c03be53a0a7507ddaca2cb51b39f0abbf24cd
000d6b78e024f02fa1e34a3c43bec385156d18d8949b21c6f0e2bde6dd7ac1db
000f9ae8ee51cb1af6e5d1e05ab9e5004453e7054acee908fac0e6d557b9c800
00181424f4e7ec2fde809f388853a75937fdab0f4c9fffb8b71ca2a1271d4707
001a536c646838c05fee33b86b98e898e226a7575b5bbcbf6370f27e0d6ce923
002137299d1082741525138dbcadf447453deac688ff7622b125b435fcc59f35
0021e552603a1abb8ac40cd60bc7d150248dc5ec6b3ed6fe3ff327415477e46b
0025fdad7348caf3e628749384781a30c438ca1621931c24e63627b314e6805d
002a0f4332fc6560491ea954e7c0d24d23624668fbcf60c48d831909b04409e9
002e34c16c2f14062a7746567682953651774ecd01aa1f0cc46b44640fb5dfb1
002e868316584fbea586dde14a75a3402114031d6447ea43d98d983e6e4a1483
0030ca35f09c9bc65314375aacf1f06dc01895b4d618f7eaff73467c5f46da14
00317b86616cdb8f7a735496a1154f9f9f361d51bbfe785950c99a79828be5ea
0033602fee92ab2ee94667de4297352c138de27d6dc11d367b0b1457f2eb37ae
0040e11cfa5f3176933faa6155b9ebff9ffcd85715a4df14f3851a71d3f8c765

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.