CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

91

sealed days

43,119

reports under this root

#975,121

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-11-08

Confirmed in Bitcoin block #975,121.

Merkle root
b0f72e053528f20bda04ee76c75e6b2ca0ba3607cb208096e3b5c09ddecbd68c
Reports covered
43119
Sealed
2026-11-09T01:17:39.152Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #975,121

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d b0f72e053528f20bda04ee76c75e6b2ca0ba3607cb208096e3b5c09ddecbd68c crawlcheck-2026-11-08.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

91 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 43119 digests sealed under this root

All 43,119 digests as plain text, one per line; the first 20:

000044c95eeb1093ab88850a4e5df1487566fbf76e194844ba7ea172b1448ec7
0006409a3a1adddf51b1a813d2f190a4ea22a230048ad46a465a5249a6d7d2dc
0008a4a01f1504ee2158763bd059fd0c8c17f11d0011fbb9cbf210d86379d081
000b04a649e39734052ecb8e6859e353934ca9b0981b0fae069ffe646ef834a9
000b41c088e782b00ab3104f64d29ab747fdaa4b4169ea7a9afdaad15f514b83
000b5751255cf3f524ef8a7689c1cac02921c03a1686ac316fb2d09011dcbeea
000e6910bc3bc4612f005f8d28a4efd23a7dbc451ba348a750d0db27101775d5
000f407cc0c8f477cf6d4c69ad8a208ff89989e030c504c4404ee5fd0605fed7
0010de3dfa03de7a166d94c38f6623a8d393c4390ee65eb5da457cca6503a04a
001157ee441eac6fd0544d70df95076cafc26007ed51f09c6c73be1718800752
0013950a21888c3fa495e135701238db1fbdcfabcfb7d8f134e38116f53a6829
001432007a60b7693341e9b770a58cdb8ce45a16fbae26777d64f8eaf121f7d7
0014ed59a4dfc43eb166be2886a3a3698f2c1f2b9ea10176fd7d09a81746388f
0014f34baca3314b62a67aaa65b55155e2a35a79340868eabe70a1c423159eba
0015399cf79f14d23663d7a90c8ab3d2405e433a014976300b847014bd5901dc
001665189b2a9226938f89de763e87a48b55a2fda39aba20ddd45f2a31155874
0016d3fbb44d945ca5153f1b394303f5b2a9bf467c4f35d21e1becb53880bc6c
0018fb1f4c4b1acda70d56a4a0823e9852cde063eaddccfe3fd0901feec5239a
00199dbbbd5d96227da354bd3efdfb934f160324d9f79931964bb910ab63fe9d
001c707f1d0624c7a603766ad1b1dfed4893529c63ced20073ea5bd31f92dbcb

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.