CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

91

sealed days

44,351

reports under this root

#975,368

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-11-10

Confirmed in Bitcoin block #975,368.

Merkle root
3d0315fb25478cf056d6a37a335e7ca46e28f86efbb9a57c8b83527ea40ce4be
Reports covered
44351
Sealed
2026-11-11T01:17:16.993Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #975,368

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d 3d0315fb25478cf056d6a37a335e7ca46e28f86efbb9a57c8b83527ea40ce4be crawlcheck-2026-11-10.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

91 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 44351 digests sealed under this root

All 44,351 digests as plain text, one per line; the first 20:

000120065b08b39663d6843c8519f42caf6160cb06042dc6a116ea3f47eb9ab1
0002f4ac07f833ba2af74121611ec043d35372115e00b25104ec45806134c131
0003cc2c665c522097addd8b039a5c7d186fb09d1b1ba375148cc2f9b8b94ada
0005092dc1581cdd8ee8630866deab1b7810191cf77e69353536e7a651d80adf
00051514c51df9387cbd91173e7da05d1fc121ba21c822d10014ef791e45ff2c
0006d926371c63b6b1b13824f0869c58e38bdf2cf74a096974b79873c937fc6f
0009131664c50adfe691ed12e212974bb4c47eb34c52df519634b41c47aeba9d
00097d3513297f86614a6cd708cb24d2853680eadcd7a2e57ce196fabc7bc0b7
0009e563aedcb079956d91efb06c11493827f003e05fb6979e6af1f24d8fe782
000b3b1bb450a5f9080e9ba5685cdc76350659d5aeaa0de7e45f986990fc07a7
000bf5988c3f264508dc7ccc06455ae9cbb0964664ac0c1207f6eab3c30b001b
000cf638fcab26c2fd84436bd2888b6bad83979d8db9d5df158d716d52d973b1
00119baf1dfdf09946c0774a057cf7b00d4a3b24915d582c997bafa647d2cb60
0015c600552eff05d652a7cdf6a1c65cdf1694a718bc3681baf96f9d1fab6d04
0017730fe66e049ed37943ea8cafa633658fbf0b676ba79fe421b5cf20b8020d
001a13e5ea970ef413c3ec5c48609560162385a9d7f8d28029959ea5e4112753
001aeeea5199db91785a3a7a4f4c7553ebf6ef9464a9f9514b8c9f8a5457c42c
001c6d7c615cebdeba6b7ccb522d483477994161e91f0fd168faa3b9174f47b6
001da687b0992873361cbe34a091eda6eb87d0d25d0263cf2da3a1f6be492331
0020e8e6662005d8d9861d2694b612d055f9742a20ca2644581118f090c21c7b

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.