CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

90

sealed days

2,024

reports under this root

#972,858

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-10-23

Confirmed in Bitcoin block #972,858.

Merkle root
31c11376aab7d05630139a56f0583a6420427877ef6a43ea539fc945f0eb0198
Reports covered
2024
Sealed
2026-10-24T01:17:14.449Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #972,858

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d 31c11376aab7d05630139a56f0583a6420427877ef6a43ea539fc945f0eb0198 crawlcheck-2026-10-23.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

90 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 2024 digests sealed under this root

All 2,024 digests as plain text, one per line; the first 20:

00171ac345a70f4312b31e545e40584f014e6e4926cbe3cc1032717b863f6b80
002997ae9caadf125e26b685294004175cce8db695d34df14d808bd31381b1ee
002e0bb3f79f2eec06092e02383acd60c216477cb8b8b62058fe371824b6b38b
003aa52e270fb3300810a73f64604dead3fd83c4157f49334b20d76a5143ac94
003d1f77c64d73e27f27a75bddbe3dd63e5477d40c6292ca3c7a3ed19a656d07
0066beb4432866aa76d2c9e4a73ce126dc0a39d7a0d4f929f934377f9a40b9fa
00825445aba6da86d1e8946cf16bb9d2c759d9a32abd4721a07381ed15d2150c
008ed156e161b56ed1ec53b8875be368ab5e063cfa5e5764572bec6929131cc7
00b8233533d4ec52790f6e8e6f9f0dcf9e1501ef744b86df5e9ac2b67e619481
00d3636523ca94940586b3ade13705044bf16b1a9386f369804a74ebc91c8825
00d382281deb180cfc633b08c9eaa66592a06ed4b7dd21ab099cd384af85beb0
00e59a18a47c0db61756db3f0cc8ad9e6e7d3c5fb127b7a162aecbeaaa82a9e5
00fc79962eba768b3bc2d4c6e732c35f6268cde7ff510ec9b7bd867b06bdb5b1
0128b53b91e8dee3934f891510302eb640e712a6256c26708ffb69c98a35c5fc
019f08e88b148b9102e7b855b5583901be42c0e77d277d62c99d1b9b512ed74a
01aba437e5f2240b83a0eba2aaf01dfdfa0cae10e1cc5d77d3f180f8790c62b5
01ae3f058354dc4cf1a4b7d8d819fa4e0871f7f8a25d56f79c9af98150460e2f
01c879bf7f9ac0d622acbdfb598d381f723a2a2bb0b036b662fea028deb8cb5c
01d7706621d14bde30df859aec42cb9593155a1569459b85045685aee7d63681
01f1659489a64f224357dc6ed69f3fc9675b5a735c507c08f9210931b712115c

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.