CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

90

sealed days

16,729

reports under this root

#973,001

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-10-24

Confirmed in Bitcoin block #973,001.

Merkle root
8092e40236659ae953698cf11c85d26ff45302b9f8b1ddc8ffe6f6e74225f5b1
Reports covered
16729
Sealed
2026-10-25T01:17:15.077Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #973,001

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d 8092e40236659ae953698cf11c85d26ff45302b9f8b1ddc8ffe6f6e74225f5b1 crawlcheck-2026-10-24.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

90 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 16729 digests sealed under this root

All 16,729 digests as plain text, one per line; the first 20:

000c2e735a8f049f28ab001dc6ec8a1d4d11c45d0b16e22920f6fbf901d0c47a
0012e03c9885482829c8bf920b3ed290f9b66a4310b64a2207d037d18f728cf4
0013adbbda9fb163ba4f9287830837d2bc8bdffa915e382507f94fd409f11b94
00192e9703bbdef2aca843b0a91086940baef6c84aaff8709f179c5026a1c687
0019ebc4fd9d75b5e6acc6e2f5acedd902293aa993f9e7562a15ad6045d06c17
00279b6cca4d94079fb655ffa3097a8964921b5de526371844b80c30288e3efd
0028c5792b1cb96e09a247ae4a2652a49be0bc4026c84ab42dd236fb5d41b240
002a621029c9c32011b5c72371635b5c0847ab3a18288401390d45ffd274d21e
0038416d69413173416765c99ce7f6e5c1ad970e9014561de566b4d9b5a96da1
00385d926000e0401dec9b006da767295351006e580910233fc8482c7d2a0052
003cc99cf768b1ff0a60061ee4e9ab8129e85a4c7db8c196fc51968ee09d446a
003d199b8d1d0457a7e4212c75654f51b8ebd5fc130b9e44a9274cfff919fe49
004840c9e1f0f82f4acc21dce7ef6943fdb98040a4a69bee171e02a5dbb22d9f
004c5028ee67b24b840fad8616cb83f9f94b98ec66106b020e0782574e75e4cf
0055237fdb048c0dc60056355bed3c8870c7a9e9ce72f0fa36dc3bc0def67ad9
0055bf498b2be0f25223c60c0a8fe34134d4408e395a5a497f745c1b4972567d
0056cf21135107c544708543a42539e6d4eb8385e22bf47eb8aec2bee0e45ff4
005fe299008fb6c906d6e5c60035b4fa7265e9ee218eaaf977ca5ae4bbab8ad3
00619e89d3a4446c0838ecb3cad861a341c6a9473e3e98a09f8414860d067ee1
0063ca84d608e0f69da1adfa7d6bdd4a85dbbfcd1341623f06ea0ddae2e76a69

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.