CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

91

sealed days

16,913

reports under this root

#974,054

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-10-31

Confirmed in Bitcoin block #974,054.

Merkle root
fcef1b006e14f72d5360d7cf7b115b96140e87f272a1089bfc5b096ef6dd2e2f
Reports covered
16913
Sealed
2026-11-01T01:17:22.806Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #974,054

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d fcef1b006e14f72d5360d7cf7b115b96140e87f272a1089bfc5b096ef6dd2e2f crawlcheck-2026-10-31.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

91 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 16913 digests sealed under this root

All 16,913 digests as plain text, one per line; the first 20:

0003a309fb3a87a707ff04e2b4682059564020f965135da532bf368bf915ec05
000576f38ef19f6b9fe39cdb6b2ec470aefe4d97c519fd44caea7088976a3928
000580cbd0671d535c716b2c5965421ef02772568ee9fd24e1feaa1ee4e82020
00060cd687fa244211cd88d10b081e0c7a0e7527b8bfb2270c5d32c60f36d6a7
00074ee6df7d193bbf0bf094abd0e4092647a65d524f996b3f43f5ee50ea2316
0012f1d1cc87dd8bc2d9c137d8a62a4b3b18b30aec507bba00bed9ef91a72c7f
001411a2718064dcda7302de1850f7e1053ceed043a033cf6c22edffe3789103
00151bfb080a14128847d9d47aec638dd11707cf9db29fc345d53e805ec1b9ae
001a2f456f215b93374d687f718b3114ac399d4ae1d9e340f6da58a884d3fe7a
001ddbd12add0d1f6401666e43c98e40a788cf0739df5f619d3e7a83daf358eb
001f343e2fe80b73ed3de7bb1c490678c72cdf5bca81c65fa9cbecb5ffe59f63
0021d46a5622e0074a5312a72f43677d7a5f577b7d12e1cb4cca7626a5957364
0023ce2551bcaf29d12d9a04bc3ece8210ff1f52356a7d45b70d284c4a248cf4
00244ec53ab4de8790ba568c6b9c4c72f8fd721882a5d331b34f72dbdc2befaa
0033a6e5b147e7b34f7060bce3b9df6e909cb456adda525402cf9bdfa595937f
00398f7b83bd36b26f0bfe9f21b939db566dfe4424760e0248057d0ec5d2f688
003d419dfedb5c9d57de45ef1c01680328f891b94d501c18a20bacd88f608fab
003eba384d8e4cf1cc28d073b8f9100d880651f5754eadc234d471b2c0f0ddac
0049de61078e157746eac8849c728395aab65748b782e2d99e69f5f48839d0b3
005541848475b74bb659a79144405ddb232773744cf4be2fac8db9fe8b95087f

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.