CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

91

sealed days

16,086

reports under this root

#975,020

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-11-07

Confirmed in Bitcoin block #975,020.

Merkle root
a4b420d1fe0efe3196bd1a8e3e84ec0530ae6ed441e9f3980c1944edbd640138
Reports covered
16086
Sealed
2026-11-08T01:17:15.894Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #975,020

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d a4b420d1fe0efe3196bd1a8e3e84ec0530ae6ed441e9f3980c1944edbd640138 crawlcheck-2026-11-07.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

91 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 16086 digests sealed under this root

All 16,086 digests as plain text, one per line; the first 20:

00008bf405224ada076763466834fbdb7bddc5ff167ea2fa8e41de1fb883aed4
00061f941bec7b60806b7162bde3a3f6d09495f960cb28a1bd401d33c965a758
00081d99e804223eaaddb7f7775a1f51def41ebcfcdf0bb464ca6ee4a575a191
000c5695cd14550b05a431f004dcc24be4250eec82173d7e7123ab20f54e1497
00147adbdad501937d46a92d2150a3efcad33a67affedf65c9813893c139223c
00176d35104379cd4c01a71769252fff1db3d8feccc520ed5f1621e8e5203492
00181a46b8cd85127a88a796a2654b4d6e8e5ae1ef596d78c079ac29651576f3
001966d8c905978a5c4da04018b1da5272b51e7844a821bfe63635c7afb445fb
001a5073ab72af40c0f1341071f6452eb0ddbe37f194af22f5a5056128f34a97
0024347e49db6dd42ec9e40e3b204a4fffcd1ad5e061b71f67284140b2a97d6c
002639a888d99f7222180a262aa1bdeefe329bfd5faa518f7748aef005aa57c8
00297aa62649011c428f0ade08df1729c7bd14aecced709cf3d8702b5fc57863
002a9ff69289c8d0cc4ddbe02a8138a906df2554a65863aac7c9eef2a1c43c5c
002d42cd3b484f9d0ccc2e25789041a9ce207c330820e41ea765adf5f5f589ec
002dde57c807ceaf6449f37727b9595de787a708310600ae6c3ccedc7c4e2ecc
003347e538bf6416ee4ca312709c3abbcd6dce45fc079bb15ca5af84bc551f91
0033806dfa561b27a2c6bcbce8cdd97c483758c64c556fc9828f2008f1c5f4db
0037a437536b220dba1d7f7b0ea3387c79898c80e88dd4df077cf903d42e4170
00414d30706c8e32e132cfb751b700e5c12dc4ce9d78c6d31a7a29414a6a67fb
00436a9b1e2e165ff9cc66b60359e8e0b016d41b86bca0da56341ea2f5868f31

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.