CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

91

sealed days

10,668

reports under this root

#971,956

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-10-17

Confirmed in Bitcoin block #971,956.

Merkle root
e6cd2d1690a53e16e61daf2dc197b4cab5a96f84873ad2a3795342a9d504f845
Reports covered
10668
Sealed
2026-10-18T05:43:25.898Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #971,956

Download the .ots proofOne report’s receipt as a single file →

Sealed late. The scheduled seal for this day did not complete, so this root was sealed on 2026-10-18. The anchor proves these reports existed by then, not by the morning after 2026-10-17.

Verify it yourself

ots verify -d e6cd2d1690a53e16e61daf2dc197b4cab5a96f84873ad2a3795342a9d504f845 crawlcheck-2026-10-17.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

91 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 10668 digests sealed under this root

All 10,668 digests as plain text, one per line; the first 20:

00047548d9805e743ed65754b1d6185d31993efeedc586f471e16cf3eeb927c7
0006dd104202634d165b16568b09b0f3157f12ce4c4d233a4b619f5f1d3cce50
0008ddbd54eb201c9a31ada4c8f0c72a58f4efad15c69f8f12df529ce1dc3d67
00148a575cf74d8a6f04abee85823131d39adcf8d74f33f759f41ded92e49695
001affb0b1743191aa490567a495257efb7d66249435b290048fa7e0a513d972
001efa879a13b7d1ae4583256afd7b459d0f1b361ef2e7b806c1248ddac9070f
00307e0809925c8d260158044ce37aa912c1db943fc9f4ebb72eb0cd85095544
00347b1003e18482972deb326ca9e38a98432ff90ba48b286f9e02851f2f774b
0036f781b111f748634c6825c0e7e449d1bb73edb5ff64683cea92509538a7b3
003cb90159856297b960c10131280daf90e5be1c8b3742d717763c52e875c83f
0042921bdd81ab74ca9f12fd4c39d6cd6804bfa6bf36a2de1f2af55297b23ab2
0047322e9766995f06a335f557ebeabfaeabcfe6dfd9c4f3d76441504e430716
00489db53d589aa78950ab3acb5b326d70fdcb2a2629ecbaefba52d6766346ad
004d5fa3518343600610ee241874f92817726af6fbbd456ce79622b9d3115b2f
00517aea6c0eaa638dea49d615331e69fa801a42912a724a62195fdde636d956
005e6a815808f7c45112f17e847a30419397ae6b880b222d3c9f6e9489e8c96b
00636a12dc5835e263d19e091f9442b1cfe5a8b3a927b1de6618c4ca7191f049
00645679cb7d8d46be21507369626c6a502909a90d393122fab6a59f4c17ca85
0065c560b494ae0c6ab817e4ef0b52dd62c5a32b1689dc20348f90c294c934b6
0073f56cc9eb4d9dd0e26212b3022cd14d0227f99ce2cd2d997285215db2433c

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.