CrawlCheck
Proof

Proof an agent can check without trusting us.

An agent that acts on what CrawlCheck measured should not have to take our word for it. Every report is fingerprinted and each day anchored to Bitcoin; every answer an agent acts on is signed; every signed answer is entered in a log that a witness we run on GitHub, outside our servers, re-checks. Each of the three is checked on the reader’s own machine.

Proves

A report with this digest existed in exactly this form by that day, and has not been edited since.

Does not prove

That the measurement was correct. Only the first claim can be settled by mathematics.

91

sealed days

46,180

reports under this root

#975,235

bitcoin block

Three proofs

What each record proves, and how to check it

Each check runs on the reader’s own machine, from files that are published and can be kept.

1 · Reports

Sealed and anchored to Bitcoin

Each report’s measurements are hashed; each day’s hashes become one root, committed to a Bitcoin block through OpenTimestamps.

Proves the report existed in exactly this form by that day.

The latest sealHow it is built

2 · Answers

Signed with Ed25519

Every Resolve answer an agent receives before it reads, cites, connects or transacts carries a signature over its canonical content.

Proves the answer came from CrawlCheck and was not altered in transit or afterwards.

The signing keysThe offline verifierWhat an answer holds

3 · History

Logged, chained and witnessed

Every signed answer names its leaf in an append-only Merkle log. Each signed tree head chains to the one before, and a witness we run on GitHub, outside our servers, re-checks and co-signs it. It is our own workflow: it proves the log was not rewritten on our servers, not that an independent party checked it.

Proves an answer was not rewritten or removed after it was logged.

The log headThe latest witness

Method

How a report becomes a proof

No wallet, no fee, no key held by us. The proof is a file, not a coin.

1Reportmeasurements only2SHA-256one digest3Merkle rootone per day4OpenTimestampstwo calendars5Bitcoin blockpermanent
  1. 1

    Measurements only

    Host, timestamp, section scores, every path and status fetched, every crawler identity and what it received, the citation state. Wording and layout are excluded, so an editorial change never breaks a proof.

  2. 2

    One digest

    Canonicalised with sorted keys and hashed with SHA-256.

  3. 3

    One root a day

    Digests sorted and folded into a Merkle tree: parent = SHA256(left || right), an odd node paired with itself. Arrival order cannot change the root.

  4. 4

    Anchored

    The root goes to the OpenTimestamps calendars, which commit it into a Bitcoin transaction.

The seal

2026-11-09

Confirmed in Bitcoin block #975,235.

Merkle root
35cbce005e0a6aee3a975da5a108af3f8f3363d7bf570833a8d0a48022093ffc
Reports covered
46180
Sealed
2026-11-10T01:17:30.005Z
Calendars
https://a.pool.opentimestamps.org
https://b.pool.opentimestamps.org
Bitcoin
block #975,235

Download the .ots proofOne report’s receipt as a single file →

Verify it yourself

ots verify -d 35cbce005e0a6aee3a975da5a108af3f8f3363d7bf570833a8d0a48022093ffc crawlcheck-2026-11-09.ots

The reference client checks the file against a Bitcoin node or a public calendar. No CrawlCheck server is needed, or trusted. A fresh seal reads pending confirmation until the next Bitcoin transaction confirms; ots upgrade then stores the block attestation in the file for good.

Ledger

Every sealed day

91 days anchored. Each links to its own root and proof file. Days are sealed once and never recomputed.

The 46180 digests sealed under this root

All 46,180 digests as plain text, one per line; the first 20:

0001919ad8b6974ba05cbbf405efad3fb545b6cc4ae4901879abaf1aab4ad279
00037674778273c4ac9da1a6ef9c3297a8630fb7c03b3708c69c65cb85f827f3
0005865ba19f47d563c70ffc2c2413b10dc9d7c158dbd290104ffdd5710d4f34
0005865de56ba1a69f874c3fdcd8849d2b9b78c4cf21416f0d3599538eeab840
0007629e6385d6a8b0ebf9b1c4f84da70179c5a50bea89b3b00a9266a0a940ab
000a72c743bbff26f06e451de9391ce5788c364b51cf3836c0fb7f8c12af9197
000c9ff6bffa07c9b8b4b554c39aa74be27267ce7da4c6c9556b8bd7eb560c88
001314818ffbc65197a28daf0cf2bb6452804ef8126d8dd87b7baf49ccd1ac16
0013e0c805ca9ade7bb001b8cd400a1f259b317b0d3294b55a9e71feebf62936
001463b8ceafa31cbff802dc213bf48dee5863f7992369d56e9abd39e9ef1dc0
00148bea9819c9c1502660f75a31d5e68e096329599751eecf6bf3822f9f6fb1
0014d2331ddca0b8d8f819327fa56dfbe541b1fb370a427dffd012dec1b0cdec
0015c3f11fe11014ee78ff0042323e367bebcc72edf4ef7d39de2609b1776709
001618d3d2c9a113d10022a21d1877c11667284a6e20ba4cabc5e258d62d5ed6
00164ff1dca8fa9f5629b5ddcadc9b059441b6d436be841210d71026e9a8e394
0019dcff703cbabb4830c8099fca9fa1bf3d8ccd8b3306091f34c30990091e6c
001aa38fa8638b2d2016130056246e962febe07eb177d0bf397d33917f9f5526
001c316b6403fd87a9135adcf7ae3783bfb67f0649a7ec2b2c948e6777a9ec4c
001d2e08ef2b55c24ae63360da7273a11377df39c46cbbb9b496ad924788f826
001f795fe867f2419100e28a121e7f8d8507ec566a471a986276be74ebc0c741

Questions about this page

QWhat does the Bitcoin anchor prove?
That a report with exactly this digest existed no later than the anchored block, and that it has not been edited since. It is a proof about time and integrity, not about correctness.
QHow do I verify a report myself?
Download the day’s .ots proof from this page, recompute the report digest from its JSON record, and verify the proof with the OpenTimestamps client against a Bitcoin node or public calendar. No CrawlCheck server is needed for the check.
QWhy exclude wording and layout from the hash?
So an editorial change never invalidates a measurement. Only the measured fields are hashed; the words around them can be improved without the proof breaking.
QWhich days are anchored?
Every sealed day is listed on this page with its digest and proof file. Days are sealed once, after they end, and never recomputed. A day sealed within the last day or so reads as pending in the reference client: the calendar holds the commitment, and the Bitcoin block confirming it is usually mined within 24 hours. The proof file here is upgraded to carry that block header once it exists, so a pending file is a proof waiting on a block, not a missing seal.