CrawlCheck

Library · Releases

Releases

In one sentence

Findings are short case studies written from real measurements — a robots.txt that answered 200 with a challenge page, a site whose GPTBot traffic came from one address wearing seven crawler names, a homepage that was two percent readable text, a contact email that bounced for one published character — each with the bytes that showed it; a business found with a defect is named only after its owner has been contacted.

What CrawlCheck measures now that it did not before.

107findings
66guides
12topics
9,163scans behind the live figures

2026-11-02 · Releases1 view · 2 min

CrawlCheck in German, French, Spanish and Portuguese

The main pages and every scan report are now served under /de/, /fr/, /es/ and /pt/. What is translated, what stays English by design, and how each page declares its language.

Read the measurements →

2026-11-13 · Releases2 min

AI crawler IP ranges from 18 operator lists, in one file

Checking whether a crawler is real means comparing its address with the operator's list. Those lists live at 18 different addresses in several formats. We now mirror them in one file.

Read the measurements →

2026-11-13 · Releases2 min

Release notes: 7 to 13 November 2026

What changed on CrawlCheck this week: ready-made block rules, operator address lists in one file, a corrected crawler catalogue, and four fixes to our own measurements.

Read the measurements →

2026-11-02 · Releases3 min

Page builders drop the main landmark: nine HTML checks

Two of our own sites had no main landmark on any page built with their page builder. The nine structural checks a screen reader needs, and how to run them from the delivered HTML.

Read the measurements →

2026-11-02 · Releases2 min

Release notes 29 Sep 2026: finding states, sitemap, trust

What changed on crawlcheck.io on 29 September 2026, and where each change can be seen on the live site.

Read the measurements →

2026-11-02 · Releases3 min

Release notes 3 Oct 2026: Resolve, census, rulebook

What changed on crawlcheck.io between 29 September and 3 October 2026: a signed answer AI agents can read before acting, the first agentic-web census and a correction to it, a public rulebook, receipts, disputes and independent observers.

Read the measurements →

2026-11-02 · Releases2 min

Share of preflight, October 2026: 0 of 5,643 requests

Of the AI agent requests that seven connected sites verified this month, none arrived with a CrawlCheck receipt. This measures CrawlCheck's own uptake, not the web.

Read the measurements →

2026-11-02 · Releases3 min

Release notes 14 Oct 2026: doors, weekly figures, trial

What changed on crawlcheck.io on 14 October 2026: the site reorganised around who is asking, every live figure gained a signed weekly edition, Watch gained a free 30-day trial, and the defects found on the way were fixed the same day.

Read the measurements →

Questions about this page

QAre these real sites?
Yes, every finding is a measurement taken on a live site. Domains are not named unless the author owns them.
QHow is a finding different from a guide?
A finding reports one measured case and what it showed. A guide explains how to run and read the check yourself.

Keep reading