Findings · 2026-09-23 · By VSNARY | Emmanuel Orta
Sites measured is not people: what our own logs showed
The sites-measured figure on our homepage is true, and it is not demand. One week of our own request logs shows who visits a site about crawlers: mostly crawlers.
The sites-measured figure on CrawlCheck's homepage counts distinct domains with a measured record, most of them scanned to build the benchmark population that every finding rate on the site rests on. It is not a count of people. In one week of the site's own request logs, scans from anyone other than the operator came to about six, from five places, while the busiest page, pricing, was fetched thousands of times by SEO and AI crawlers. The figure stays, labelled as what it is, next to a count of visitor scans.
The number on the homepage is counted to completion: the distinct domains that carry a measured record. What it is not is people. Most of those domains were measured on purpose, so that every finding rate on the dataset page has a population under it. A rate means nothing without enough sites behind it, and that is what the figure is for. Under a hero headline, though, it reads as demand. This note corrects that reading.
Live, as you read this: the corpus now holds 503,711 domains across 6,406 scans. The figures in this piece were measured on the date above; this line is not.
Who fetched the site in one week? #
We read seven days of the site's own request logs, page views only. The site that measures crawlers turns out to be visited mostly by them.
| Path | Requests, 7 days | Who |
|---|---|---|
| /pricing | 9,485 | SEO and AI crawlers, almost entirely |
| / (homepage) | 2,608 | mixed; the only page with a large human share |
| /demo | 425 | the demo report |
| each blog post | 45–60 | nearly identical on every post |
The pricing page was the busiest page on the site by a factor of three, and almost no human saw it. Every report and registry page links to it with the scanned domain in the query string, so crawlers walking those pages follow thousands of variants of one page. The variants carry a canonical to the bare URL and are served from cache, so the cost is close to zero and it is not a defect. It is just not demand.
How do you tell a crawler from a reader in your own logs? #
By shape. A post people read has a spike, a tail, and a count that differs from its neighbours. Forty-five to sixty requests on every post, regardless of age or subject, is the signature of sitemap walkers. If your analytics show every page with roughly the same count, you are looking at crawlers. Before counting any of them as a named crawler, check them: claimed hits are not verified visits, and a missing referrer is not direct traffic either.
How many people ran a scan? #
| Scans, 7 days | Count |
|---|---|
| Successful scan requests | 557 |
| From our own browsers and tests | about 545 |
| From anyone else | about 6, from five places |
That is the honest denominator for anything this site says about itself, and it is the same discipline CrawlCheck applies to everyone else: a count that mixes the operator's own traffic with the public's is not a count of the public.
What changed? #
The sites-measured figure stays, because it is true and because the benchmark population is the reason the rates on this site mean anything. It gained a neighbour: scans by visitors in the last thirty days, with our own traffic excluded. When that number is small, it says so. It was not the first figure here that was correct and misleading at once; the first was a cohort percentage that only means something with a second day under it. The fix each time is the same: name the population, or do not print the number.
This is one of the corrections CrawlCheck publishes when its own output was wrong. All of them are listed together.
Share this finding:LinkedInXEmailFollow Emmanuel on LinkedIn
Questions this post answers
Is the sites-measured figure on the CrawlCheck homepage wrong?
No. It counts distinct domains that carry a measured record, counted to completion. Most were measured to build the benchmark population behind the finding rates, not scanned by visitors.
How many people actually used CrawlCheck that week?
About six scans came from addresses that were not the operator's, from five places. Nearly all other successful scan requests were the operator's own browsers and tests.
Why does a pricing page get so much crawler traffic?
When every report links to it with a different query string, crawlers follow each variant as a separate URL. A canonical to the bare URL and caching keep the cost near zero, but the requests are not demand.
How do you tell a bot from a reader on a blog?
Shape. A post people read has a spike and a tail and a count that differs from its neighbours. Near-identical counts on every post regardless of age point to sitemap walkers.
Related findings
The main product
Found this on your own site? We fix it for $100.
Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.
Free browser extension
Check this on any site you open, without leaving the page.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.
Write for the library
Measured something yourself? Submit an article.
Send your finding, the data and the method. A person reads every submission and replies by email; accepted articles are published under your name with a link to your study. Free to submit, and placement is never for sale.
How it works · a reply within about a week
Spotted a mistake or a bug on this page? Report an error. Confirmed errors are fixed and logged on the corrections page.
Comments
Comments are read before they appear. Nothing is published automatically, and no account is needed. An AI agent with a published signing key can post without the queue: it signs the request and the signature is checked against its own domain (how). Those appear at once, marked verified.
Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.