Findings · 2026-08-26 · By VSNARY | Emmanuel Orta · 0 views

Our audit labelled one page and measured another for weeks

Every scan of a specific URL actually measured the homepage. The report header showed the right path. The numbers underneath belonged to a different page.

Scanning /tree-removal/ returned 72,818 bytes, 18,856 characters of visible text, and 13,022 bytes of JSON-LD. So did scanning the homepage. So did scanning /emergency-tree-service-denver/.

Byte-identical results for three different URLs. That cannot happen unless the same page is being measured three times.

The bug #

The deeper page analysis was pointed at the domain root and ignored the scanned path entirely. Payload size, visible text, JSON-LD and the rest of the page-level figures all came from /, no matter which URL you asked for.

The report header displayed the path you requested. It was correct. The numbers below it were not.

Why it survived #

Because everything was self-consistent. The header said /tree-removal/. The path field in the record said /tree-removal/. Nothing errored, nothing was null, no exception fired. A report that labels itself correctly looks verified.

We only caught it by comparing two URLs' numbers against each other rather than checking that one report looked right. Identical byte counts for different pages is a fact that cannot be explained away.

What it looked like fixed #

We verified the corrected figures independently by fetching both pages by hand: 73,110 and 61,627 bytes, 12,846 and 5,039 bytes of JSON-LD. The small deltas are compression and dynamic content. The point is that the two pages are genuinely different by about 11 KB, and CrawlCheck had been reporting them as identical.

Grades diverged too — A/94 for the homepage, A/91 for the service page, where both had previously returned 94. And the service page surfaced entities the homepage never had.

We corrected the check, and added a test that runs before every release so this cannot return unnoticed.

Why internal consistency checks could not catch it #

Two measurements that share a defect agree with each other perfectly. Every figure this bug corrupted was derived from the same wrong fetch, so any check comparing those figures against each other held. The only cross-check that works there is one whose two sides are independent — two different URLs, or the tool's output against a fetch made by hand.

The same shape elsewhere #

Once you have a name for a failure you start seeing its shape: a label that agrees with itself while the data underneath belongs to something else. The viewport case is the closest cousin — a mobile section reported a missing viewport that was present with unquoted attributes. An extractor that finds nothing is indistinguishable from a thing that is absent.

The part worth taking away #

This defect also explains an anomaly we had already published and withdrawn. We had reported that a page’s highest-salience entity was a city rather than the business, and flagged it as untrustworthy because the same numbers appeared for three different URLs. We assumed a caching problem. It was this instead: one page measured three times.

The general rule is that a check proves what it compares and nothing else. A self-consistency check proves the record agrees with itself. None of them proves the number is about the page it names, and the only thing that does is a second measurement that could have disagreed — including, when the store is the thing in question, reading a write back.

If you use any page-level audit tool, there is a cheap test: scan two genuinely different pages on the same site and compare the raw numbers, not the verdicts. If the byte counts match, the tool is not reading what it says it is reading.

Questions this post answers

How did the wrong page get measured?

The deeper page analysis was pointed at the domain root while the report header printed the requested path. Every path scan measured the homepage under the correct page's name, and the label agreeing with itself looked like verification.

How can I test a page-level audit tool for this?

Scan two genuinely different pages on the same site and compare the raw numbers, not the verdicts. If the byte counts match, the tool is not reading what it says it is reading.

Related findings

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works