Findings · 2026-08-15 · By VSNARY | Emmanuel Orta · 0 views

Site audits go stale: the day you audited it is not the finding

Measuring the same sites every day produces a number a one-off audit structurally cannot: how many were broken at least once. The gap between “broken today” and “ever broken” is the case for a series over a snapshot.

Every audit report answers one question: what was true at the moment we looked. The defects that make that answer misleading are the intermittent ones — a cache that serves stale robots rules for an hour a day, a deploy that briefly drops a header, a rate limiter that refuses crawlers only under load. Check on Tuesday and the site is clean. It was broken on Sunday, and it will be broken on Thursday, and no Tuesday audit at any budget can see either.

The measurement

This instrument scans a fixed panel of sites every day and keeps each day’s measurements permanently. That makes two percentages computable over the same sites and the same window: how many carry a defect on their latest reading, and how many carried one on at least one day. A single crawl — anyone’s, including ours — can only ever produce the first number. The second needs the sites to have been measured before the question was asked, which is why nobody quoting a one-off audit can produce it.

The exclusions, because they move the number

A site added after the window opened has had fewer chances to be seen broken, so late joiners are excluded from both percentages rather than allowed to drag the “ever” figure down. A site whose latest reading carries no findings count is excluded from both denominators and counted openly. And the whole table refuses to render below its own publishability gate — with too few days, “at least once” and “right now” are the same set by construction, and publishing them as different numbers would be theatre.

Those gates are named constants rather than numbers buried in a function, because each one is a claim about when a statistic becomes honest and each is worth arguing with directly. The window must hold at least two days. At least twenty sites must carry a usable reading, because a percentage computed from a handful is a number that gets quoted back at you. And the gate travels with the numbers rather than being checked by the caller: the function returns publishable: false and the reason alongside the figures, so a surface cannot render them without also receiving the reason they are not fit to publish.

Which defects are intermittent, and which just sit there

The headline gap says how many sites have a hidden day. The more useful question is which defects hide, and that is a different computation: for every finding code, how many sites were ever seen carrying it against how many carry it on their latest reading. A large gap is an intermittent fault. A gap of zero is a defect that has simply been there the whole time.

The two behave nothing alike. A missing sitemap does not come and go; it is a fact about a site until someone changes it, and “ever” and “now” are the same number. A stale cached copy of a machine file, a challenge page pinned at the edge, an origin refusing datacentre traffic under load — these are the ones with a gap, and they are also the ones a customer is most likely to be told do not exist, because the auditor looked on a good day.

That per-code table is available as JSON. It is deliberately not rendered into this page: it costs one store read per record, so it is served on demand, where the cost is paid by whoever asks for it, and it is capped — with the cap and the number of records actually read reported in the response, because a rate over an unstated sample is the failure this whole approach exists to avoid.

A cached number is a Tuesday audit

The same failure mode applies to statistics. An answer engine recently summarised this site and quoted our public scan counter — accurately, for the day it had read the page, which put it about 140 scans behind the live figure by the time the summary was shown. Nothing lied; a snapshot aged. It is the whole argument of this post applied to a single number, and it is why every figure on this page is recomputed from the live record at load time and why the citable unit here is the URL, not the quote.

The same reasoning governs what this site publishes about itself. Counts that appear in product copy are read from the thing they describe at render time rather than typed — a hand-written figure drifted from the scorer it described across several builds before that rule existed, and it was true on the day someone wrote it, which is the point. A number that agrees with itself is not the same as a number that is still true.

What this sample is, stated plainly

The panel is seeded and submitted — sites we chose to watch and sites people asked us to scan — not a random sample of the web. Quote it as “among the sites CrawlCheck measures daily”, nothing broader. No scanned domain is named here or anywhere else on this site.

One more limit worth stating, because it bounds every figure above. A defect is counted from the findings recorded on a day’s reading, so the series can only see what the scanner could detect on that date. When a check is added — and several have been — sites that carried that defect for months first appear to acquire it on the day the check shipped. The window figures are therefore a floor on how much was broken, never a ceiling, and no number here should be read as a trend in the web’s health rather than a trend in what we can see.

Three intermittent defects the daily panel has actually caught #

A robots.txt that alternated between a real file and a managed replacement as an edge setting flipped. A challenge page cached in place of a machine file for the length of one TTL, five minutes in which every crawler was told the file did not exist. A homepage served from an edge copy nearly four hours old while the origin had already changed. None of the three would survive a single audit; each is visible only because the site was measured the day before and the day after. The per-site history that makes this possible is the dated series, and a difference between two of its points is a change receipt.

How to build the second number yourself #

You cannot buy it after the fact, which is the whole argument. Start recording today: fetch your machine files and homepage once a day, store status, byte count, content type and a hash, and keep every day. After thirty days you can answer how many days was it broken, which no one-off audit can. How to measure AI visibility describes what to record; the purge that returned 200 is the kind of thing the record catches.

Questions this post answers

Why is a one-off audit not enough?

It can only report the state on the day it ran. It structurally cannot say how many sites were broken at least once, which is the number that describes reliability.

What is the gap between broken now and broken ever?

Measuring the same sites daily produces both. The difference between them is invisible to any single crawl, at any budget.

Which exclusions matter when reading that number?

Sites that joined part-way through the window are excluded, because counting them would shrink the ever-broken share for a reason that has nothing to do with the sites.

Related findings

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works