Guides · 2026-08-28 · By VSNARY | Emmanuel Orta · 1 view

SEO audit tools compared: what each one actually fetches

Site crawlers, platform audits and Lighthouse are three different instruments pointed at three different things. The gaps between them are where defects live, and one of them cannot support a finding from a single run.

Every tool in this category produces a list of issues, and the lists overlap enough that comparisons usually come down to interface and price. The more useful question is narrower: what request does this tool actually send, and as whom? Almost every blind spot in an audit traces back to that.

Disclosure: CrawlCheck publishes this comparison and is one of the products in it. Every figure about another company comes from that company’s own pricing or documentation page, linked where it appears. Where CrawlCheck does less than a competitor, the page says so.

The blind spot they share #

Rows one through four all fetch as something that is either you, or a browser, or the vendor. None of them asks the question an answer engine asks, which is why two of the most common defects in the public dataset here are invisible to all of them.

Stale cache. STALE_CACHE_SERVED fires on 17.4% of scans — the single most common finding. It hides from testing because most tools append cache-busting parameters, or fetch from an IP that has a warm, correct object. A crawler requests the bare URL and gets whatever the edge is holding.

Identity-varied responses. ANSWER_ENGINE_REFUSED fires on 5.8%, and CRAWLER_SERVED_LESS on 0.5%. On one site, GPTBot and ClaudeBot received 502 while Googlebot and PerplexityBot were served normally, with robots.txt allowing all four. A crawler that only ever sends its own user-agent cannot see this. It is not a bug in those tools; it is outside what they measure.

The variance problem, measured #

Lighthouse deserves separate treatment because it is the most quoted and the least stable. Two runs of the same page, unchanged, produced a 225% spread in this measurement.

That is not a defect in Lighthouse — it is a lab tool honestly reporting a noisy environment, with contention, network conditions and throttling all in play. The defect is downstream, in every report and dashboard that quotes a single run as a finding. One Lighthouse run cannot support a conclusion. If a tool shows you a performance number without a distribution behind it, that number is one draw from a wide range, and the next draw could reverse whatever it appears to prove.

The correct use is repeated runs with the spread reported, or field data from real users, which measures a different and more honest thing.

The staleness problem, measured #

Nearly every tool in the first four rows is sold as an audit: a report on a day. Measured continuously rather than once, the difference is not small:

A one-off audit at any budget can only ever produce the first figure. The second requires the same sites to have been watched before the question was asked — a difference in kind, not in thoroughness. The day you audited it is not the finding. It is one sample of a system that changes with every deploy, cache purge and plugin update.

Which to use for what #

These are complements, and treating them as substitutes is the actual mistake.

Use a desktop crawler for structural work at scale: broken links, redirect chains, duplicate titles, orphan pages. Nothing else does that job as well.

Use a platform audit when you need the crawl joined to keyword and backlink data you already own, and when the value is the schedule rather than the depth.

Use Lighthouse for diagnosis, never for reporting, and never from one run. Google’s PageSpeed Insights adds field data from real Chrome users, the better number where a page has the traffic; what neither measures for AI crawlers is set out in PageSpeed Insights and Lighthouse vs an AI crawler audit.

Use a machine-layer check before any of them, because it answers a prior question. There is no point auditing the content of a page that a given crawler cannot fetch, or that is being served from a cache holding last month's copy.

The issue count is not the finding #

Most tools in the first two rows report a total: 487 issues found. That number is a property of the crawler's checklist and your page count, not of your site's health. Double the URLs and it doubles. Two sites with identical machine-layer defects can differ tenfold on it.

The defect distribution in the public dataset here is deliberately reported the other way — how often each finding occurs, with severity kept separate from frequency, because the two are independent. The most common defect in the corpus is medium severity and the critical ones are rare. A tool that multiplies frequency by severity into one score has thrown away the only two facts you needed.

What to read instead: which findings, how severe each is, and whether each is capped or excluded from the grade. A report that cannot tell you why a grade was capped will be misread — including by an LLM asked to summarise it, which will attach the cap to whatever sits nearest it on the page.

Where CrawlCheck is weaker #

CrawlCheck covers the last row and none of the others well. Its free scan reads one homepage and the machine files around it, and its deep crawl tops out at 500 or 5,000 pages by plan, so it is no substitute for a desktop crawler on structural work, for a platform audit joined to keyword and backlink data, or for Lighthouse on performance. Side-by-side detail: desktop crawlers vs CrawlCheck and schema validators compared.

The one question to ask any audit tool #

Ask what it sent and what it received. Not what it concluded. If a tool tells you your robots.txt is fine, ask whether it read the content type or the status code, because a challenge page returning 200 with HTML is parsed as no directives, everything allowed — and every uptime monitor and most validators will call that healthy.

A tool that reports what it received lets you disagree with it on evidence. A tool that only reports a verdict is asking you to take its word.

Findings behind this #

Every figure above comes from a measurement written up in full, with the date, the method and the raw numbers.

• We measured a 225% spread between two Lighthouse runs of the same page
• Our audit labelled one page and measured another for weeks
• Site audits go stale: the day you audited it is not the finding

Questions this post answers

What is the best SEO audit tool?

They are complements rather than competitors. Desktop crawlers are best for structural work at scale, platform audits for joining a crawl to keyword and link data, Lighthouse for diagnosis. None of them fetches as a named AI crawler or checks cache state, which is where the two most common defects in this dataset sit.

Why do my Lighthouse scores keep changing?

Because it is a lab measurement in a noisy environment. Two runs of the same unchanged page produced a 225% spread in this measurement. A single run cannot support a finding; use repeated runs with the spread reported, or field data from real users.

How often should I run a site audit?

More often than most people do. Across the panel measured here every day, the share carrying a defect on the latest reading is materially lower than the share that carried one at least once in the window — the live figures and the window sit on the site-audits guide. A single audit can only ever produce the first number.

Related findings

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All guides · The dataset · How the dataset works