CrawlCheck

Directory

SEO and AI-visibility tools, measured on their own sites

In one sentence

The CrawlCheck directory runs the same scan on each SEO and AI-visibility vendor’s own homepage that a customer gets — reach, read, quote, the grade and the findings — for 25 tools across AI visibility monitoring, content and GEO, enterprise SEO, SEO suites and site crawlers, re-measured roughly one vendor per hour with a dated ledger of what changed, exported as CSV and JSON under CC BY. Nothing is self-reported and no vendor is listed or removed for payment.

Every tool listed here sells some judgement about other people’s websites. This page runs the same scan on each vendor’s own homepage that a customer gets: can a named crawler reach it, can a machine read what it got, is there a fact it can quote. Nothing is self-reported.

25

tools listed

23

measured and graded

77

median score

11/23

publish llms.txt

0/23

publish an entity map

Re-measured on a rolling cadence, roughly one vendor per hour, so every card is under a day old. Index last rebuilt 2026-09-11 04:18 UTC. Highest today: Sitebulb at 85.

85
Sitebulbsitebulb.comSite crawler
A
Reach100Read76Quote74
2 findings · worst info✗ llms.txt✗ entity map
scanned 2026-09-10 · 6 on recordDetails Evidence
84
Rankscalerankscale.aiAI visibility monitoring
C
Reach95Read69Quote85
2 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-10 · 7 on recordDetails Evidence
82
Mozmoz.comSEO suite
C
Reach96Read63Quote82
4 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-10 · 11 on recordDetails Evidence
81
Lumarlumar.ioSite crawler
B
Reach96Read57Quote85
1 finding · worst info✗ llms.txt✗ entity map
scanned 2026-09-10 · 6 on recordDetails Evidence
80
Semrushsemrush.comSEO suite
C
Reach89Read69Quote78
3 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-10 · 12 on recordDetails Evidence
80
Athenaathenahq.aiAI visibility monitoring
B
Reach100Read76Quote56
1 finding · worst low✓ llms.txt✗ entity map
scanned 2026-09-10 · 6 on recordDetails Evidence
80
Goodiehigoodie.comAI visibility monitoring
C
Reach89Read68Quote81
3 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-11 · 7 on recordDetails Evidence
80
Writesonicwritesonic.comContent and GEO
C
Reach92Read68Quote76
2 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-11 · 9 on recordDetails Evidence
79
Surfersurferseo.comContent and GEO
C
Reach92Read55Quote87
4 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-11 · 9 on recordDetails Evidence
78
Ahrefsahrefs.comSEO suite
C
Reach94Read62Quote72
5 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-10 · 9 on recordDetails Evidence
78
Botifybotify.comSite crawler
C
Reach92Read63Quote75
4 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-10 · 9 on recordDetails Evidence
77
SE Rankingseranking.comSEO suite
C
Reach96Read57Quote73
5 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-10 · 10 on recordDetails Evidence
76
Screaming Frogscreamingfrog.co.ukSite crawler
C
Reach90Read48Quote84
5 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-10 · 10 on recordDetails Evidence
75
Frasefrase.ioContent and GEO
C
Reach93Read62Quote63
2 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-11 · 8 on recordDetails Evidence
74
Oncrawloncrawl.comSite crawler
B
Reach80Read76Quote63
2 findings · worst info✓ llms.txt✗ entity map
scanned 2026-09-10 · 8 on recordDetails Evidence
74
Seobilityseobility.netSite crawler
C
Reach76Read68Quote76
6 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-10 · 10 on recordDetails Evidence
73
Similarwebsimilarweb.comSEO suite
C
Reach84Read56Quote75
2 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-10 · 10 on recordDetails Evidence
68
Peec AIpeec.aiAI visibility monitoring
C
Reach100Read57Quote35
4 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-10 · 12 on recordDetails Evidence
67
Conductorconductor.comEnterprise SEO
C
Reach84Read49Quote63
6 findings · worst medium✓ llms.txt✗ entity map
scanned 2026-09-10 · 12 on recordDetails Evidence
66
BrightEdgebrightedge.comEnterprise SEO
C
Reach85Read56Quote51
0 findings✓ llms.txt✗ entity map
scanned 2026-09-10 · 11 on recordDetails Evidence
66
Profoundtryprofound.comAI visibility monitoring
C
Reach100Read50Quote35
2 findings · worst medium✗ llms.txt✗ entity map
scanned 2026-09-10 · 9 on recordDetails Evidence
62
Clearscopeclearscope.ioContent and GEO
D
Reach83Read55Quote40
3 findings · worst high✗ llms.txt✗ entity map
scanned 2026-09-11 · 10 on recordDetails Evidence
44
Sitelinersiteliner.comSite crawler
F
Reach44Read29Quote60
9 findings · worst critical✗ llms.txt✗ entity map
scanned 2026-09-10 · 16 on recordDetails Evidence
Otterlyotterly.aiAI visibility monitoring

Refused this scanner — not graded

scanned 2026-09-10Details Evidence
Scrunchscrunch.aiAI visibility monitoring

Refused this scanner — not graded

scanned 2026-09-10Details Evidence
Median score by category — ours, unscored, and a description of what each product is sold as
AI visibility monitoring 580
Content and GEO 479
Enterprise SEO 267
SEO suite 578
Site crawler 776
Table view — every row, for copying or comparing
ToolCategory (ours)Grade · scoreReach · read · quoteFindingsLast scanned
Sitebulb
sitebulb.com
Site crawlerA85100 · 76 · 7422026-09-10
evidence
Rankscale
rankscale.ai
AI visibility monitoringC8495 · 69 · 8522026-09-10
evidence
Moz
moz.com
SEO suiteC8296 · 63 · 8242026-09-10
evidence
Lumar
lumar.io
Site crawlerB8196 · 57 · 8512026-09-10
evidence
Semrush
semrush.com
SEO suiteC8089 · 69 · 7832026-09-10
evidence
Athena
athenahq.ai
AI visibility monitoringB80100 · 76 · 5612026-09-10
evidence
Goodie
higoodie.com
AI visibility monitoringC8089 · 68 · 8132026-09-11
evidence
Writesonic
writesonic.com
Content and GEOC8092 · 68 · 7622026-09-11
evidence
Surfer
surferseo.com
Content and GEOC7992 · 55 · 8742026-09-11
evidence
Ahrefs
ahrefs.com
SEO suiteC7894 · 62 · 7252026-09-10
evidence
Botify
botify.com
Site crawlerC7892 · 63 · 7542026-09-10
evidence
SE Ranking
seranking.com
SEO suiteC7796 · 57 · 7352026-09-10
evidence
Screaming Frog
screamingfrog.co.uk
Site crawlerC7690 · 48 · 8452026-09-10
evidence
Frase
frase.io
Content and GEOC7593 · 62 · 6322026-09-11
evidence
Oncrawl
oncrawl.com
Site crawlerB7480 · 76 · 6322026-09-10
evidence
Seobility
seobility.net
Site crawlerC7476 · 68 · 7662026-09-10
evidence
Similarweb
similarweb.com
SEO suiteC7384 · 56 · 7522026-09-10
evidence
Peec AI
peec.ai
AI visibility monitoringC68100 · 57 · 3542026-09-10
evidence
Conductor
conductor.com
Enterprise SEOC6784 · 49 · 6362026-09-10
evidence
BrightEdge
brightedge.com
Enterprise SEOC6685 · 56 · 5102026-09-10
evidence
Profound
tryprofound.com
AI visibility monitoringC66100 · 50 · 3522026-09-10
evidence
Clearscope
clearscope.io
Content and GEOD6283 · 55 · 4032026-09-11
evidence
Siteliner
siteliner.com
Site crawlerF4444 · 29 · 6092026-09-10
evidence
Otterly
otterly.ai
AI visibility monitoringrefused this scanner — not graded2026-09-10
evidence
Scrunch
scrunch.ai
AI visibility monitoringrefused this scanner — not graded2026-09-10
evidence

Take the data: CSV · JSON, with the report URL behind every score, under CC BY 4.0. Cite it as “CrawlCheck directory, retrieved <date>” — every row regenerates, so a link always shows today’s state.

What changed, across every vendor

Every vendor here has been measured since 14 August, so a change between two readings is a dated fact. A finding code means the same thing before and after a change to our rubric, so appeared, resolved and returned are always shown; a score movement is only published when both readings ran under the same score version.

  1. 2026-09-10Athena a stale edge copy was served againreturned
  2. 2026-09-10Conductor a new finding appeared: NAP_NAME_DRIFT againreturned
  3. 2026-09-10Conductor a new finding appeared: NAP_UNDECLARED_LISTING againreturned
  4. 2026-09-10Conductor a stale edge copy was served againreturned
  5. 2026-09-10Screaming Frog a stale edge copy was served againreturned
  6. 2026-09-10Ahrefs no llms.txt was found againreturned
  7. 2026-09-10Ahrefs the homepage read as mostly markup againreturned
  8. 2026-09-10Ahrefs a render resource was blocked to crawlers againreturned
  9. 2026-09-10Ahrefs robots.txt no longer named a sitemap againreturned
  10. 2026-09-10Ahrefs a stale edge copy was served againreturned
  11. 2026-09-10Ahrefs resolved: this origin refused the scanner, so the site was never measuredresolved
  12. 2026-09-09Athena the stale edge copy clearedresolved
  13. 2026-09-09Conductor resolved: NAP_NAME_DRIFTresolved
  14. 2026-09-09Conductor resolved: NAP_UNDECLARED_LISTINGresolved
  15. 2026-09-09Screaming Frog the stale edge copy clearedresolved
  16. 2026-09-08Conductor the stale edge copy clearedresolved
  17. 2026-09-08Seobility an answer engine was refused againreturned
  18. 2026-09-08Seobility score moved 80 → 77score-down
  19. 2026-09-08Oncrawl a new finding appeared: https is served without a strict-transport-security headerappeared
  20. 2026-09-08Botify a stale edge copy was served againreturned
  21. 2026-09-08Botify score moved 82 → 81score-down
  22. 2026-09-08Sitebulb a new finding appeared: https is served without a strict-transport-security headerappeared
  23. 2026-09-08Screaming Frog score moved 75 → 76score-up
  24. 2026-09-08Similarweb score moved 75 → 74score-down
  25. 2026-09-08SE Ranking score moved 77 → 76score-down
  26. 2026-09-08Moz score moved 85 → 83score-down
  27. 2026-09-08Frase score moved 82 → 83score-up
  28. 2026-09-08Clearscope score moved 73 → 72score-down
  29. 2026-09-08Surfer a stale edge copy was served againreturned
  30. 2026-09-07Goodie score moved 83 → 82score-down
  31. 2026-09-07Rankscale score moved 86 → 85score-down
  32. 2026-09-07Athena a stale edge copy was servedappeared
  33. 2026-09-07Profound score moved 72 → 73score-up
  34. 2026-09-07Similarweb score moved 74 → 75score-up
  35. 2026-09-07SE Ranking score moved 76 → 77score-up
  36. 2026-09-07Moz score moved 83 → 85score-up
  37. 2026-09-06Peec AI score moved 73 → 72score-down
  38. 2026-09-06BrightEdge score moved 73 → 72score-down
  39. 2026-09-06Seobility score moved 81 → 80score-down
  40. 2026-09-06Screaming Frog score moved 76 → 75score-down

This is the one thing a single scan can never produce. A tool that starts measuring the day you ask it can tell you what is true now; only a record kept beforehand can tell you what moved. The same ledger runs on any domain under Watch.

How to read it

The score

The same AI visibility score every report on this site uses, under the current score version: reach (was a named crawler served the page), read (is it mostly text and are the machine files parseable), quote (is there a declared, consistent fact), in that order.

What moves it, and what cannot

Only the measurement. No vendor is listed for payment, no listing is removed for payment, and no score is edited on request — the same rule that governs every report here. A vendor who believes a reading is wrong gets a re-scan, published as measured.

Refused and opted out

A vendor that refused this scanner is listed without a grade: a refusal is a fact about our datacentre address, not a defect of the site. A vendor that opts out in robots.txt is listed as opted out, unmeasured, and can reverse it in the same file.

Categories and prices

Categories are ours. They describe what each product is primarily sold as, not what it can do, and they are not scored. Pricing is not listed, because a price we did not measure is a claim we cannot stand behind.

Why this page exists

Every directory of these tools we could find ranks them by self-reported features and affiliate fit, and none turns its own instrument on the vendors it recommends. We scanned 33 of them once and found a 92-to-41 spread and no entity maps. This page is that finding kept alive, in a form that can be quoted and checked.

Questions about this page

QHow are vendors scored?
By the same scan any domain gets, on the vendor’s own homepage: named-crawler reach, machine readability, quotable facts, every section scored and every finding listed with evidence. Scores come only from stored measurements.
QCan a vendor pay to be listed, removed or rescored?
No. Listing is by category, not payment; nothing is removed for payment; a dispute is a re-scan published as measured. A vendor can opt out of scoring with a line in robots.txt and stays listed as opted out.
QHow often is each vendor re-measured?
On a rolling cadence of roughly one vendor per hour, so every row is under a day old, with the index rebuilt after each pass.
QCan I download the directory data?
Yes: /directory.csv and /directory.json, with the report URL for every row, the change events, and a CC BY 4.0 licence.