CrawlCheck

The registry

Every domain this scanner has measured, filed on the shelves it earned. A row exists because a fetch produced it — nothing here is submitted, and no payment moves a shelf. 263 sites listed.

Recently fetched

SiteGradeShelvesLast fetched
conductor.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
siteliner.comFllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
seobility.netCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
crawlcheck.ioAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
oncrawl.comBllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
emmanuelorta.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
botify.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
lumar.ioBllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
sitebulb.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
screamingfrog.co.ukCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
similarweb.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
seranking.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
skillsuploadjr.deCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
moz.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
semrush.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
who.intllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
cdc.govllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
harvard.edullms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
vodafone-stiftung.deCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
skillsuploadjr.euCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
hautarzt-bockmann.deCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
terrariumstation.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
supremelandscapersdenver.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
treeservicedenverllc.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
dotaseo.comDllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
credentialrecord.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
supremeexcavationdenver.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
supremeconcreteco.comBllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
supremefencingdenver.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
supremearvadafencing.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
arvadatreeservicellc.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
supremefencinglakewood.comAllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
bbc.co.ukllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
nist.govllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
mit.edullms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
europa.eullms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
frase.ioCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
wordpress.orgllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
wikipedia.orgllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
cloudflare.comllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
react.devllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
clearscope.ioDllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
stripe.comllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
gov.ukllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
modalpoint.comllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
ewrdigital.comllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
bestseopodcast.comllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
surferseo.comCllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
matthewbertram.comllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11
lakewoodfencingsolution.comllms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable2026-09-11

Machine-readable: /api/registry. Rows re-derive from each new scan; a row older than 30 days is marked. To be removed, put User-agent: CrawlCheck and Disallow: / in your robots.txt — it is honoured on the next fetch and the row goes with it, or write to the opt-out address.