AI access policy
robots.txt names AI crawlers by name rather than leaving them to the wildcard. 97 of 263 measured sites qualify.
| Site | Grade | Shelves | Last fetched |
|---|---|---|---|
| 303fence.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| adactio.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| aethelauto.in | B | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-05 |
| aicrawlercheck.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-27 |
| airbnb.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| aireserv.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| allbirds.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| angi.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-04 |
| apple.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| arborcaretreesolutions.com | B | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| arborday.org | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-14 |
| arnoldsmithlaw.com | B | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| arvadaco.gov | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-26 |
| arvadatreeservicellc.com | A | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| bbc.co.uk | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| benjaminfranklinplumbing.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| booking.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| callrail.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| canva.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| cleversearch.ai | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-16 |
| cloudflare.com | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| coalharbourlaw.ca | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| coloradopremiertreecare.org | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| deathwishcoffee.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-23 |
| denverpost.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| dlcandtree.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| dnb.com | F | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-12 · unverified since |
| dryerventwizard.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| duckduckgo.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| en.wikipedia.org | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-14 |
| environmentaltreecarellc.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-25 |
| etsy.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| europa.eu | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| ewrdigital.com | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| excalidraw.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-02 |
| fenceright.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| fivestarpainting.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| frase.io | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| github.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| glassdoctor.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| gov.uk | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| gulchdentalstudio.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| gymshark.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| hartzer.com | F | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-13 |
| highlandslandscaping.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| homeadvisor.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| ikea.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| indianaccent.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| lucky-spins.shop | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-05 |
| mademoisellepirate.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-06 |
| mercadolibre.com.ar | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| mistersparky.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| mollymaid.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| moz.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| mrelectric.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| nerdwallet.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| notion.so | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| nytimes.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-05 |
| oncrawl.com | B | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| onehourheatandair.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| otto.de | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| precisiondoor.net | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| rainbowintl.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| rankscale.ai | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| samtheconcreteman.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| savatree.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| semrush.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| senske.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-10 |
| seobility.net | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| seranking.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| serpstat.com | B | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-18 |
| shopify.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| simplycodes.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-28 |
| sonicsenses.com | A | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-19 |
| spyfu.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-18 |
| statista.com | F | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-12 · unverified since |
| stripe.com | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| supremearvadafencing.com | A | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| supremefencingdenver.com | A | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| supremefencinglakewood.com | A | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| terminix.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| theguardian.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| trakkr.ai | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| treeservicedenvercolorado.com | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| treeservicedenverllc.com | A | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| trugreen.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| tryrankly.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-09 |
| who.int | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| wikipedia.org | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| wikiwand.com | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-25 |
| wired.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| woorank.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-08-18 |
| wordpress.org | — | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-11 |
| ycombinator.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| yelp.com | F | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| zalando.de | D | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
| zapier.com | C | llms.txtagents.mdMedia kitEntity graphAI access policyAgent-callable | 2026-09-01 |
Machine-readable: /api/registry?shelf=aipolicy.