CrawlCheck

Directory · Content and GEO

Frase — what a crawler receives from frase.io

77
Cgrade

Answer engines can reach and read this site. It needs more it can quote.

AI visibility 77/100, measured 2026-09-02 22:17 UTC from outside its network, as fifteen named crawler identities and one unnamed client. An engine can reach and read this site. What is left is giving it something specific to say.

Held at C A medium-severity finding sets the letter.

Open the full report frase.io ↗

2

findings

llms.txt on its host

entity map on its host

4

scans on record since 2026-08-18

The three stages, in the order an engine hits a site

Each stage gates the next. An engine that cannot reach a page never reads it; one that cannot read it never quotes it. That is why reach is a ceiling and not just a weight.

93

Stage 1 · 40% of the score

Reach

Can a named answer-engine crawler get your pages at all?

Every named crawler was served the same page a browser gets.

62

Stage 2 · 30% of the score

Read

Once it has the bytes, can it find the words?

The content is buried in markup, or the files that guide a crawler are missing.

70

Stage 3 · 30% of the score

Quote

Is there a specific fact it can state and attribute?

An assistant would have to paraphrase your page instead of quoting a fact.

The stage to fix first is read, because the three run in order.

What to fix, in order

Findings first: each one holds the grade down until it is gone (the top one sets the letter). Then headroom: each one lifts the score by the points shown. Open a card for what we saw, why it matters and how to fix it.

Files, generated from this scan

Each file says at its top exactly what was changed or filled and what was left as a TODO. Publish, then re-scan: done is measured.

1robots.txt rules do not apply to named agents11 named User-agent groups (11 agents) declare no Disallow rules while the `*` group carries 1. A crawler obeys only its own most-specific group and ignores `*`Medium

What we saw

11 named User-agent groups (11 agents) declare no Disallow rules while the `*` group carries 1. A crawler obeys only its own most-specific group and ignores `*` entirely, so those rules do not apply to any of the named agents. Worst group: `gptbot` bypasses /sentry-test. This includes a SEARCH crawler (applebot-extended) — the bypassed paths are being crawled and can enter the index. /robots.txt

Why it matters

A crawler reads only the User-agent group that matches it best and ignores every other group, including `*`. Naming an agent and giving it only `Allow: /` therefore deletes all of your `*` Disallow rules for that agent. The file still parses and the agent still reaches your homepage, so this is invisible on inspection — but the paths you meant to keep out of search are open, and search engines will crawl and may index them.

How to fix it

Every User-agent group that names a crawler must repeat the Disallow rules you want applied. A named group with no rules means 'allow everything' for that crawler.

ROBOTS_RULES_SHADOWED · full transcript

2Two business records on this page claim the same identity1 identity collision in this page's structured data: two nodes share the same url.Medium

What we saw

1 identity collision in this page's structured data: two nodes share the same url.

Why it matters

A system resolving which business this page is about has more than one candidate for one identity. Two LocalBusiness or Organization nodes sharing a phone number, an @id or a coordinate pair give a retrieval system no way to decide which record is the entity, so it may pick either or neither. This reads only what the page declares - it does not assert the business is duplicated or wrongly located.

How to fix it

Merge the duplicate business nodes into one, keep a single @id, and remove any plugin that emits a second empty LocalBusiness record.

ENTITY_COLLISION · full transcript

Headroom +18 points if every row passes

Nothing here is broken. Each row is a measure that currently fails its optimal range, and what fixing it is worth to the score.

Watch this domain

Twice-daily scans, the date each finding first appeared, and a change receipt when something moves. Free for 30 days, no card.

Start watching frase.io

Get this fixed for you

Every finding above fixed on your site, then re-measured - done is measured, not asserted. One-time, from $750.

See the fix service

The chain a crawler follows

The chain breaks at entity graph. Everything after that point is only reachable by a crawler guessing the conventional path.

robots.txtEvery crawler reads this first
sitemapNamed in robots.txt and resolves
llms.txtPoints onward, links stay on this host
entity graphParses as JSON and points home

A machine file that does not parse is worth less than one that is absent, because it looks present. Rows behind this.

Where the bytes go

Of the 363,067 decompressed bytes the homepage delivers, 5.2% is text a reader or a model can actually use. Most of what a crawler downloads here is not words.

  • Readable text 5.2% · 18,805 B
  • Markup & attributes 29.9% · 108,387 B
  • Structured data (JSON-LD) 1.3% · 4,845 B
  • Inline CSS 0% · 128 B
  • Inline JavaScript 50.3% · 182,695 B
  • Inline SVG 13.2% · 47,915 B
  • HTML comments 0.1% · 292 B

What changed

  1. 2026-09-01a named-agent group shadowed the wildcard rulesappeared
  2. 2026-09-01two schema nodes shared one identifierappeared

4 scans on record, first 2026-08-18. Score movements are shown only between readings taken under the same score version; finding codes are comparable across all of them.

Is this your site?

This listing reads the public record. Claim it and Watch keeps that record on your terms: every list this page holds back, finding age, and twice-daily re-measurement. Free for 30 days, no card.

Claim the record for frase.io

Dispute or re-scan

If a reading here is wrong, it is our instrument that is wrong, and we want to know. Email hello@crawlcheck.io and the site is re-measured; the result is published as measured. Nothing about a listing, a payment or a request changes a number.

Category “Content and GEO” is our label for what the product is primarily sold as; it is not scored. Back to the directory.