CrawlCheck

Directory · LLM observability and evaluation

OpenRouter — what a crawler receives from openrouter.ai

Website measurement, not a product review. This page records what machine clients received from openrouter.ai. It does not evaluate OpenRouter software, features, pricing or support.

86
Dgrade

AI assistants are being shown something other than your site.

AI visibility 86/100, measured 2026-10-31 19:34 UTC from outside its network, as each named AI and search crawler and as a browser. An engine can reach and read this site. What is left is giving it something specific to say.

Held at D A high-severity finding sets the letter.

Open the full report openrouter.ai ↗

941. ReachCan a named answer-engine crawler get your pages at all?862. ReadOnce it has the bytes, can it find the words?763. QuoteIs there a specific fact it can state and attribute?

5

findings

✓

llms.txt on its host

✗

entity map on its host

8

scans on record since 2026-09-18

The three stages, in the order an engine hits a site

Each stage gates the next: an engine that cannot reach a page never reads it, and one that cannot read it never quotes it.

94

Stage 1 · 40% of the score

Reach

Can a named answer-engine crawler get your pages at all?

Every named crawler was served the same page a browser gets.

86

Stage 2 · 30% of the score

Read

Once it has the bytes, can it find the words?

The delivered HTML is mostly content, and the machine files point at the rest.

76

Stage 3 · 30% of the score

Quote

Is there a specific fact it can state and attribute?

An assistant would have to paraphrase your page instead of quoting a fact.

The stage to fix first is quote, the lowest of the three.

What to fix

5 findings on this site, 2 of them scored serious. Every one is named below; the top one is shown in full, free. What we saw, why each matters, how to fix it and the bytes behind it are what a licence buys — or we implement the whole list for you.

1 High 1 Medium 1 Low 2 Note

1The sitemap is refusedHTTP 403, 4,554 bytes.HighSetting

What we saw

HTTP 403, 4,554 bytes. /wp-sitemap.xml

Why it matters

SAME LIMIT AS ABOVE: if a named application firewall appears in the response, this may be blocking CrawlCheck's IP rather than crawlers. Confirm in server logs grouped by source IP. Otherwise: the server refuses this sitemap rather than returning it or a clean 404. Crawlers cannot enumerate the site and fall back to link-following, so deep and newly published pages go undiscovered.

How to fix it

Allow the sitemap URL through the security layer so crawlers can read it.

Setting: a setting on the server, CDN, DNS or a plugin — no code and no writing

On Next.js (detected)

  1. App Router: add app/robots.ts that returns a MetadataRoute.Robots object (rules per userAgent, sitemap URL). Pages Router or static export: put robots.txt in public/.
  2. Ship one of the two, not both: the route handler and a public/robots.txt collide.

In Cloudflare (this site is behind it)

  1. If Cloudflare's managed robots.txt is on (Security → Bots, or AI Crawl Control → robots.txt), Cloudflare adds its own lines above yours. Turn it off, or keep it knowing those lines are there.
  2. Robots.txt is cached at the edge: purge it (Caching → Configuration → Custom purge → the robots.txt URL) after you change it.

All Next.js fixes

Site deployed from GitHub

  1. Add this workflow file as .github/workflows/crawlcheck.yml (change robots-path if robots.txt lives elsewhere).
  2. Run it from the Actions tab: it opens one pull request with the corrected robots.txt, every change listed. Review, merge, deploy.

How the action works

Fix it with a coding agent

Paste into Claude Code, Cursor or Codex, opened on the code that serves this site.

Show prompt
You are working in the codebase (or server/CDN configuration) that serves openrouter.ai. A crawler audit found the problem below. Fix it at the source, keep the change as small as possible, and do not change anything a visitor sees unless the fix requires it.

Finding: The sitemap is refused (SITEMAP_BLOCKED, high severity)
Page: https://openrouter.ai/wp-sitemap.xml
What was measured: HTTP 403, 4,554 bytes.
Why it matters: SAME LIMIT AS ABOVE: if a named application firewall appears in the response, this may be blocking CrawlCheck's IP rather than crawlers. Confirm in server logs grouped by source IP. Otherwise: the server refuses this sitemap rather than returning it or a clean 404. Crawlers cannot enumerate the site and fall back to link-following, so deep and newly published pages go undiscovered.
How to fix it: Allow the sitemap URL through the security layer so crawlers can read it.
Detected platform: Next.js. Make the change where this platform controls it.
Rule and examples: https://crawlcheck.io/rules/SITEMAP_BLOCKED
Full report (markdown): https://crawlcheck.io/r/6hrwt4ugfi.md

Evidence (bytes the crawler received):
```
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" conten
```

When you are done:
1. Tell me which files or settings you changed and why.
2. If the fix is a server, CDN or cache setting you cannot change from the code, say exactly where it is set.
3. After deploying, re-scan at https://crawlcheck.io/?url=openrouter.ai . The fix counts when SITEMAP_BLOCKED is gone from the new report.
Why this was decided
RuleSITEMAP_BLOCKED · rule pack sv34 · scope core
Evaluated2026-10-31 19:34:28 UTC
Decisiond1:363cefff508f1cd991e7 · finding f1:c46cf01055c28bd2a756 resolve either at /api/graph/<id> with a key
Observercrawlcheck-worker build 77ae0f18 from LAX · audit v4

What it rests on

IdentityStatusBytessha256Kept
read: /sitemap.xml2001,153606042ad69b0kept
read: /sitemap_index.xml404155,688fa968d98a48ekept
read: /wp-sitemap.xml4034,554f7ce47f2e756kept

21 more in the manifest.

Artifacts: manifest:ea6c191841d61e7e7a10b1fdc8df9f9317bea52065c441cff0cd6d777bd2d5ac

Evidence bytes
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" conten

Grade effect Caps the letter at D. Without it the letter would be C (score 86). Moves no points; the score is built from the measured rows.

Rule SITEMAP_BLOCKED under scoring rules v34 · fingerprint SITEMAP_BLOCKED|/wp-sitemap.xml · persisting (3 scans)

Verify the fix Re-scan after the change. This finding is cleared when SITEMAP_BLOCKED is no longer reported at /wp-sitemap.xml; if it comes back after clearing, it is marked a regression, not a new finding.

SITEMAP_BLOCKED · full transcript

2Almost nothing a machine receives from this page is readable text🔒 What we saw, why it matters, the fix and the evidence bytes are in the licensed report.Medium
3Links send crawlers to the same pages under many different addresses🔒 What we saw, why it matters, the fix and the evidence bytes are in the licensed report.Low
4The homepage is reached through a chain of redirects🔒 What we saw, why it matters, the fix and the evidence bytes are in the licensed report.Note
5A business record gives an address but no coordinates🔒 What we saw, why it matters, the fix and the evidence bytes are in the licensed report.Note

Open the other 4

What we saw, why it matters, how to fix it, and the evidence bytes for every finding — on this report and every re-scan. From $29 a month.

Unlock the findings

Or have them fixed

We implement the list on your site and re-measure, with a sealed before and after. One site, $100 once.

Get this fixed — $100

The chain a crawler follows

The chain breaks at entity graph. Everything after that point is only reachable by a crawler guessing the conventional path.

✓robots.txtEvery crawler reads this first
✓sitemapNamed in robots.txt and resolves
✓llms.txtPoints onward, links stay on this host
✗entity graphParses as JSON and points home

A machine file that does not parse is worth less than one that is absent, because it looks present. Rows behind this.

Where the bytes go

Of the 191,359 decompressed bytes the homepage delivers, 1.7% is text a reader or a model can actually use. Most of what a crawler downloads here is not words.

  • Readable text 1.7% · 3,292 B
  • Markup & attributes 50% · 95,676 B
  • Structured data (JSON-LD) 0.6% · 1,230 B
  • Inline CSS 2.2% · 4,300 B
  • Inline JavaScript 38.1% · 72,871 B
  • Inline SVG 7.2% · 13,705 B
  • HTML comments 0.1% · 285 B

The individual blocks that weigh the most, so the fix is a search rather than a hunt:

  • Inline JavaScript · 43,637 B (22.8%) — unnamedopens self.__next_f.push([1,"1:\"$Sreact.fragment\"\n2:\"/_next/static/immut
  • Inline JavaScript · 8,854 B (4.6%) — unnamedopens self.__next_f.push([1,"a3:I[382691,[\"$23\",\"$24\",\"$25\",\"$4\",\"$
  • Inline JavaScript · 7,035 B (3.7%) — unnamedopens self.__next_f.push([1,"b3:\"ContainedContextProvider\"\nb4:I[48190,[\"
  • Inline JavaScript · 5,300 B (2.8%) — unnamedopens (function eY(a){try{let b=window.localStorage.getItem(a.storageKey);if
  • Inline CSS · 3,012 B (1.6%) — unnamedopens :root { --bprogress-color: hsl(var(--primary)); --bprogress-height: 2p
Section average over 2 scans on this score version · held at 86 since 2026-10-31
0501002026-10-23 19:37 · 87/100 · 4 findings2026-10-31 19:34 · 86/100 · 5 findings2026-10-232026-10-31

What changed

  1. 2026-10-31a new finding appeared: links send crawlers to the same pages under many different addressesappeared
  2. 2026-10-23a new finding appeared: the sitemap is refusedappeared
  3. 2026-10-11the homepage began taking more than one redirect to reachappeared
  4. 2026-09-27the declared sitemap resolvedresolved
  5. 2026-09-27a new finding appeared: an entity is missing geographic coordinatesappeared
  6. 2026-09-18resolved: the machine files cost more than the pageresolved
  7. 2026-09-18a declared sitemap did not resolveappeared

8 scans on record, first 2026-09-18. Score movements are shown only between readings taken under the same score version; finding codes are comparable across all of them.

Is this your site?

This listing reads the public record. Claim it and Watch keeps that record on your terms: every list this page holds back, finding age, and twice-daily re-measurement. Free for 30 days, no card.

Claim the record for openrouter.ai

Is this your company?

This listing came from a fetch, not a submission, so nobody was asked. If openrouter.ai is yours you can prove it in a minute and for nothing — a DNS TXT record or one meta tag on the home page, both checkable by anyone.

Claiming marks you as the verified owner and lets you correct the category and the company name. It moves no score, no rank and no position.

Claim this listing — free or ask about a verified profile

Dispute or re-scan

If a reading here is wrong, it is our instrument that is wrong, and we want to know. Tell us what is wrong and the site is re-measured; the result is published as measured. Nothing about a listing, a payment or a request changes a number.

By email instead: hello@crawlcheck.io. Category “LLM observability and evaluation” is our label for what the product is primarily sold as; it is not scored. Back to the directory.