# Round 2 grading rules

Ground truth is the test site's server code (what the server actually sends), spot-checked live on 2026-11-03 ~11:30 PM MST with curl
(PerplexityBot 403, Applebot 200 challenge, ClaudeBot 302 → /unavailable, GPTBot 429, Bingbot 503 on /pricing,
roof-repair 404, /specials soft 404, promo loop, /faq/ 200, sitemap-blog 404, riverbend canonical target 404,
sameAs target 404, cookie without Secure/HttpOnly, security.txt as text/html, X-Robots-Tag noindex on /pricing,
no HSTS on lab2, HSTS on the clean twin). `key.json` is the list of 63 planted faults, with one correction (R1, below).

## Correction to the key: R1 (ClaudeBot)

The key says ClaudeBot "may crawl every path". That is only one reading. lab2's robots.txt:

```
7  User-agent: ClaudeBot
8  Crawl-delay: 10
9
10 User-agent: Googlebot-Image
11 Disallow: /
```

- Google's reference parser (github.com/google/robotstxt, built and run here): ClaudeBot **DISALLOWED** on `/`, matched at line 11.
  Crawl-delay is not a rule and blank lines are not separators, so the ClaudeBot and Googlebot-Image lines form one group.
- Python `urllib.robotparser` and npm `robots-parser` 3.0.1: ClaudeBot **allowed** (empty group).

So the real fault is that ClaudeBot's access is ambiguous and, under the reference implementation of RFC 9309, blocked
site-wide by a rule written for Google's image crawler. Grading: a tool that reports ClaudeBot blocked by robots.txt
(line 11) is **caught**; a tool that explains the ambiguity is **caught** with top evidence; a tool that reports ClaudeBot
allowed is **missed**, not wrong, because widely used parsers agree with it.

Round 1 consequence (lab.crawlcheck.io): there ClaudeBot's empty group merges with GPTBot's `Allow: /` (line 9) under
Google's parser. Round 1 criticised aicrawlercheck.com and Swetrix for "quoting another group's line (line 9)". Under the
reference parser line 9 is the line that applies. That criticism was wrong and is withdrawn on the Round 1 page.

## Verdicts per fault

- **caught** (1): the output names the fault or its direct symptom at the right place (URL, file, or crawler), so an
  owner reading it would find and fix that fault.
- **partial** (0.5): the symptom is visible but misdescribed or unlocated (e.g. "no JSON-LD found" for JSON-LD that exists
  but does not parse; a points deduction with no statement of what is wrong).
- **missed** (0).
- **wrong**: the output states as fact the opposite of the truth for that fault (e.g. "PerplexityBot can read this page"
  when it gets 403). Counted separately and shown next to the score; a wrong verdict scores 0.
- **unreached**: the fault is on a page or file the tool never examined (from its output and the lab request log).
  Still scores 0 out of 63; also reported as a share of the faults it reached, so single-page tools are judged on what
  they chose to look at as well as on the whole site.

Reach classes (fixed by where each fault lives): `home` (homepage response for some agent), `file` (robots.txt, llms.txt,
security.txt, sitemaps), `sub` (any other page). A tool that only fetched the homepage and site files reaches `home`+`file`.

## False alarms (clean twin)

lab2-clean has none of the 63 faults. On the clean twin:
- **false alarm**: a factual claim that is untrue (e.g. "crawler X is blocked", "JSON-LD missing" when it parses).
- **advisory**: a true statement about something optional or emerging (no llms-full.txt, no Content-Signal, no MCP card,
  no markdown negotiation). Not counted against the tool; listed for completeness.
A wrong claim on lab2 about something that is not one of the 63 faults is also a false alarm.

## Evidence quality (per tool, 0-3)

0 = score or label only; 1 = names the issue; 2 = names the issue with the URL/agent and observed value (status code,
line, header); 3 = adds the standard or a reproducible check (line number, raw response, request it made).

## CrawlCheck

Graded on its first runs (scan wyka6piddc faulty / kcys45p8jf clean, 41-page crawls i61juq337r / 6vgln95jk3), as a
free user sees them. Findings shown as locked titles count by their title only. Grading of CrawlCheck is done by a
separate grader with no stake in the result.