{"ok":true,"kind":"crawlcheck-rulebook","v":1,"generated_at":"2026-11-22T01:33:53.527Z","score_version":43,"schema":"https://crawlcheck.io/schemas/rulebook.json","counts":{"rules":111,"by_kind":{"scan":86,"answer_correlation":5,"capability_mismatch":6,"self_audit":12,"mcp_server":2},"by_family":{"site":18,"headers":4,"mostly_code":3,"machine_files":12,"other":9,"ard":4,"webmcp":2,"trust_path":2,"nap":3,"robots":17,"delivery":9,"jsonld":3,"answers":5,"capabilities":6,"self_audit":12,"mcp":1,"watch":1},"revised":10,"scored":82},"scans_counted":9304,"severity_scale":[{"level":"info"},{"level":"low"},{"level":"medium"},{"level":"high"},{"level":"critical"}],"method":{"revisions":"A rule that changes what it reports gets a new revision; a record keeps the revision that decided it, and a replay uses that revision, never today's.","share":"Share of counted scans that carried the code at least once, the same numbers /data publishes. Self-scans and opted-out sites are never counted.","grade":"Each finding has a level from info to critical. Every graded finding lowers the score of its section; an open finding at medium or above also caps the letter grade, so fix those first."},"rules":[{"code":"FRONTIER_BLOAT","kind":"scan","family":{"id":"other","label":"Other"},"title":"Links send crawlers to the same pages under many different addresses","meaning":"A crawler keeps a queue of every address it has found and still has to fetch (its crawl frontier). Each distinct URL is a separate entry and a separate fetch, so one page linked under several query strings, a session id in every link, “add to cart” and “reply to comment” links, or a calendar that links one month further forever all fill that queue with addresses that teach it nothing. Search engines and AI crawlers spend a limited budget per site; what goes on duplicates is not spent on the pages that matter, and the duplicates compete with each other. Counted from real links on the pages read, not guessed from the sitemap.","severity":{"level":"low"},"scored":true,"measured_on":"every scan","revision":{"current":1,"revised_at":null},"fix":{"advice":"Link every page by one address: drop session ids and tracking parameters from internal links, add rel=\"nofollow\" (or a robots.txt Disallow) to add-to-cart, reply-to and calendar-paging links, and give sort/filter variants a canonical tag pointing at the plain page.","effort":"content","endpoint":null},"share_of_scans":{"pct":1.2,"scans":116,"of_scans":9304,"basis":"share of counted scans that carried it; the same numbers /data publishes"},"links":{"self":"https://crawlcheck.io/rules#FRONTIER_BLOAT","api":"https://crawlcheck.io/api/rules?code=FRONTIER_BLOAT","glossary":[],"workflow":null,"lineage":"https://crawlcheck.io/docs/lineage-coverage","explain_template":"https://crawlcheck.io/api/explain?id={report_id}&code=FRONTIER_BLOAT"}}],"filter":{"code":"FRONTIER_BLOAT"}}