{"source":"measured now with CrawlCheck's own requests (a plan from the stored scan needs a licence)","domain":"www.denverpost.com","report_id":null,"scanned_at":null,"fetched_at":"2026-11-12T16:58:19.507Z","rule":"MACHINE_CHAIN_HEAVY fires when the files read before the page exceed both the page and 50 KB, or absent agent files answer 404 with over 50 KB on average","now":{"chain_bytes":1718191,"page_bytes":317099,"chain_fires":true,"probe_404_avg":95287,"probe_404_paths":5,"probe_404_fires":true,"files":[{"path":"/sitemap.xml","bytes":888937,"source":"machine"},{"path":"/image-sitemap.xml","bytes":824687,"source":"machine"},{"path":"/robots.txt","bytes":4567,"source":"machine"}]},"steps":[{"step":1,"id":"file","path":"/robots.txt","action":"Drop comment lines from robots.txt (808 B) and merge groups whose rules are identical by stacking their User-agent lines (763 B)","bytes_before":4567,"bytes_after":2996,"measured":"the file fetched now, every line counted","chain_after":1716620,"detector_fires_after":true},{"step":2,"id":"short-404","action":"Answer the absent agent paths (/agents.md, /.well-known/agent-skills/index.json, /.well-known/mcp/server-card.json, /.well-known/api-catalog, /.well-known/media-kit.json) with a short 404: plain text under 1 KB, instead of the full HTML 404 page","bytes_before":95287,"bytes_after":1024,"measured":"the scan's own probes: 5 absent paths averaged 95,287 B; 1 KB is the target, not a measurement","chain_after":1716620,"detector_fires_after":true}],"clears":false,"clears_at_step":null,"verdict":"Does not clear with the reductions measurable from outside: the chain would still be 1,716,620 B against a 317,099 B page. What is left is the content of the files named above; only 1,399,521 B more needs to go.","assumption":"a sitemap index entry is about 110 bytes and a short 404 is 1 KB; every other number is a byte count of a file fetched now or of the scan's own probes"}