{"kind":"crawlcheck-conduct","v":1,"generated_at":"2026-11-02T01:04:36.408Z","population":"verifiable agents observed on crawlcheck.io","signature_coverage":{"agents_assessed":33,"agents_signing":2,"verified_requests":76207,"signed_requests":1276,"failed_signatures":2405,"by_agent":[{"agent":"AhrefsBot","operator":"Ahrefs","signed_requests":1272,"verified_requests":7937,"signed_share":0.16},{"agent":"DuckDuckBot","operator":"DuckDuckGo","signed_requests":4,"verified_requests":90,"signed_share":0.044}],"signed_share":0.017,"counted_correctly_since":"2026-10-29T17:45:00Z","reading":"signed_requests are verified requests that carried a Web Bot Auth signature which verified against the operator's own key directory; signed_share is that count over verified_requests. An agent whose operator publishes no key directory cannot sign, so a low share is a measure of adoption, not of conduct"},"reading":"A census of one property, not a sample of the web: what each agent did HERE, verified by source address against the operator's own published ranges. conduct is no_violations_observed when at least 50 verified requests show no fetch of a path robots.txt disallows; violations_observed when a disallowed path was fetched; insufficient_data below 50. The busiest minute is reported, not judged: this site publishes no crawl rate. site_checks_14d are counts from Visitor Resolve calls other sites made about this agent.","thresholds":{"min_verified_requests":50,"published_rate_limit":null,"disallowed_rules":["/r/","/backup/","/admin-old/"]},"verdicts":{"respected":"robots.txt fetched, and no disallowed path fetched","robots_txt_not_fetched":"no disallowed path fetched, but robots.txt was never fetched under this agent's name here, so it cannot be credited with reading the rules","disallowed_paths_fetched":"a verified fetch of a disallowed path, kept with its time, path and network in disallowed_fetch_records","disallowed_fetch_unrecorded":"disallowed fetches were counted before 2026-10-15, when no per-fetch record was kept; reported, not judged (conduct: not_judged)"},"grades":{"A":"read robots.txt and obeyed it","B":"obeyed, but never fetched robots.txt under its own name here","D":"recorded fetches of disallowed paths, under 0.5% of its verified requests","F":"a fetch of a decoy path named only in robots.txt, or disallowed fetches at 0.5% or more","none":"not enough verified requests, or not judgeable"},"agents":[{"agent":"Amazonbot","operator":"Amazon","purpose":"indexing","user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-13T20:30:33.503Z","last_seen":"2026-11-02T00:37:51.438Z"},"identity":{"verified_requests":14030,"forged_claims":76,"forgery_share":0.005,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Amazon publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":0,"disallowed_fetches":30,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[{"at":"2026-10-18T08:45:12.519Z","path":"/r/xufza280d1","net":"18.215.77.0/24"},{"at":"2026-10-18T18:02:03.808Z","path":"/r/xufza280d1","net":"52.205.113.0/24"},{"at":"2026-10-18T18:04:19.333Z","path":"/r/xufza280d1","net":"34.227.156.0/24"},{"at":"2026-10-18T19:28:54.132Z","path":"/r/xufza280d1","net":"54.163.169.0/24"},{"at":"2026-10-18T20:50:07.376Z","path":"/r/xufza280d1","net":"44.220.2.0/24"},{"at":"2026-10-19T23:29:49.080Z","path":"/r/xufza280d1","net":"23.23.137.0/24"},{"at":"2026-10-23T04:17:44.813Z","path":"/r/hsem60u0pb","net":"18.214.186.0/24"},{"at":"2026-10-24T07:16:41.220Z","path":"/r/hsem60u0pb","net":"184.73.68.0/24"},{"at":"2026-10-24T07:16:45.273Z","path":"/r/hsem60u0pb","net":"3.221.156.0/24"},{"at":"2026-10-24T13:42:45.118Z","path":"/r/hsem60u0pb","net":"18.209.137.0/24"},{"at":"2026-10-26T02:57:16.905Z","path":"/r/cz9abl7ob3","net":"18.215.77.0/24"},{"at":"2026-10-27T12:38:34.291Z","path":"/r/cz9abl7ob3","net":"44.197.76.0/24"},{"at":"2026-10-27T14:53:27.270Z","path":"/r/cz9abl7ob3","net":"52.1.106.0/24"},{"at":"2026-10-27T17:03:47.170Z","path":"/r/cz9abl7ob3","net":"107.23.62.0/24"},{"at":"2026-10-27T22:34:39.166Z","path":"/r/cz9abl7ob3","net":"34.231.77.0/24"},{"at":"2026-10-27T23:05:38.747Z","path":"/r/1fb80ss94n","net":"52.45.29.0/24"},{"at":"2026-10-28T06:16:36.232Z","path":"/r/1fb80ss94n","net":"44.207.69.0/24"},{"at":"2026-10-28T08:23:26.019Z","path":"/r/up0rftpp5z","net":"54.147.238.0/24"},{"at":"2026-10-28T11:12:05.504Z","path":"/r/1fb80ss94n","net":"52.204.89.0/24"},{"at":"2026-10-28T11:12:06.136Z","path":"/r/1fb80ss94n","net":"44.194.139.0/24"}],"counted_since":"2026-10-18T06:50:57.435Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":24,"peak_at":"2026-10-19T05:19","published_limit":null,"measured_since":"2026-10-18T06:50:57.435Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-18T06:50:57.435Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-20":1042,"2026-10-21":1354,"2026-10-22":1064,"2026-10-23":533,"2026-10-24":648,"2026-10-25":785,"2026-10-26":861,"2026-10-27":1018,"2026-10-28":916,"2026-10-29":1009,"2026-10-30":914,"2026-10-31":534,"2026-11-01":605,"2026-11-02":13},"grade":"D","grade_reasons":["30 recorded fetches of disallowed paths, 0.21% of its 14,030 verified requests"]},{"agent":"PerplexityBot","operator":"Perplexity","purpose":"indexing","user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-13T02:14:50.805Z","last_seen":"2026-11-02T00:34:17.665Z"},"identity":{"verified_requests":11684,"forged_claims":86,"forgery_share":0.007,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Perplexity publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":156,"disallowed_fetches":23,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[{"at":"2026-10-22T08:38:56.498Z","path":"/r/hsem60u0pb","net":"18.97.9.0/24"},{"at":"2026-10-22T15:59:08.322Z","path":"/r/hsem60u0pb","net":"18.97.9.0/24"},{"at":"2026-10-22T15:59:20.457Z","path":"/r/uazt7utoub","net":"18.97.9.0/24"},{"at":"2026-10-24T07:38:31.838Z","path":"/r/hsem60u0pb","net":"18.97.9.0/24"},{"at":"2026-10-24T07:38:34.319Z","path":"/r/phuaa0zazw","net":"18.97.9.0/24"},{"at":"2026-10-25T14:17:44.938Z","path":"/r/cz9abl7ob3","net":"18.97.9.0/24"},{"at":"2026-10-25T14:17:45.149Z","path":"/r/phuaa0zazw","net":"18.97.9.0/24"},{"at":"2026-10-25T14:17:45.922Z","path":"/r/phuaa0zazw","net":"18.97.9.0/24"},{"at":"2026-10-25T14:17:46.331Z","path":"/r/phuaa0zazw","net":"18.97.9.0/24"},{"at":"2026-10-25T14:17:47.936Z","path":"/r/cz9abl7ob3","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:02.461Z","path":"/r/cz9abl7ob3","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:04.413Z","path":"/r/cz9abl7ob3","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:07.938Z","path":"/r/up0rftpp5z","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:08.999Z","path":"/r/1fb80ss94n","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:09.428Z","path":"/r/up0rftpp5z","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:11.912Z","path":"/r/1fb80ss94n","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:18.485Z","path":"/r/up0rftpp5z","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:18.928Z","path":"/r/up0rftpp5z","net":"18.97.9.0/24"},{"at":"2026-10-27T06:50:46.518Z","path":"/r/cz9abl7ob3","net":"18.97.9.0/24"},{"at":"2026-10-28T20:49:42.689Z","path":"/r/khlmjzqasm","net":"18.97.9.0/24"}],"counted_since":"2026-10-07T04:51:16.683Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":119,"peak_at":"2026-10-11T23:32","published_limit":null,"measured_since":"2026-10-07T04:51:16.683Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T04:51:16.683Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":211,"verified":135,"spoofed":76},"daily_verified":{"2026-10-20":62,"2026-10-21":141,"2026-10-22":155,"2026-10-23":298,"2026-10-24":205,"2026-10-25":142,"2026-10-26":121,"2026-10-27":250,"2026-10-28":181,"2026-10-29":96,"2026-10-30":173,"2026-10-31":289,"2026-11-01":144,"2026-11-02":385},"grade":"D","grade_reasons":["23 recorded fetches of disallowed paths, 0.2% of its 11,684 verified requests"]},{"agent":"Googlebot","operator":"Google","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T02:14:51.611Z","last_seen":"2026-11-02T01:00:58.683Z"},"identity":{"verified_requests":8707,"forged_claims":164,"forgery_share":0.018,"signed_requests":0,"failed_signatures":2,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":143,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-07T03:18:09.861Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":13,"peak_at":"2026-10-19T02:10","published_limit":null,"measured_since":"2026-10-07T03:18:09.861Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":811,"share_of_verified_since_measured":0.093,"measured_since":"2026-10-07T03:18:09.861Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":1674,"verified":1124,"spoofed":550},"daily_verified":{"2026-10-20":98,"2026-10-21":152,"2026-10-22":152,"2026-10-23":136,"2026-10-24":106,"2026-10-25":129,"2026-10-26":89,"2026-10-27":69,"2026-10-28":48,"2026-10-29":139,"2026-10-30":103,"2026-10-31":193,"2026-11-01":162,"2026-11-02":9},"grade":"A","grade_reasons":["fetched robots.txt and fetched no path it disallows"]},{"agent":"AhrefsBot","operator":"Ahrefs","purpose":"tool","user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-14T02:11:27.871Z","last_seen":"2026-11-02T00:28:54.968Z"},"identity":{"verified_requests":7937,"forged_claims":1,"forgery_share":0,"signed_requests":1272,"failed_signatures":2356,"signed_share":0.16,"strongest_proof":"signature","reading":"verified_requests came from the ranges Ahrefs publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":153,"disallowed_fetches":6,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[{"at":"2026-10-26T07:48:14.796Z","path":"/r/1fb80ss94n","net":"51.68.247.0/24"},{"at":"2026-10-26T08:03:14.376Z","path":"/r/1fb80ss94n","net":"51.75.236.0/24"},{"at":"2026-10-26T08:44:02.288Z","path":"/r/1fb80ss94n","net":"5.39.109.0/24"}],"counted_since":"2026-10-07T23:53:27.541Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":64,"peak_at":"2026-11-01T16:47","published_limit":null,"measured_since":"2026-10-07T23:53:27.541Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":8,"share_of_verified_since_measured":0.001,"measured_since":"2026-10-07T23:53:27.541Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-20":532,"2026-10-21":97,"2026-10-22":586,"2026-10-23":802,"2026-10-24":345,"2026-10-25":289,"2026-10-26":315,"2026-10-27":158,"2026-10-28":134,"2026-10-29":217,"2026-10-30":30,"2026-10-31":30,"2026-11-01":1118,"2026-11-02":19},"grade":"D","grade_reasons":["6 recorded fetches of disallowed paths, 0.08% of its 7,937 verified requests"]},{"agent":"ClaudeBot","operator":"Anthropic","purpose":"training","user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-13T02:14:49.992Z","last_seen":"2026-11-01T23:24:49.678Z"},"identity":{"verified_requests":7553,"forged_claims":94,"forgery_share":0.012,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Anthropic publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":244,"disallowed_fetches":3,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[{"at":"2026-10-16T09:27:57.482Z","path":"/r/uazt7utoub","net":"216.73.216.0/24"},{"at":"2026-10-16T09:31:12.061Z","path":"/r/uazt7utoub","net":"216.73.216.0/24"},{"at":"2026-10-16T09:38:15.950Z","path":"/r/uazt7utoub","net":"216.73.216.0/24"}],"counted_since":"2026-10-07T03:48:54.788Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":61,"peak_at":"2026-10-19T01:34","published_limit":null,"measured_since":"2026-10-07T03:48:54.788Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:48:54.788Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":283,"verified":206,"spoofed":77},"daily_verified":{"2026-10-19":1483,"2026-10-20":191,"2026-10-21":32,"2026-10-22":45,"2026-10-23":169,"2026-10-24":158,"2026-10-25":263,"2026-10-26":14,"2026-10-27":430,"2026-10-28":20,"2026-10-29":18,"2026-10-30":81,"2026-10-31":22,"2026-11-01":17},"grade":"D","grade_reasons":["3 recorded fetches of disallowed paths, 0.04% of its 7,553 verified requests"]},{"agent":"Applebot","operator":"Apple","purpose":"indexing","user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-13T12:32:42.174Z","last_seen":"2026-11-02T00:49:13.450Z"},"identity":{"verified_requests":6439,"forged_claims":84,"forgery_share":0.013,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Apple publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":67,"disallowed_fetches":18,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[{"at":"2026-10-23T13:32:15.215Z","path":"/r/8dsc0mu5ks","net":"17.166.233.0/24"},{"at":"2026-10-23T13:39:38.180Z","path":"/r/8dsc0mu5ks","net":"17.166.232.0/24"},{"at":"2026-10-24T07:28:50.585Z","path":"/r/8dsc0mu5ks","net":"17.166.235.0/24"},{"at":"2026-10-24T07:32:58.177Z","path":"/r/8dsc0mu5ks","net":"17.166.236.0/24"},{"at":"2026-10-24T16:13:56.254Z","path":"/r/mkx08rapce","net":"17.166.232.0/24"},{"at":"2026-10-24T16:17:40.243Z","path":"/r/mkx08rapce","net":"17.166.235.0/24"},{"at":"2026-10-26T15:31:58.651Z","path":"/r/8dsc0mu5ks","net":"17.166.232.0/24"},{"at":"2026-10-26T15:34:20.954Z","path":"/r/8dsc0mu5ks","net":"17.166.232.0/24"},{"at":"2026-10-26T16:01:58.942Z","path":"/r/8dsc0mu5ks","net":"17.166.25.0/24"},{"at":"2026-10-26T16:06:50.241Z","path":"/r/8dsc0mu5ks","net":"17.166.21.0/24"},{"at":"2026-10-27T20:22:26.222Z","path":"/r/mkx08rapce","net":"17.166.235.0/24"},{"at":"2026-10-27T20:37:56.625Z","path":"/r/mkx08rapce","net":"17.166.233.0/24"},{"at":"2026-10-28T01:54:46.237Z","path":"/r/mkx08rapce","net":"17.166.21.0/24"},{"at":"2026-10-28T01:55:55.986Z","path":"/r/mkx08rapce","net":"17.166.22.0/24"},{"at":"2026-10-28T12:27:38.731Z","path":"/r/mkx08rapce","net":"17.166.153.0/24"},{"at":"2026-10-28T12:33:52.496Z","path":"/r/mkx08rapce","net":"17.166.152.0/24"},{"at":"2026-10-30T18:31:10.592Z","path":"/r/khlmjzqasm","net":"17.166.153.0/24"},{"at":"2026-10-30T18:31:54.377Z","path":"/r/khlmjzqasm","net":"17.166.154.0/24"}],"counted_since":"2026-10-07T03:15:19.709Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":8,"peak_at":"2026-10-21T17:47","published_limit":null,"measured_since":"2026-10-07T03:15:19.709Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:15:19.709Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":636,"verified":600,"spoofed":36},"daily_verified":{"2026-10-20":205,"2026-10-21":127,"2026-10-22":243,"2026-10-23":103,"2026-10-24":125,"2026-10-25":114,"2026-10-26":31,"2026-10-27":39,"2026-10-28":49,"2026-10-29":35,"2026-10-30":117,"2026-10-31":94,"2026-11-01":40,"2026-11-02":4},"grade":"D","grade_reasons":["18 recorded fetches of disallowed paths, 0.28% of its 6,439 verified requests"]},{"agent":"Bingbot","operator":"Microsoft","purpose":"indexing","user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-14T06:25:41.331Z","last_seen":"2026-11-02T01:03:34.875Z"},"identity":{"verified_requests":6141,"forged_claims":49,"forgery_share":0.008,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Microsoft publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":85,"disallowed_fetches":1,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[{"at":"2026-10-27T19:17:43.386Z","path":"/r/uazt7utoub","net":"40.77.167.0/24"}],"counted_since":"2026-10-07T03:20:08.676Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":8,"peak_at":"2026-11-01T02:34","published_limit":null,"measured_since":"2026-10-07T03:20:08.676Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":337,"share_of_verified_since_measured":0.055,"measured_since":"2026-10-07T03:20:08.676Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":842,"verified":767,"spoofed":75},"daily_verified":{"2026-10-20":225,"2026-10-21":214,"2026-10-22":163,"2026-10-23":238,"2026-10-24":238,"2026-10-25":215,"2026-10-26":186,"2026-10-27":242,"2026-10-28":229,"2026-10-29":229,"2026-10-30":199,"2026-10-31":182,"2026-11-01":361,"2026-11-02":15},"grade":"D","grade_reasons":["1 recorded fetch of disallowed paths, 0.02% of its 6,141 verified requests"]},{"agent":"GPTBot","operator":"OpenAI","purpose":"training","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-12T06:20:42.861Z","last_seen":"2026-11-01T04:57:47.738Z"},"identity":{"verified_requests":4343,"forged_claims":142,"forgery_share":0.032,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges OpenAI publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"robots_txt_not_fetched","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-07T18:09:36.470Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":83,"peak_at":"2026-10-18T03:37","published_limit":null,"measured_since":"2026-10-07T18:09:36.470Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T18:09:36.470Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":121,"verified":21,"spoofed":97},"daily_verified":{"2026-10-19":8,"2026-10-20":2,"2026-10-21":2,"2026-10-22":2,"2026-10-23":10,"2026-10-24":2,"2026-10-25":2,"2026-10-26":2,"2026-10-27":2,"2026-10-28":48,"2026-10-29":4,"2026-10-30":2,"2026-10-31":6,"2026-11-01":21},"grade":"B","grade_reasons":["no disallowed path fetched, but it never fetched robots.txt under its own name here, so it cannot be credited with reading the rules"]},{"agent":"OAI-SearchBot","operator":"OpenAI","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T02:14:49.166Z","last_seen":"2026-11-02T01:03:23.299Z"},"identity":{"verified_requests":4065,"forged_claims":96,"forgery_share":0.023,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges OpenAI publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":235,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-07T03:30:28.967Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":119,"peak_at":"2026-10-29T12:58","published_limit":null,"measured_since":"2026-10-07T03:30:28.967Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:30:28.967Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":522,"verified":467,"spoofed":55},"daily_verified":{"2026-10-20":6,"2026-10-21":20,"2026-10-22":42,"2026-10-23":188,"2026-10-24":45,"2026-10-25":5,"2026-10-26":67,"2026-10-27":74,"2026-10-28":35,"2026-10-29":1054,"2026-10-30":234,"2026-10-31":77,"2026-11-01":96,"2026-11-02":40},"grade":"A","grade_reasons":["fetched robots.txt and fetched no path it disallows"]},{"agent":"YandexBot","operator":"Yandex","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-19T13:53:12.960Z","last_seen":"2026-11-02T01:03:24.872Z"},"identity":{"verified_requests":2631,"forged_claims":12,"forgery_share":0.005,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"reverse_dns","reading":"verified_requests passed forward-confirmed reverse DNS to Yandex's own domains (yandex.ru, yandex.net, yandex.com); forged_claims used this agent's name from addresses that did not (impersonators, not the agent). Reverse-DNS checks began 10 October 2026; earlier hits stay unverifiable"},"robots_txt":{"verdict":"respected","robots_txt_fetches":272,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-10T12:54:23.939Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":25,"peak_at":"2026-10-15T05:43","published_limit":null,"measured_since":"2026-10-10T12:54:23.939Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-10T12:54:23.939Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-18":2,"2026-10-20":22,"2026-10-21":62,"2026-10-22":70,"2026-10-23":17,"2026-10-24":52,"2026-10-25":7,"2026-10-26":6,"2026-10-28":7,"2026-10-29":384,"2026-10-30":163,"2026-10-31":221,"2026-11-01":243,"2026-11-02":14},"grade":"A","grade_reasons":["fetched robots.txt and fetched no path it disallows"]},{"agent":"ChatGPT-User","operator":"OpenAI","purpose":"live fetch","user_triggered":true,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T22:52:23.214Z","last_seen":"2026-11-02T00:15:10.544Z"},"identity":{"verified_requests":1020,"forged_claims":84,"forgery_share":0.076,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges OpenAI publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"not_applicable","robots_txt_fetches":1,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-07T03:26:16.042Z","reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"measured","peak_requests_per_minute":13,"peak_at":"2026-10-15T19:36","published_limit":null,"measured_since":"2026-10-07T03:26:16.042Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:26:16.042Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":154,"verified":134,"spoofed":20},"daily_verified":{"2026-10-20":14,"2026-10-21":39,"2026-10-22":28,"2026-10-23":32,"2026-10-24":16,"2026-10-25":21,"2026-10-26":26,"2026-10-27":26,"2026-10-28":27,"2026-10-29":21,"2026-10-30":29,"2026-10-31":19,"2026-11-01":13,"2026-11-02":1},"grade":"A","grade_reasons":["fetches pages a person asked for; its operator states robots.txt does not govern those fetches, and every counted request came from its published addresses"]},{"agent":"SeznamBot","operator":"Seznam","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-10-10T13:04:32.776Z","last_seen":"2026-11-02T00:57:21.311Z"},"identity":{"verified_requests":704,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"reverse_dns","reading":"verified_requests passed forward-confirmed reverse DNS to Seznam's own domains (seznam.cz); forged_claims used this agent's name from addresses that did not (impersonators, not the agent). Reverse-DNS checks began 10 October 2026; earlier hits stay unverifiable"},"robots_txt":{"verdict":"respected","robots_txt_fetches":18,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-10T13:04:32.776Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-10T22:12","published_limit":null,"measured_since":"2026-10-10T13:04:32.776Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":588,"share_of_verified_since_measured":0.835,"measured_since":"2026-10-10T13:04:32.776Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-20":32,"2026-10-21":26,"2026-10-22":39,"2026-10-23":24,"2026-10-24":24,"2026-10-25":31,"2026-10-26":29,"2026-10-27":35,"2026-10-28":24,"2026-10-29":19,"2026-10-30":50,"2026-10-31":30,"2026-11-01":22,"2026-11-02":6},"grade":"A","grade_reasons":["fetched robots.txt and fetched no path it disallows"]},{"agent":"Baiduspider","operator":"Baidu","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T21:31:01.445Z","last_seen":"2026-11-02T00:52:32.846Z"},"identity":{"verified_requests":376,"forged_claims":30,"forgery_share":0.074,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"reverse_dns","reading":"verified_requests passed forward-confirmed reverse DNS to Baidu's own domains (baidu.com, baidu.jp); forged_claims used this agent's name from addresses that did not (impersonators, not the agent). Reverse-DNS checks began 10 October 2026; earlier hits stay unverifiable"},"robots_txt":{"verdict":"robots_txt_not_fetched","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-10T13:14:25.713Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":5,"peak_at":"2026-10-26T00:08","published_limit":null,"measured_since":"2026-10-10T13:14:25.713Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-10T13:14:25.713Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-20":25,"2026-10-21":23,"2026-10-22":7,"2026-10-23":10,"2026-10-24":24,"2026-10-25":14,"2026-10-26":29,"2026-10-27":12,"2026-10-28":17,"2026-10-29":5,"2026-10-30":24,"2026-10-31":29,"2026-11-01":10,"2026-11-02":3},"grade":"B","grade_reasons":["no disallowed path fetched, but it never fetched robots.txt under its own name here, so it cannot be credited with reading the rules"]},{"agent":"Google-CWS","operator":"Google","purpose":"store listing fetch","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-09-18T16:25:39.959Z","last_seen":"2026-11-01T23:55:51.250Z"},"identity":{"verified_requests":248,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"robots_txt_not_fetched","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-07T11:31:33.296Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-07T20:31","published_limit":null,"measured_since":"2026-10-07T11:31:33.296Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T11:31:33.296Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-18":2,"2026-10-20":2,"2026-10-21":11,"2026-10-22":6,"2026-10-23":5,"2026-10-24":5,"2026-10-25":1,"2026-10-26":2,"2026-10-29":10,"2026-10-30":8,"2026-10-31":14,"2026-11-01":15},"grade":"B","grade_reasons":["no disallowed path fetched, but it never fetched robots.txt under its own name here, so it cannot be credited with reading the rules"]},{"agent":"Claude-User","operator":"Anthropic","purpose":"live fetch","user_triggered":true,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-12T22:57:31.977Z","last_seen":"2026-11-01T15:46:44.357Z"},"identity":{"verified_requests":97,"forged_claims":54,"forgery_share":0.358,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Anthropic publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"not_applicable","robots_txt_fetches":14,"disallowed_fetches":1,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[{"at":"2026-10-17T04:44:31.376Z","path":"/r/zqdnb5b58f","net":"160.79.106.0/24"}],"counted_since":"2026-10-07T11:03:04.563Z","reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-08T16:11","published_limit":null,"measured_since":"2026-10-07T11:03:04.563Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T11:03:04.563Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":25,"verified":4,"spoofed":21},"daily_verified":{"2026-10-25":1},"grade":"A","grade_reasons":["fetches pages a person asked for; its operator states robots.txt does not govern those fetches, and every counted request came from its published addresses"]},{"agent":"Google-InspectionTool","operator":null,"purpose":null,"user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-10-27T11:15:26.957Z","last_seen":"2026-11-01T21:35:38.688Z"},"identity":{"verified_requests":96,"forged_claims":4,"forgery_share":0.04,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"robots_txt_not_fetched","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-29T14:57:13.241Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":7,"peak_at":"2026-10-29T21:37","published_limit":null,"measured_since":"2026-10-29T14:57:13.241Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-29T14:57:13.241Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":1,"verified":0,"spoofed":1},"daily_verified":{"2026-10-29":57,"2026-10-30":18,"2026-10-31":10,"2026-11-01":11},"grade":"B","grade_reasons":["no disallowed path fetched, but it never fetched robots.txt under its own name here, so it cannot be credited with reading the rules"]},{"agent":"DuckDuckBot","operator":"DuckDuckGo","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-12T20:29:50.246Z","last_seen":"2026-11-02T01:00:01.559Z"},"identity":{"verified_requests":90,"forged_claims":0,"forgery_share":0,"signed_requests":4,"failed_signatures":40,"signed_share":0.044,"strongest_proof":"signature","reading":"verified_requests came from the ranges DuckDuckGo publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"robots_txt_not_fetched","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-10T21:21:35.448Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":2,"peak_at":"2026-10-10T21:21","published_limit":null,"measured_since":"2026-10-10T21:21:35.448Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-10T21:21:35.448Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-20":5,"2026-10-21":4,"2026-10-22":7,"2026-10-23":4,"2026-10-24":3,"2026-10-25":3,"2026-10-26":7,"2026-10-27":6,"2026-10-28":7,"2026-10-29":8,"2026-10-30":1,"2026-10-31":4,"2026-11-01":3,"2026-11-02":2},"grade":"B","grade_reasons":["no disallowed path fetched, but it never fetched robots.txt under its own name here, so it cannot be credited with reading the rules"]},{"agent":"GoogleOther","operator":"Google","purpose":"training","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-28T07:22:36.856Z","last_seen":"2026-11-01T20:53:30.774Z"},"identity":{"verified_requests":16,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-28T07:22:36.856Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":1,"peak_at":"2026-10-28T07:22","published_limit":null,"measured_since":"2026-10-28T07:22:36.856Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-28T07:22:36.856Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":1,"verified":0,"spoofed":1},"daily_verified":{"2026-10-28":6,"2026-10-29":5,"2026-10-30":2,"2026-10-31":1,"2026-11-01":2},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Perplexity-User","operator":"Perplexity","purpose":"live fetch","user_triggered":true,"conduct":"insufficient_data","window":{"first_seen":"2026-08-14T01:04:08.513Z","last_seen":"2026-10-23T09:41:36.723Z"},"identity":{"verified_requests":8,"forged_claims":51,"forgery_share":0.864,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Perplexity publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":14,"verified":0,"spoofed":14},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Google-NotebookLM","operator":"Google","purpose":null,"user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-09T19:50:34.360Z","last_seen":"2026-10-13T20:31:29.773Z"},"identity":{"verified_requests":7,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-09T19:50:34.360Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":4,"peak_at":"2026-10-09T19:50","published_limit":null,"measured_since":"2026-10-09T19:50:34.360Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-09T19:50:34.360Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Google-Agent","operator":"Google","purpose":"live fetch","user_triggered":true,"conduct":"insufficient_data","window":{"first_seen":"2026-10-31T20:31:56.218Z","last_seen":"2026-11-01T22:11:40.865Z"},"identity":{"verified_requests":5,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":5,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-31T20:31:56.218Z","reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-31T20:31","published_limit":null,"measured_since":"2026-10-31T20:31:56.218Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-31T20:31:56.218Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-31":4,"2026-11-01":1},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"AdsBot-Google","operator":"Google","purpose":"tool","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-27T09:03:19.135Z","last_seen":"2026-10-30T02:16:38.887Z"},"identity":{"verified_requests":4,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-27T09:03:19.135Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":2,"peak_at":"2026-10-27T09:03","published_limit":null,"measured_since":"2026-10-27T09:03:19.135Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-27T09:03:19.135Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-27":2,"2026-10-30":2},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Yeti","operator":"Naver","purpose":"indexing","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-20T06:52:03.909Z","last_seen":"2026-10-29T16:45:48.700Z"},"identity":{"verified_requests":3,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"reverse_dns","reading":"verified_requests passed forward-confirmed reverse DNS to Naver's own domains (naver.com); forged_claims used this agent's name from addresses that did not (impersonators, not the agent). Reverse-DNS checks began 10 October 2026; earlier hits stay unverifiable"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":2,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-20T06:52:03.909Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":1,"peak_at":"2026-10-20T06:52","published_limit":null,"measured_since":"2026-10-20T06:52:03.909Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-20T06:52:03.909Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-20":1,"2026-10-29":2},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Google-Site-Verification","operator":"Google","purpose":"ownership check","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-03T21:54:40.584Z","last_seen":"2026-10-03T21:54:40.980Z"},"identity":{"verified_requests":2,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Google-Safety","operator":null,"purpose":null,"user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-29T05:16:06.085Z","last_seen":"2026-10-29T05:16:06.085Z"},"identity":{"verified_requests":1,"forged_claims":0,"forgery_share":0,"signed_requests":0,"failed_signatures":0,"signed_share":0,"strongest_proof":"address","reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-29T05:16:06.085Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":1,"peak_at":"2026-10-29T05:16","published_limit":null,"measured_since":"2026-10-29T05:16:06.085Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-29T05:16:06.085Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-29":1},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Google-Extended","operator":"Google","purpose":"training","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-15T09:11:18.716Z","last_seen":"2026-09-14T23:53:48.097Z"},"identity":{"verified_requests":0,"forged_claims":49,"forgery_share":1,"signed_requests":0,"failed_signatures":0,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Claude-SearchBot","operator":"Anthropic","purpose":"indexing","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-16T01:38:23.331Z","last_seen":"2026-10-29T21:47:27.912Z"},"identity":{"verified_requests":0,"forged_claims":55,"forgery_share":1,"signed_requests":0,"failed_signatures":0,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges Anthropic publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":50,"verified":0,"spoofed":50},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"anthropic-ai","operator":"Anthropic","purpose":"training","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-19T11:41:41.601Z","last_seen":"2026-09-24T22:51:40.795Z"},"identity":{"verified_requests":0,"forged_claims":4,"forgery_share":1,"signed_requests":0,"failed_signatures":0,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges Anthropic publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Applebot-Extended","operator":"Apple","purpose":"training","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-19T11:41:43.412Z","last_seen":"2026-08-19T11:42:02.678Z"},"identity":{"verified_requests":0,"forged_claims":1,"forgery_share":1,"signed_requests":0,"failed_signatures":0,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges Apple publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"MistralAI-User","operator":"Mistral","purpose":"live fetch","user_triggered":true,"conduct":"insufficient_data","window":{"first_seen":"2026-08-16T01:38:24.858Z","last_seen":"2026-09-12T16:50:37.743Z"},"identity":{"verified_requests":0,"forged_claims":0,"forgery_share":null,"signed_requests":0,"failed_signatures":0,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges Mistral publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":1,"verified":0,"spoofed":1},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"FeedFetcher-Google","operator":"Google","purpose":"feed fetch","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-09-28T08:19:13.036Z","last_seen":"2026-09-28T08:19:13.036Z"},"identity":{"verified_requests":0,"forged_claims":1,"forgery_share":1,"signed_requests":0,"failed_signatures":0,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Google-CloudVertexBot","operator":"Google","purpose":"training","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-30T15:10:18.766Z","last_seen":"2026-10-30T15:10:25.755Z"},"identity":{"verified_requests":0,"forged_claims":3,"forgery_share":1,"signed_requests":0,"failed_signatures":0,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]},{"agent":"Storebot-Google","operator":"Google","purpose":"indexing","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-31T06:04:36.487Z","last_seen":"2026-10-31T08:43:58.767Z"},"identity":{"verified_requests":0,"forged_claims":2,"forgery_share":1,"signed_requests":0,"failed_signatures":2,"signed_share":null,"strongest_proof":"address","reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":null,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{},"grade":null,"grade_reasons":["fewer than 50 verified requests so far"]}],"not_assessed":[{"agent":"Meta-ExternalAgent","claimed_hits":58493,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Bytespider","claimed_hits":344,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"CCBot","claimed_hits":1120,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Diffbot","claimed_hits":2,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"SemrushBot","claimed_hits":15429,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"DataForSeoBot","claimed_hits":11149,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Timpibot","claimed_hits":4,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"YouBot","claimed_hits":1359,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Scrapy","claimed_hits":13,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"cohere-ai","claimed_hits":41,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"FacebookBot","claimed_hits":1,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"}],"census":"https://crawlcheck.io/api/telemetry","visitor_checks":"https://crawlcheck.io/api/v1/visitor/stats","code":{"version_id":"85b8d8fc-6322-4a81-91f5-1fd108f27717","script_sha16":"c661233baf78516d","state":"attested","deployed":{"at":"2026-11-02T01:01:37.825Z","commit":"282855d04980","run":"41370192575"},"build_log_entry":1248,"score_version":34,"scoring_policy_sha256":"76ee7526c402d711f2c976ad88ae13d4d3411aaa9d5ca3dc87ab3102396b1370","decision_rules_sha256":"18dd5636e0896692ab518050cb401744edea2e6f1086dff94fead314959bdc3e","check":"https://crawlcheck.io/api/v1/builds/85b8d8fc-6322-4a81-91f5-1fd108f27717"},"sha256":"fe52526f6d6ed9fcbb7cffb5d29556c292efb59115244fa599228706b9f5b1e4","signature":{"v":1,"alg":"Ed25519","kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","over":"sha256","message":"crawlcheck-data-v1\nfe52526f6d6ed9fcbb7cffb5d29556c292efb59115244fa599228706b9f5b1e4","sig":"0uPd7rk7i3b5nEqrNjhlS-pMLp1I_7XnDGCsNG2KsEuDvavDZHUFVExfCDgcisQxlwHrJiiALWdRDHPJVfckBQ","signed_at":"2026-11-02T01:04:36.408Z","key":{"kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","jwk":{"kty":"OKP","crv":"Ed25519","x":"5TmKCOx0TTHepq7HwyXBHvGQpVCSBd2lV8cLzYVqFxQ"},"published_in":"https://crawlcheck.io/.well-known/http-message-signatures-directory"},"canonical":"sha256 over canonical JSON of every field except sha256 and signature"}}