{"kind":"crawlcheck-conduct","v":1,"generated_at":"2026-10-15T02:09:36.825Z","population":"verifiable agents observed on crawlcheck.io","reading":"A census of one property, not a sample of the web: what each agent did HERE, verified by source address against the operator's own published ranges. conduct is no_violations_observed when at least 50 verified requests show no fetch of a path robots.txt disallows; violations_observed when a disallowed path was fetched; insufficient_data below 50. The busiest minute is reported, not judged: this site publishes no crawl rate. site_checks_14d are counts from Visitor Resolve calls other sites made about this agent.","thresholds":{"min_verified_requests":50,"published_rate_limit":null,"disallowed_rules":["/r/","/backup/","/admin-old/"]},"agents":[{"agent":"PerplexityBot","operator":"Perplexity","purpose":"indexing","user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-13T02:14:50.805Z","last_seen":"2026-10-15T01:41:23.942Z"},"identity":{"verified_requests":7703,"forged_claims":72,"forgery_share":0.009,"reading":"verified_requests came from the ranges Perplexity publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":58,"disallowed_fetches":1,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":119,"peak_at":"2026-10-11T23:32","published_limit":null,"measured_since":"2026-10-07T04:51:16.683Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T04:51:16.683Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":90,"verified":70,"spoofed":20}},{"agent":"Googlebot","operator":"Google","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T02:14:51.611Z","last_seen":"2026-10-15T01:32:15.648Z"},"identity":{"verified_requests":6147,"forged_claims":93,"forgery_share":0.015,"reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":37,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":6,"peak_at":"2026-10-13T08:32","published_limit":null,"measured_since":"2026-10-07T03:18:09.861Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":131,"share_of_verified_since_measured":0.021,"measured_since":"2026-10-07T03:18:09.861Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":1025,"verified":613,"spoofed":412}},{"agent":"Applebot","operator":"Apple","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T12:32:42.174Z","last_seen":"2026-10-15T02:09:15.266Z"},"identity":{"verified_requests":4086,"forged_claims":83,"forgery_share":0.02,"reading":"verified_requests came from the ranges Apple publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":37,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":4,"peak_at":"2026-10-07T14:15","published_limit":null,"measured_since":"2026-10-07T03:15:19.709Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:15:19.709Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":190,"verified":183,"spoofed":7}},{"agent":"ClaudeBot","operator":"Anthropic","purpose":"training","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T02:14:49.992Z","last_seen":"2026-10-15T00:40:28.128Z"},"identity":{"verified_requests":4078,"forged_claims":68,"forgery_share":0.016,"reading":"verified_requests came from the ranges Anthropic publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":118,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":23,"peak_at":"2026-10-07T18:54","published_limit":null,"measured_since":"2026-10-07T03:48:54.788Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:48:54.788Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":98,"verified":72,"spoofed":26}},{"agent":"GPTBot","operator":"OpenAI","purpose":"training","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-12T06:20:42.861Z","last_seen":"2026-10-15T01:27:57.335Z"},"identity":{"verified_requests":4071,"forged_claims":104,"forgery_share":0.025,"reading":"verified_requests came from the ranges OpenAI publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":21,"peak_at":"2026-10-11T08:51","published_limit":null,"measured_since":"2026-10-07T18:09:36.470Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T18:09:36.470Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":298,"verified":11,"spoofed":287}},{"agent":"Bingbot","operator":"Microsoft","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-14T06:25:41.331Z","last_seen":"2026-10-15T01:48:36.580Z"},"identity":{"verified_requests":2312,"forged_claims":47,"forgery_share":0.02,"reading":"verified_requests came from the ranges Microsoft publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":41,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":4,"peak_at":"2026-10-08T07:21","published_limit":null,"measured_since":"2026-10-07T03:20:08.676Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:20:08.676Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":723,"verified":705,"spoofed":18}},{"agent":"OAI-SearchBot","operator":"OpenAI","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T02:14:49.166Z","last_seen":"2026-10-15T01:45:53.233Z"},"identity":{"verified_requests":1759,"forged_claims":79,"forgery_share":0.043,"reading":"verified_requests came from the ranges OpenAI publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":137,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":35,"peak_at":"2026-10-09T19:57","published_limit":null,"measured_since":"2026-10-07T03:30:28.967Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:30:28.967Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":714,"verified":690,"spoofed":24}},{"agent":"AhrefsBot","operator":null,"purpose":null,"user_triggered":false,"conduct":"violations_observed","window":{"first_seen":"2026-08-14T02:11:27.871Z","last_seen":"2026-10-15T02:04:58.491Z"},"identity":{"verified_requests":1757,"forged_claims":0,"forgery_share":0,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_paths_fetched","robots_txt_fetches":40,"disallowed_fetches":3,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":7,"peak_at":"2026-10-13T02:07","published_limit":null,"measured_since":"2026-10-07T23:53:27.541Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T23:53:27.541Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":7,"verified":2,"spoofed":1}},{"agent":"ChatGPT-User","operator":"OpenAI","purpose":"live fetch","user_triggered":true,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T22:52:23.214Z","last_seen":"2026-10-15T01:28:03.927Z"},"identity":{"verified_requests":602,"forged_claims":74,"forgery_share":0.109,"reading":"verified_requests came from the ranges OpenAI publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"not_applicable","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"measured","peak_requests_per_minute":2,"peak_at":"2026-10-07T21:58","published_limit":null,"measured_since":"2026-10-07T03:26:16.042Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T03:26:16.042Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":181,"verified":170,"spoofed":11}},{"agent":"SeznamBot","operator":"Seznam","purpose":"search indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-10-10T13:04:32.776Z","last_seen":"2026-10-14T06:41:22.017Z"},"identity":{"verified_requests":177,"forged_claims":0,"forgery_share":0,"reading":"verified_requests passed forward-confirmed reverse DNS to Seznam's own domains (seznam.cz); forged_claims used this agent's name from addresses that did not (impersonators, not the agent). Reverse-DNS checks began 10 October 2026; earlier hits stay unverifiable"},"robots_txt":{"verdict":"respected","robots_txt_fetches":4,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-10T22:12","published_limit":null,"measured_since":"2026-10-10T13:04:32.776Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":158,"share_of_verified_since_measured":0.893,"measured_since":"2026-10-10T13:04:32.776Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":1,"verified":1,"spoofed":0}},{"agent":"YandexBot","operator":"Yandex","purpose":"search indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-19T13:53:12.960Z","last_seen":"2026-10-15T02:00:05.158Z"},"identity":{"verified_requests":172,"forged_claims":1,"forgery_share":0.006,"reading":"verified_requests passed forward-confirmed reverse DNS to Yandex's own domains (yandex.ru, yandex.net, yandex.com); forged_claims used this agent's name from addresses that did not (impersonators, not the agent). Reverse-DNS checks began 10 October 2026; earlier hits stay unverifiable"},"robots_txt":{"verdict":"respected","robots_txt_fetches":24,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":9,"peak_at":"2026-10-14T16:35","published_limit":null,"measured_since":"2026-10-10T12:54:23.939Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-10T12:54:23.939Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":2,"verified":1,"spoofed":1}},{"agent":"Google-CWS","operator":"Google","purpose":"store listing fetch","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-09-18T16:25:39.959Z","last_seen":"2026-10-15T02:04:09.685Z"},"identity":{"verified_requests":143,"forged_claims":0,"forgery_share":0,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"respected","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-07T20:31","published_limit":null,"measured_since":"2026-10-07T11:31:33.296Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T11:31:33.296Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0}},{"agent":"Claude-User","operator":"Anthropic","purpose":"live fetch","user_triggered":true,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-12T22:57:31.977Z","last_seen":"2026-10-15T01:54:42.259Z"},"identity":{"verified_requests":93,"forged_claims":53,"forgery_share":0.363,"reading":"verified_requests came from the ranges Anthropic publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"not_applicable","robots_txt_fetches":13,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-08T16:11","published_limit":null,"measured_since":"2026-10-07T11:03:04.563Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T11:03:04.563Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":15,"verified":5,"spoofed":10}},{"agent":"Baiduspider","operator":"Baidu","purpose":"search indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-13T21:31:01.445Z","last_seen":"2026-10-14T19:48:20.654Z"},"identity":{"verified_requests":70,"forged_claims":1,"forgery_share":0.014,"reading":"verified_requests passed forward-confirmed reverse DNS to Baidu's own domains (baidu.com, baidu.jp); forged_claims used this agent's name from addresses that did not (impersonators, not the agent). Reverse-DNS checks began 10 October 2026; earlier hits stay unverifiable"},"robots_txt":{"verdict":"respected","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":3,"peak_at":"2026-10-10T13:14","published_limit":null,"measured_since":"2026-10-10T13:14:25.713Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-10T13:14:25.713Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":3,"verified":1,"spoofed":0}},{"agent":"DuckDuckBot","operator":null,"purpose":null,"user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-12T20:29:50.246Z","last_seen":"2026-10-14T06:49:54.369Z"},"identity":{"verified_requests":8,"forged_claims":0,"forgery_share":0,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":2,"peak_at":"2026-10-10T21:21","published_limit":null,"measured_since":"2026-10-10T21:21:35.448Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-10T21:21:35.448Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":3,"verified":0,"spoofed":1}},{"agent":"Perplexity-User","operator":"Perplexity","purpose":"live fetch","user_triggered":true,"conduct":"insufficient_data","window":{"first_seen":"2026-08-14T01:04:08.513Z","last_seen":"2026-10-06T23:10:26.222Z"},"identity":{"verified_requests":8,"forged_claims":50,"forgery_share":0.862,"reading":"verified_requests came from the ranges Perplexity publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"this agent fetches pages a user asked for; its operator states robots.txt does not govern those fetches, so disallowed fetches are reported, not judged"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":5,"verified":0,"spoofed":5}},{"agent":"Google-NotebookLM","operator":null,"purpose":null,"user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-09T19:50:34.360Z","last_seen":"2026-10-13T20:31:29.773Z"},"identity":{"verified_requests":7,"forged_claims":0,"forgery_share":0,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":4,"peak_at":"2026-10-09T19:50","published_limit":null,"measured_since":"2026-10-09T19:50:34.360Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-09T19:50:34.360Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0}},{"agent":"Google-Site-Verification","operator":"Google","purpose":"ownership check","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-10-03T21:54:40.584Z","last_seen":"2026-10-03T21:54:40.980Z"},"identity":{"verified_requests":2,"forged_claims":0,"forgery_share":0,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0}},{"agent":"Google-Extended","operator":"Google","purpose":"training","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-15T09:11:18.716Z","last_seen":"2026-09-14T23:53:48.097Z"},"identity":{"verified_requests":0,"forged_claims":49,"forgery_share":1,"reading":"verified_requests came from the ranges Google publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":5,"verified":0,"spoofed":5}},{"agent":"Claude-SearchBot","operator":"Anthropic","purpose":"indexing","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-16T01:38:23.331Z","last_seen":"2026-09-12T16:50:34.092Z"},"identity":{"verified_requests":0,"forged_claims":37,"forgery_share":1,"reading":"verified_requests came from the ranges Anthropic publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":14,"verified":0,"spoofed":14}},{"agent":"anthropic-ai","operator":"Anthropic","purpose":"robots.txt token, not a crawler","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-19T11:41:41.601Z","last_seen":"2026-09-24T22:51:40.795Z"},"identity":{"verified_requests":0,"forged_claims":4,"forgery_share":1,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0}},{"agent":"Applebot-Extended","operator":"Apple","purpose":"training","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-19T11:41:43.412Z","last_seen":"2026-08-19T11:42:02.678Z"},"identity":{"verified_requests":0,"forged_claims":1,"forgery_share":1,"reading":"verified_requests came from the ranges Apple publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0}},{"agent":"MistralAI-User","operator":null,"purpose":null,"user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-08-16T01:38:24.858Z","last_seen":"2026-09-12T16:50:37.743Z"},"identity":{"verified_requests":0,"forged_claims":0,"forgery_share":null,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0}},{"agent":"FeedFetcher-Google","operator":"Google","purpose":"feed fetch","user_triggered":false,"conduct":"insufficient_data","window":{"first_seen":"2026-09-28T08:19:13.036Z","last_seen":"2026-09-28T08:19:13.036Z"},"identity":{"verified_requests":0,"forged_claims":1,"forgery_share":1,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"insufficient_data","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"not_yet_measured","peak_requests_per_minute":null,"peak_at":null,"published_limit":null,"measured_since":null,"reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":null,"measured_since":null,"reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0}}],"not_assessed":[{"agent":"Meta-ExternalAgent","claimed_hits":23926,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Amazonbot","claimed_hits":38948,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Bytespider","claimed_hits":244,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"CCBot","claimed_hits":922,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Diffbot","claimed_hits":1,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"SemrushBot","claimed_hits":8887,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"DataForSeoBot","claimed_hits":3852,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Timpibot","claimed_hits":4,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"YouBot","claimed_hits":297,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"Scrapy","claimed_hits":10,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"cohere-ai","claimed_hits":9,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"},{"agent":"FacebookBot","claimed_hits":1,"why":"no address-range file this record reads, so no request can be tied to the operator; nothing it did here can be told from an impersonator"}],"census":"https://crawlcheck.io/api/telemetry","visitor_checks":"https://crawlcheck.io/api/v1/visitor/stats","code":{"version_id":"fcf10558-7138-4953-a74a-8bacb4f9f896","script_sha16":"c0549076e55aaf26","state":"attested","deployed":{"at":"2026-10-15T01:52:58.584Z","commit":"ab2ab631515c","run":"38667976278"},"build_log_entry":302,"score_version":31,"scoring_policy_sha256":"86898d2a8c88d6b56e6710f3aadf4375cbb5342d4a2f9aa017b536d95602d671","decision_rules_sha256":"18dd5636e0896692ab518050cb401744edea2e6f1086dff94fead314959bdc3e","check":"https://crawlcheck.io/api/v1/builds/fcf10558-7138-4953-a74a-8bacb4f9f896"},"sha256":"2f587ccd00df6f3923b7f02459964ee88d4e2a619a72dd511a41c3aa67c5f8bf","signature":{"v":1,"alg":"Ed25519","kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","over":"sha256","message":"crawlcheck-data-v1\n2f587ccd00df6f3923b7f02459964ee88d4e2a619a72dd511a41c3aa67c5f8bf","sig":"F_YjDVaUqin8ce7zvcxg0nd9cD5Ca8Ei6BFrLEV0oEHWjgnfXTeXWx2erizcIeFI8ne4Y_y1VMpatAJhI05sAA","signed_at":"2026-10-15T02:09:36.900Z","key":{"kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","jwk":{"kty":"OKP","crv":"Ed25519","x":"5TmKCOx0TTHepq7HwyXBHvGQpVCSBd2lV8cLzYVqFxQ"},"published_in":"https://crawlcheck.io/.well-known/http-message-signatures-directory"},"canonical":"sha256 over canonical JSON of every field except sha256 and signature"}}