{"kind":"crawlcheck-conduct","v":1,"generated_at":"2026-11-10T00:34:25.360Z","population":"verifiable agents observed on crawlcheck.io, and on the other sites that report through their edge Workers (portfolio)","reading":"A census of one property, not a sample of the web: what each agent did HERE, verified by source address against the operator's own published ranges. conduct is no_violations_observed when at least 50 verified requests show no fetch of a path robots.txt disallows; violations_observed when a disallowed path was fetched; insufficient_data below 50. The busiest minute is reported, not judged: this site publishes no crawl rate. site_checks_14d are counts from Visitor Resolve calls other sites made about this agent.","thresholds":{"min_verified_requests":50,"published_rate_limit":null,"disallowed_rules":["/r/","/backup/","/admin-old/"]},"agents":[{"agent":"DuckDuckBot","operator":"DuckDuckGo","purpose":"indexing","user_triggered":false,"conduct":"no_violations_observed","window":{"first_seen":"2026-08-12T20:29:50.246Z","last_seen":"2026-11-09T20:32:58.270Z"},"identity":{"verified_requests":133,"forged_claims":0,"forgery_share":0,"signed_requests":19,"failed_signatures":62,"signed_share":0.143,"strongest_proof":"signature","reading":"verified_requests came from the ranges DuckDuckGo publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"robots_txt_not_fetched","robots_txt_fetches":0,"disallowed_fetches":0,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-10T21:21:35.448Z","reason":null,"robots_txt_own_fetches":0,"robots_txt_sibling_fetches":0,"robots_txt_shared_from_google":0,"reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":2,"peak_at":"2026-10-10T21:21","published_limit":null,"measured_since":"2026-10-10T21:21:35.448Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-10T21:21:35.448Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":0,"verified":0,"spoofed":0},"daily_verified":{"2026-10-27":6,"2026-10-28":7,"2026-10-29":8,"2026-10-30":1,"2026-10-31":4,"2026-11-01":3,"2026-11-02":7,"2026-11-03":6,"2026-11-04":4,"2026-11-05":6,"2026-11-06":6,"2026-11-07":4,"2026-11-08":5,"2026-11-09":3},"grade":"B","grade_reasons":["no disallowed path fetched, but it never fetched robots.txt under its own name here, so it cannot be credited with reading the rules"]}],"census":"https://crawlcheck.io/api/telemetry","visitor_checks":"https://crawlcheck.io/api/v1/visitor/stats","code":{"version_id":"0a0e84b4-b421-474c-875a-5f1b63b956f7","script_sha16":"9760663cc6199781","state":"attested","deployed":{"at":"2026-11-09T15:55:41.199Z","commit":"55d750e1fb21","run":"42649144093"},"build_log_entry":1939,"score_version":34,"scoring_policy_sha256":"21af5c858fca26199e01525fec7b19b353df30151b1d29fdeba63a28b90d44a2","decision_rules_sha256":"18dd5636e0896692ab518050cb401744edea2e6f1086dff94fead314959bdc3e","check":"https://crawlcheck.io/api/v1/builds/0a0e84b4-b421-474c-875a-5f1b63b956f7"},"sha256":"ae411971473eeff3ae5a72309c9c5a33410d32689f32a1b1cb0c8e819406b0d7","signature":{"v":1,"alg":"Ed25519","kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","over":"sha256","message":"crawlcheck-data-v1\nae411971473eeff3ae5a72309c9c5a33410d32689f32a1b1cb0c8e819406b0d7","sig":"HqOrtswxfUtMO6xxe5dys_TdAMpU--MBksdfUAGACXIR1u7DLo27Gev_x0dbbpGZkL-Bfm84ET0NhYFPY3usBA","signed_at":"2026-11-10T00:41:28.972Z","key":{"kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","jwk":{"kty":"OKP","crv":"Ed25519","x":"5TmKCOx0TTHepq7HwyXBHvGQpVCSBd2lV8cLzYVqFxQ"},"published_in":"https://crawlcheck.io/.well-known/http-message-signatures-directory"},"canonical":"sha256 over canonical JSON of every field except sha256 and signature"}}