{"kind":"crawlcheck-conduct","v":1,"generated_at":"2026-10-16T10:28:06.898Z","population":"verifiable agents observed on crawlcheck.io","reading":"A census of one property, not a sample of the web: what each agent did HERE, verified by source address against the operator's own published ranges. conduct is no_violations_observed when at least 50 verified requests show no fetch of a path robots.txt disallows; violations_observed when a disallowed path was fetched; insufficient_data below 50. The busiest minute is reported, not judged: this site publishes no crawl rate. site_checks_14d are counts from Visitor Resolve calls other sites made about this agent.","thresholds":{"min_verified_requests":50,"published_rate_limit":null,"disallowed_rules":["/r/","/backup/","/admin-old/"]},"agents":[{"agent":"AhrefsBot","operator":"Ahrefs","purpose":null,"user_triggered":false,"conduct":"not_judged","window":{"first_seen":"2026-08-14T02:11:27.871Z","last_seen":"2026-10-16T09:34:04.772Z"},"identity":{"verified_requests":2563,"forged_claims":0,"forgery_share":0,"reading":"verified_requests came from the ranges the operator publishes; forged_claims used this agent's name from outside them (impersonators, not the agent)"},"robots_txt":{"verdict":"disallowed_fetch_unrecorded","robots_txt_fetches":47,"disallowed_fetches":3,"disallowed_rules":["/r/","/backup/","/admin-old/"],"disallowed_fetch_records":[],"counted_since":"2026-10-07T23:53:27.541Z","reading":"verified requests to paths this site's robots.txt disallows for every agent, including a group that names this one; a lower bound before 7 Oct 2026 (per-path map capped at 40 paths)"},"rate":{"verdict":"measured","peak_requests_per_minute":7,"peak_at":"2026-10-13T02:07","published_limit":null,"measured_since":"2026-10-07T23:53:27.541Z","reading":"the busiest minute of verified requests observed here since the counter started, all of the operator's addresses combined; a lower bound. This site publishes no crawl rate (its robots.txt has no Crawl-delay), so the rate is reported, never judged"},"revalidation":{"conditional_requests":0,"share_of_verified_since_measured":0,"measured_since":"2026-10-07T23:53:27.541Z","reading":"requests carrying If-None-Match or If-Modified-Since: an agent that revalidates instead of refetching"},"site_checks_14d":{"checked":7,"verified":2,"spoofed":1}}],"census":"https://crawlcheck.io/api/telemetry","visitor_checks":"https://crawlcheck.io/api/v1/visitor/stats","code":{"version_id":"03a2fe48-da7f-4760-83a0-11e9b8db31a6","script_sha16":"75004d4e7c7f1c4a","state":"attested","deployed":{"at":"2026-10-16T05:34:14.964Z","commit":"4021022fa581","run":"38856409951"},"build_log_entry":419,"score_version":32,"scoring_policy_sha256":"d3c9c95d409dae4e90b8e868ff113e631e445e33c14a0cc008dda144293821b3","decision_rules_sha256":"18dd5636e0896692ab518050cb401744edea2e6f1086dff94fead314959bdc3e","check":"https://crawlcheck.io/api/v1/builds/03a2fe48-da7f-4760-83a0-11e9b8db31a6"},"sha256":"8a477b93c6686798e1d963e70e7273ff251493cebc1d72b5d730d9b87d6d0fbb","signature":{"v":1,"alg":"Ed25519","kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","over":"sha256","message":"crawlcheck-data-v1\n8a477b93c6686798e1d963e70e7273ff251493cebc1d72b5d730d9b87d6d0fbb","sig":"b_p0dU9S4s2z3PbGCmnbsOokx3GbExuMDeeVmJV1g7cWwdLUUCW39vFZ0eiarKPUMKxSWG1v8Oi1gijiNA8ZAw","signed_at":"2026-10-16T10:28:07.306Z","key":{"kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","jwk":{"kty":"OKP","crv":"Ed25519","x":"5TmKCOx0TTHepq7HwyXBHvGQpVCSBd2lV8cLzYVqFxQ"},"published_in":"https://crawlcheck.io/.well-known/http-message-signatures-directory"},"canonical":"sha256 over canonical JSON of every field except sha256 and signature"}}