{"kind":"crawlcheck-resolve","v":1,"schema":"https://crawlcheck.io/schemas/resolve.json","domain":"facebook.com","answered_at":"2026-11-20T13:40:35.851Z","reading":"Each section is declared (what the site says about itself), observed (what CrawlCheck was served) or not_measured, with when it was observed. There is no overall score: decide on the section your action depends on.","status":"measured","subject":{"domain":"facebook.com","host":"www.facebook.com","path":"/","report_id":"fzlg7wu0bh","scanned_at":"2026-11-15T04:09:30.950Z","score_version":43},"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","age_hours":129.5,"latest_report_id":"fzlg7wu0bh","is_latest":true,"newer_scan_exists":false,"last_second_reading_at":"2026-11-20T06:29:29.758Z","note":"observed_at is when CrawlCheck fetched the site; nothing here says the site is unchanged since","last_verified_at":"2026-11-20T06:29:29.758Z","fresh_until":"2026-11-22T04:09:30.950Z","stale":false,"fresh_policy":"an answer is fresh for 168 hours after observed_at; each action below has its own maximum age. last_verified_at is the latest reading of this domain by any enrolled observer."},"observers":{"primary":{"version_tag":"6792255c2bbc84a4","vantage":{"kind":"crawlcheck-edge","colo":null}},"independent_readings":4,"by_field":"https://crawlcheck.io/api/corroboration?domain=facebook.com","directory":"https://crawlcheck.io/.well-known/crawlcheck-observers.json","samples_by_network":{"networks":2,"samples":4,"groups":[{"network_class":"vultr","as_org":"Vultr Holdings, LLC","asn":20473,"samples":3,"observers":["ind:vsnary-cloudways-1"],"last_at":"2026-11-20T06:29:29.758Z"},{"network_class":"microsoft-azure","as_org":"Microsoft Corporation","asn":8075,"samples":1,"observers":["gha:emmanuelorta/crawlcheck-core"],"last_at":"2026-11-18T13:27:31.067Z"}],"window":"the latest 12 independent readings kept for this domain; the primary scan is not counted here"}},"crawl_policy":{"state":"declared","observed_at":"2026-11-15T04:09:30.950Z","source":"https://www.facebook.com/robots.txt","http_status":200,"path_tested":"/","agents":[{"agent":"KlaviyoAIBot","label":"Klaviyo Customer Agent knowledge-source crawler","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Googlebot-Image","label":"Google image crawler","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Googlebot-News","label":"Google News","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Googlebot-Video","label":"Google video crawler","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Storebot-Google","label":"Google Shopping","role":"search","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Mediapartners-Google","label":"Google AdSense ad matching; no effect on Search","role":"tool","allowed_homepage":true,"allowed_path":true,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"APIs-Google","label":"Google APIs push-notification delivery","role":"tool","allowed_homepage":true,"allowed_path":true,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"BingPreview","label":"Bing page snapshots; does not build the index","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"msnbot","label":"Microsoft crawler retired in favour of bingbot","role":"retired","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Slurp","label":"Yahoo Slurp","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Yandex","label":"Yandex (umbrella token)","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"coccocbot-web","label":"Coc Coc","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Facebot","label":"Meta link preview, legacy name","role":"social","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Magpie-crawler","label":"Brandwatch social listening","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"img2dataset","label":"img2dataset image corpus","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AwarioRssBot","label":"Awario brand monitoring, RSS","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AwarioSmartBot","label":"Awario brand monitoring","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"TurnitinBot","label":"Turnitin","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"archive.org_bot","label":"Internet Archive","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"ia_archiver","label":"Internet Archive (legacy)","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AhrefsBot","label":"Ahrefs link index","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"SemrushBot","label":"Semrush crawler","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"MJ12bot","label":"Majestic link index","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"DotBot","label":"Moz link index","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"rogerbot","label":"Moz site crawler","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"DataForSeoBot","label":"DataForSEO","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"BLEXBot","label":"WebMeUp link index","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Yeti","label":"Naver","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"YoudaoBot","label":"Youdao","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Exabot","label":"Exalead web search, retired as a standalone service","role":"retired","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Sogou web spider","label":"Sogou","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"YisouSpider","label":"Shenma mobile search","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"360Spider","label":"360 Search","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Sosospider","label":"Soso search, merged into Sogou in 2013","role":"retired","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"meta-webindexer","label":"Meta AI search index","role":"search","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"omgili","label":"Webz.io omgili","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"cohere-training-data-crawler","label":"Cohere training corpus","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"PanguBot","label":"Huawei PanGu","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Ai2Bot-Dolma","label":"Allen Institute Dolma","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"FriendlyCrawler","label":"FriendlyCrawler ML","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"VelenPublicWebCrawler","label":"Hunter (Velen) public web crawler","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"MyCentralAIScraperBot","label":"MyCentral AI","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"DeepSeekBot","label":"DeepSeek","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"ICC-Crawler","label":"NICT ICC","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"GoogleOther","label":"Google non-search fetch","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Google-CloudVertexBot","label":"Vertex AI Agents crawl, at the site owner’s request","role":"tool","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"CloudflareBrowserRenderingCrawler","label":"Cloudflare Browser Run /crawl","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Cloudflare-AutoRAG","label":"Cloudflare AutoRAG","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Cloudflare-AI-Search-External","label":"Cloudflare AI Search External","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Peer39_Crawler","label":"Peer39 ad context","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AdsBot-Google","label":"Google Ads quality","role":"tool","allowed_homepage":true,"allowed_path":true,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AmazonAdBot","label":"Amazon Ads","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AdIdxBot","label":"Microsoft Ads","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Twitterbot","label":"X link preview","role":"social","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"LinkedInBot","label":"LinkedIn link preview","role":"social","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"facebookexternalhit","label":"Facebook link preview","role":"social","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Pinterestbot","label":"Pinterest","role":"social","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Slackbot-LinkExpanding","label":"Slack unfurl","role":"social","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"OAI-SearchBot","label":"ChatGPT search index","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"ChatGPT-User","label":"ChatGPT live fetch","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Claude-SearchBot","label":"Claude search index","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Claude-User","label":"Claude live fetch","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"PerplexityBot","label":"Perplexity index","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"Perplexity-User","label":"Perplexity live fetch","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Gemini-Deep-Research","label":"Gemini research agent","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"MistralAI-User","label":"Le Chat live fetch","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"DuckAssistBot","label":"DuckDuckGo AI assist","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"YouBot","label":"You.com","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"PhindBot","label":"Phind","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Kagibot","label":"Kagi search index","role":"search","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Copilot-User","label":"Microsoft Copilot fetch","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Meta-ExternalFetcher","label":"Meta AI live fetch","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Googlebot","label":"Google Search and AI Overviews","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"bingbot","label":"Bing and Copilot index","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Applebot","label":"Apple and Siri","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Amazonbot","label":"Amazon general crawl; may train Amazon AI models","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"DuckDuckBot","label":"DuckDuckGo","role":"search","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"YandexBot","label":"Yandex","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"Baiduspider","label":"Baidu","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Seznambot","label":"Seznam","role":"regional","allowed_homepage":true,"allowed_path":true,"has_own_group":true,"wildcard_rules_shadowed":true},{"agent":"Neevabot","label":"Neeva search, closed in 2023","role":"retired","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"PetalBot","label":"Huawei Petal","role":"regional","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"GPTBot","label":"OpenAI training and index","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"ClaudeBot","label":"Anthropic training","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"anthropic-ai","label":"Anthropic legacy agent","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Claude-Web","label":"Anthropic legacy agent","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Google-Extended","label":"Gemini training and grounding","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"Applebot-Extended","label":"Apple training","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"CCBot","label":"Common Crawl","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Bytespider","label":"ByteDance","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"meta-externalagent","label":"Meta AI training","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"FacebookBot","label":"Meta legacy","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"cohere-ai","label":"Cohere","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Diffbot","label":"Diffbot knowledge graph","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Omgilibot","label":"Webz.io","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"ImagesiftBot","label":"Imagesift","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Timpibot","label":"Timpi","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AI2Bot","label":"Allen Institute","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Scrapy","label":"Scrapy framework default user agent; anyone can be behind it","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"SemrushBot-OCOB","label":"Semrush Content Toolkit crawler","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Applebot-Extended-Ads","label":"Token seen in robots.txt files; not on Apple’s crawler page","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":true,"wildcard_rules_shadowed":false},{"agent":"TikTokSpider","label":"TikTok","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"QuillBot","label":"QuillBot","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Webzio-Extended","label":"Webz.io extended","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"ProRataInc","label":"ProRata","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"AwarioBot","label":"Awario brand monitoring","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Amzn-SearchBot","label":"Amazon search eligibility, separate from Amazonbot","role":"search","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Amzn-User","label":"Amazon live fetch for Alexa questions","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"MistralAI-Index","label":"Mistral search index for Vibe","role":"search","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"MistralAI-Training","label":"Mistral training crawl","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"OAI-AdsBot","label":"OpenAI ChatGPT ads page validation","role":"tool","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Google-Agent","label":"Google user-triggered agent, acts on the web","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"Google-GeminiNotebook","label":"Gemini Notebook user-supplied URL fetch","role":"answer","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"GoogleOther-Image","label":"Google common crawler, public image URLs","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"GoogleOther-Video","label":"Google common crawler, public video URLs","role":"train","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false},{"agent":"ExaSearchBot","label":"Exa AI search index, Web Bot Auth signed","role":"search","allowed_homepage":false,"allowed_path":false,"has_own_group":false,"wildcard_rules_shadowed":false}],"signal_conflicts":[],"note":"robots.txt governs access and Content-Signal governs use; both are what the site declares, not what it serves"},"delivery":{"state":"observed","observed_at":"2026-11-15T04:09:30.950Z","url":"https://www.facebook.com/","identity_basis":"every identity was sent by CrawlCheck from its own network; none is the named operator's own crawler","identities":[{"identity":"anon","label":"Unnamed client","role":"control","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"gptbot","label":"GPTBot","role":"train","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"claudebot","label":"ClaudeBot","role":"train","status":200,"words":92,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"oaisearch","label":"OAI-SearchBot","role":"answer","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"claudesearch","label":"Claude-SearchBot","role":"answer","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"perplexity","label":"PerplexityBot","role":"answer","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"chatgpt_user","label":"ChatGPT-User","role":"answer","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"claude_user","label":"Claude-User","role":"answer","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"perplexity_user","label":"Perplexity-User","role":"answer","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"googlebot","label":"Googlebot","role":"search","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"googlebot_mobile","label":"Googlebot Smartphone","role":"search","status":200,"words":65,"blocker":null,"redirected_to":"m.facebook.com","x_robots_tag":null},{"identity":"browser","label":"Mobile browser","role":"browser","status":400,"words":30,"blocker":null,"redirected_to":"m.facebook.com","x_robots_tag":null},{"identity":"bingbot","label":"Bingbot","role":"search","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"applebot","label":"Applebot","role":"search","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"amazonbot","label":"Amazonbot","role":"train","status":400,"words":30,"blocker":null,"redirected_to":"m.facebook.com","x_robots_tag":null},{"identity":"bytespider","label":"Bytespider","role":"train","status":200,"words":0,"blocker":null,"redirected_to":"/login/","x_robots_tag":null},{"identity":"metaexternal","label":"Meta-ExternalAgent","role":"train","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null},{"identity":"ccbot","label":"CCBot","role":"train","status":200,"words":80,"blocker":null,"redirected_to":null,"x_robots_tag":null}]},"machine_files":{"state":"observed","observed_at":"2026-11-15T04:09:30.950Z","files":[{"path":"/robots.txt","status":200,"content_type":"text/plain; charset=utf-8","bytes":44806,"verdict":"fail"},{"path":"/sitemap.xml","status":200,"content_type":"text/html; charset=\"utf-8\"","bytes":365329,"verdict":"absent"},{"path":"/sitemap_index.xml","status":200,"content_type":"text/html; charset=\"utf-8\"","bytes":365350,"verdict":"absent"},{"path":"/wp-sitemap.xml","status":200,"content_type":"text/html; charset=\"utf-8\"","bytes":365224,"verdict":"absent"},{"path":"/llms.txt","status":200,"content_type":"text/html; charset=\"utf-8\"","bytes":365135,"verdict":"absent"},{"path":"/entitymap.json","status":null,"content_type":null,"bytes":null,"verdict":"absent","source":"entity map reading"}],"read_before_page":[],"hashes":"every fetched body's SHA-256 is in the sealed manifest: https://crawlcheck.io/api/artifact/6d6360dcaa2df2aa47f6e9ef4c3c234821603567c03895598e195be67267489b?body=1"},"capabilities":{"state":"observed","observed_at":"2026-11-15T04:09:30.950Z","checked":[{"kind":"a2a_agent_card","url":"https://www.facebook.com/.well-known/agent-card.json","status":404}],"declared":0,"by_safety_class":{},"verified":0,"mismatches":0,"invocation_rule":"capabilities are read from declarations only; nothing is invoked. Later verification may call read_only_public operations alone; every other class needs the site owner's authorisation and a test account, and unknown is never called","certificate":{"id":"ccap1:4fd8e26a8aee3b5acf1470568d76243597f607ed7ec605658817d0f9231e3b9b","signed":true,"entry":"https://crawlcheck.io/api/registry/capabilities?domain=facebook.com","verify":"https://crawlcheck.io/api/registry/capabilities/verify?domain=facebook.com"},"items":"https://crawlcheck.io/api/capabilities?domain=facebook.com"},"entity":{"state":"observed","observed_at":"2026-11-15T04:09:30.950Z","name":null,"wikidata_item_found":false,"links":[],"states":"corroborated = the linked page links back; unreciprocated = readable and does not; unverifiable = could not be read from here"},"mcp_registry":{"state":"observed","listed":1,"observed":1,"initialized":0,"auth_required":1,"drifted":0,"shadowing":{"flagged":0,"servers":[],"shadowed_by":0,"shadowed_by_servers":[]},"auth_audit":{"audited":1,"with_findings":1,"servers":[{"name":"io.github.facebook/devtools","url":"https://mcp.facebook.com/devtools","at":"2026-11-18T13:52:30.051Z","severity":"low","codes":["client_registration_legacy","issuer_param_not_advertised"],"requested":{"source":"challenge","count":3,"scopes":["developer_tools_mcp_app_read","developer_tools_mcp_app_management","business_management"]},"resource":"https://mcp.facebook.com/devtools"}]},"last_observed_at":"2026-10-21T11:47:54.228Z","servers":[{"name":"io.github.facebook/devtools","version":"1.0.0","endpoint":"https://mcp.facebook.com/devtools","outcome":"auth_required","tools":null}],"url":"https://crawlcheck.io/api/mcp/servers?host=facebook.com","reading":"listed by the registry (the publisher’s claim); outcome is what the endpoint answered to the MCP handshake"},"content_clock":{"homepage":{"url":"https://www.facebook.com/","observed":true,"content_sha256":"ff83d02a3b049da27ae359ce87e67cac867cf26e0069498e84fdd96d29306abf","first_observed":"2026-11-18T05:42:58.641Z","last_read":"2026-11-18T11:52:09.713Z","changes_observed":0,"main_content_last_changed":null,"unchanged_since_first_observed":"2026-11-18T05:42:58.641Z","declared":{},"date_moved_words_did_not":null,"words_changed_date_did_not":null,"api":"https://crawlcheck.io/api/v1/content-clock?url=https%3A%2F%2Ffacebook.com%2F"},"reading":"When the main content (not the bytes) of these URLs last changed, from CrawlCheck's own reads. Full history and the page's declared dates: the api link."},"self_claims":{"observed":true,"first_read":"2026-11-18T03:19:49.203Z","last_read":"2026-11-18T17:21:29.781Z","reads":3,"claims":[],"api":"https://crawlcheck.io/api/v1/claims?domain=facebook.com","reading":"what the site says about itself on its own pages, with when CrawlCheck first and last saw each value; not a statement that the claim is true"},"read_equivalence":{"url":"https://facebook.com/","observed":true,"verdict":"unverifiable","checked_at":"2026-11-18T18:18:57.669Z","first_observed":"2026-11-18T11:00:33.484Z","ai_identities_served":null,"identities":[],"api":"https://crawlcheck.io/api/v1/read-equivalence?url=https%3A%2F%2Ffacebook.com%2F","reading":"whether the main text served to AI identities is the same text a person sees in a browser, from one simultaneous read of the homepage; evidence and the differing sentences at the api link"},"rights":{"observed":true,"path":"/","read_at":"2026-11-18T23:09:59.064Z","api":"https://crawlcheck.io/api/v1/rights?domain=facebook.com","reading":"one value per use for the homepage, with the signal that decided it; signals considered, conflicts and per-path answers at the api link","train":{"value":"denied","decided_by":{"signal":"robots.txt","url":"https://facebook.com/robots.txt","evidence":"Disallow: / (group for GPTBot, its own group)","line":9,"agents":["GPTBot","ClaudeBot","Google-Extended","CCBot","Applebot-Extended","meta-externalagent"]},"conflicts":0},"ground":{"value":"denied","decided_by":{"signal":"robots.txt","url":"https://facebook.com/robots.txt","evidence":"Disallow: / (group for OAI-SearchBot, the * group)","line":9,"agents":["OAI-SearchBot","ChatGPT-User","Claude-SearchBot","Claude-User","PerplexityBot","Perplexity-User"]},"conflicts":0},"summarize":{"value":"denied","decided_by":{"signal":"robots.txt","url":"https://facebook.com/robots.txt","evidence":"Disallow: / (group for OAI-SearchBot, the * group)","line":9,"agents":["OAI-SearchBot","ChatGPT-User","Claude-SearchBot","Claude-User","PerplexityBot","Perplexity-User"]},"conflicts":0},"cite":{"value":"not_declared","decided_by":{"signal":"robots.txt","url":"https://facebook.com/robots.txt","evidence":"access is open to OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot, Bingbot on / and no signal states whether citing it is allowed"},"conflicts":0},"transact":{"value":"denied","decided_by":{"signal":"robots.txt","url":"https://facebook.com/robots.txt","evidence":"Disallow: / (group for ChatGPT-User, the * group)","line":9,"agents":["ChatGPT-User","Claude-User","Perplexity-User"]},"conflicts":0}},"reversibility":{"state":"not_applicable","observed_at":"2026-11-15T04:09:30.950Z","returns":null,"irreversible_capabilities":0,"by_class":{"irreversible_write":0,"financial":0},"reading":"no products and no irreversible or financial capability: nothing here to undo","rule":"declared = the site's own product markup states a return policy; irreversible_capabilities counts declared capabilities classed as irreversible writes or financial. Declarations only: CrawlCheck does not test that a return or refund is honoured"},"findings":{"access":"public","rule":"every finding in full; finding history, changes and the licensed sections need a licence for this domain","items":[{"fid":"f1:1136ca42b450ce8f3f7b","decision_id":"d1:0be930d6bcbc0ecc68eb","code":"ROBOTS_DISALLOW_ALL","severity":5,"path":"/robots.txt","title":"robots.txt blocks every crawler it does not name","rule_revision":2,"span_id":"5f838264088c8130","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":1,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15","the decision names the 1 fetch it read"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"every scan of this domain since it was first seen"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:0be930d6bcbc0ecc68eb","code":"ROBOTS_DISALLOW_ALL","path":"/robots.txt","severity":5,"level":"critical","points":0,"cap":"F","binding":false,"letter_now":"D","letter_without_it":"D","same_level_others":0,"says":"Would cap the letter at F, but the score (56) already reads C, so the letter is the same without it. Moves no points."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:0be930d6bcbc0ecc68eb"},"url":"https://crawlcheck.io/api/explain?id=d1:0be930d6bcbc0ecc68eb"},"disputes":[]},{"fid":"f1:6880a95c3a7841dff5bd","decision_id":"d1:50f62b2ce5915088386b","code":"CRAWLER_SERVED_EXTRA","severity":4,"path":"/","title":"Crawlers are given scripts, links or instructions that visitors never see","rule_revision":2,"span_id":"089c7243878380ec","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":2,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15","the decision names the 2 fetches it compared"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"full scans only (a daily light scan cannot see this finding)"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:50f62b2ce5915088386b","code":"CRAWLER_SERVED_EXTRA","path":"/","severity":4,"level":"high","points":0,"cap":"D","binding":true,"letter_now":"D","letter_without_it":"F","same_level_others":0,"says":"Caps the letter at D. Without it the letter would be F (score 56). Moves no points; the score is built from the measured rows."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:50f62b2ce5915088386b"},"url":"https://crawlcheck.io/api/explain?id=d1:50f62b2ce5915088386b"},"disputes":[]},{"fid":"f1:01b7273681d8dd78174d","decision_id":"d1:15a2cb2ce2a48c3984b1","code":"ROBOTS_RULES_SHADOWED","severity":3,"path":"/robots.txt","title":"robots.txt rules do not apply to named agents","rule_revision":2,"span_id":"ebcf1f3c5bd1b8d2","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":1,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15","the decision names the 1 fetch it read"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"every scan of this domain since it was first seen"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:15a2cb2ce2a48c3984b1","code":"ROBOTS_RULES_SHADOWED","path":"/robots.txt","severity":3,"level":"medium","points":0,"cap":"C","binding":true,"letter_now":"D","letter_without_it":"F","same_level_others":1,"says":"Caps the letter at C. Without it the letter would be F (score 56). Moves no points; the score is built from the measured rows."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:15a2cb2ce2a48c3984b1"},"url":"https://crawlcheck.io/api/explain?id=d1:15a2cb2ce2a48c3984b1"},"disputes":[]},{"fid":"f1:233d61c7a603121758a2","decision_id":"d1:38d85d662e4f52f7bdc8","code":"PAGE_IS_MOSTLY_CODE","severity":3,"path":"/","title":"Almost nothing a machine receives from this page is readable text","rule_revision":1,"span_id":"ac69927930dd7282","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":1,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15","the decision names the 1 fetch it read"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"full scans only (a daily light scan cannot see this finding)"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:38d85d662e4f52f7bdc8","code":"PAGE_IS_MOSTLY_CODE","path":"/","severity":3,"level":"medium","points":0,"cap":"C","binding":true,"letter_now":"D","letter_without_it":"F","same_level_others":1,"says":"Caps the letter at C. Without it the letter would be F (score 56). Moves no points; the score is built from the measured rows."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:38d85d662e4f52f7bdc8"},"url":"https://crawlcheck.io/api/explain?id=d1:38d85d662e4f52f7bdc8"},"disputes":[]},{"fid":"f1:902c1a70740267896f40","decision_id":"d1:d8e5da9385ddd2e2f3d7","code":"DECLARED_SITEMAP_REFUSED","severity":2,"path":"/robots.txt","title":"The sitemap robots.txt declares refused this client","rule_revision":1,"span_id":"2d2af50082b63d19","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":4,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15","the decision names the 4 fetches it read"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"every scan of this domain since it was first seen"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:d8e5da9385ddd2e2f3d7","code":"DECLARED_SITEMAP_REFUSED","path":"/robots.txt","severity":2,"level":"low","points":0,"cap":null,"binding":false,"letter_now":"D","letter_without_it":"F","same_level_others":1,"says":"Does not cap the letter: only medium, high and critical findings do. Moves no points either; the score is built from the measured rows."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:d8e5da9385ddd2e2f3d7"},"url":"https://crawlcheck.io/api/explain?id=d1:d8e5da9385ddd2e2f3d7"},"disputes":[]},{"fid":"f1:6b76b9c95d615beb484c","decision_id":"d1:a1b8ecd297d4a7928b88","code":"HSTS_PRELOAD_INELIGIBLE","severity":2,"path":"/","title":"The HSTS header asks for browser preloading but does not qualify for it","rule_revision":1,"span_id":"e868e45f58e2de38","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":1,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15","the decision names the 1 fetch it read"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"every scan of this domain since it was first seen"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:a1b8ecd297d4a7928b88","code":"HSTS_PRELOAD_INELIGIBLE","path":"/","severity":2,"level":"low","points":0,"cap":null,"binding":false,"letter_now":"D","letter_without_it":"F","same_level_others":1,"says":"Does not cap the letter: only medium, high and critical findings do. Moves no points either; the score is built from the measured rows."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:a1b8ecd297d4a7928b88"},"url":"https://crawlcheck.io/api/explain?id=d1:a1b8ecd297d4a7928b88"},"disputes":[]},{"fid":"f1:ddc338a7485046f661aa","decision_id":"d1:8d57dcde97639c130f0d","code":"REDIRECT_CHAIN","severity":1,"path":"/","title":"The homepage is reached through a chain of redirects","rule_revision":1,"span_id":"a73039ced4165a74","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":0,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"every scan of this domain since it was first seen"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:8d57dcde97639c130f0d","code":"REDIRECT_CHAIN","path":"/","severity":1,"level":"info","points":0,"cap":null,"binding":false,"letter_now":"D","letter_without_it":"F","same_level_others":1,"says":"Does not cap the letter: only medium, high and critical findings do. Moves no points either; the score is built from the measured rows."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:8d57dcde97639c130f0d"},"url":"https://crawlcheck.io/api/explain?id=d1:8d57dcde97639c130f0d"},"disputes":[]},{"fid":"f1:2ffb876c77f1c88c3c58","decision_id":"d1:4da18a9961c3dbcd852b","code":"NO_LLMS_TXT","severity":1,"path":"/llms.txt","title":"No llms.txt","rule_revision":1,"span_id":"f3cd137e3425a621","state":"new","valid_time":{"valid_from":{"after":null,"by":"2026-11-15T04:09:30.950Z"},"valid_to":null,"recorded_at":"2026-11-15T04:09:56.836Z","superseded_at":null,"superseded_by":null,"earlier_intervals":0,"before_bt1":false},"observations_linked":1,"trust":{"freshness":{"observed_at":"2026-11-15T04:09:30.950Z","last_observed":"2026-11-15T04:09:30.950Z","held_since":"2026-11-15T04:09:56.836Z","state":"new"},"confidence":{"level":"single_reading","means":"one reading from one network","basis":["seen in 1 scan since 2026-11-15","the decision names the 1 fetch it read"],"sample":{"cross_network":"not_applicable","why":"this finding does not depend on the requesting identity or network, so other networks' readings neither confirm nor contradict it","readings":1,"diversity":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{}},"scans":{"seen":1,"could_have_seen":1,"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"counts":"every scan of this domain since it was first seen"},"interval":{"k":1,"n":1,"share":1,"low":0.207,"high":1,"level":0.95,"method":"Wilson score interval"},"interval_of":"scans that saw it, of scans that could have","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":[]},"score_effect":{"decision_id":"d1:4da18a9961c3dbcd852b","code":"NO_LLMS_TXT","path":"/llms.txt","severity":1,"level":"info","points":0,"cap":null,"binding":false,"letter_now":"D","letter_without_it":"F","same_level_others":1,"says":"Does not cap the letter: only medium, high and critical findings do. Moves no points either; the score is built from the measured rows."},"root_cause":null,"corroboration":{"status":"not_applicable","reading":"this finding does not depend on the requesting identity or network"},"explain":{"mcp_tool":"explain_finding","arguments":{"id":"d1:4da18a9961c3dbcd852b"},"url":"https://crawlcheck.io/api/explain?id=d1:4da18a9961c3dbcd852b"},"disputes":[]}],"summary":{"total":8},"withheld":null},"confidence":{"identities_answered":{"n":16,"of":18,"ratio":0.889},"vantages":1,"vantage_state":{"state":"single_vantage","means":"only one network read this site near the scan; nothing here is cross-checked","networks":1,"readings":1,"countries":0,"findings_contested":0,"findings_cross_confirmed":0,"states":{"single_vantage":"only one network read this site near the scan; nothing here is cross-checked","insufficient_samples":"more than one network read it, but fewer than 3 readings: too few to state agreement","cross_confirmed":"two or more networks read it, 3 or more readings, and no finding is contested between them","network_divergence":"networks disagree on at least one finding","regional_divergence":"networks in more than one country disagree on at least one finding: what is served depends on where the request comes from"}},"note":"counts and ratios only - there is no composite confidence number, because the parts do not combine into one honestly","sample":{"readings":1,"observers":1,"distinct_networks":1,"networks":{"crawlcheck":1},"distinct_countries":0,"countries":{},"window_hours":6,"all_time":{"independent_readings":4,"networks":["vultr","microsoft-azure"]},"reading":"readings of this domain within 6 hours of the scan: CrawlCheck's own plus every independent observer's, with the networks and countries they arrived from","note":"interval: the 95% Wilson score interval for the share, from k of n; it is wide when n is small, and that width is the point. networks: autonomous systems the readings arrived from (two observers on one network are one network). countries: where the connection was measured; CrawlCheck's own reading leaves from an edge location and carries no country here."}},"contradictions":{"site":[],"between_networks_open":[],"locked_findings_not_shown":0},"facts":{"state":"observed","observed_at":"2026-11-15T04:09:30.950Z","count":0,"facts":[],"not_facts":{"page_only":0,"page_inconsistent":0,"unverified":0,"contradicted":0},"rule":"A fact here is a claim the site makes that an outside source (a listing, a Business Profile, a profile that links back, registration data) was seen to agree with on observed_at. It is signed with the rest of this answer. Claims only the page makes are not facts.","claims":"https://crawlcheck.io/r/fzlg7wu0bh.claims.json"},"evidence":{"report":"https://crawlcheck.io/r/fzlg7wu0bh","machine_record":"https://crawlcheck.io/api/v1/machine-record?id=fzlg7wu0bh","manifest_sha256":"6d6360dcaa2df2aa47f6e9ef4c3c234821603567c03895598e195be67267489b","verify":"https://crawlcheck.io/api/verify?id=fzlg7wu0bh","bundle":"https://crawlcheck.io/api/bundle?id=fzlg7wu0bh","offline_verifier":"https://crawlcheck.io/crawlcheck-verify.mjs?v=6a51f060110f86b8","verify_this_answer":"save it as answer.json, then: node crawlcheck-verify.mjs answer.json"},"impersonation":{"state":"known_site","verdict":"known_site","brand_frame":"Tranco top 10,000 (Y83YG)"},"stored_sections":{"sections":["mcp_registry"],"as_of":"2026-11-20T06:29:54.360Z","reading":"these sections did not answer within 180 ms, so they are shown as they were when this answer was stored; none of them changes a decision"},"identity":{"state":"unknown","name":null,"type":null,"id":null,"independent_confirmations":0,"confirmed_by":[],"claimed_profiles_not_linking_back":0,"related_sites_linking_back":[],"confirm_with":null,"observed_at":"2026-11-15T04:09:30.950Z","reading":"the site does not say which business it is","rule":"confirmed when at least one source independent of the site (a profile on a public platform that links back, a Business Profile or a listing that shows the same name or phone) agrees; each source counts once. Other websites that link back are listed as related sites and not counted, since one operator can run several. Signed with this answer."},"actions":{"v":1,"decisions":"allow | warn | require_confirmation | block | unsupported","rule":"decided from this answer's own sections at signing time; a policy sent with the request can only tighten it; the deployment's own policy decides whether its agent acts","read":{"decision":"warn","minimum_assurance":"reachable","assurance_found":["declared","reachable"],"needs":"robots.txt lets AI agents read the path, and the page was served to at least one AI-crawler identity","why":"11 of 12 AI-crawler identities were served the page; robots.txt allows 18 of 116 tested agents - check your own user agent in crawl_policy.agents","doubts":["agent_specific_robots"],"max_age_hours":168},"cite":{"decision":"warn","minimum_assurance":"reachable + entity observed","assurance_found":["declared","reachable"],"needs":"everything read needs, plus an observed entity and no contradiction the site makes with itself","scope":{"covers":["access: robots.txt and what was actually served to AI-crawler identities","permissions: the rights and content signals the site declares","identity: the entity the site declares and who has proven control of the domain","provenance: when, from where and by whom it was observed, signed and logged"],"does_not_cover":"whether the content is true, accurate, current or complete; a cite decision says the source may be cited and attributed, not that what it says is right"},"why":"11 of 12 AI-crawler identities were served the page; robots.txt allows 18 of 116 tested agents - check your own user agent in crawl_policy.agents; the publisher's identity was not observed, so attribution is unconfirmed","doubts":["agent_specific_robots","entity_unobserved"],"max_age_hours":168},"connect":{"decision":"warn","minimum_assurance":"conformant","assurance_found":["listed"],"needs":"a declared capability that CrawlCheck invoked read-only and found to behave as declared","why":"the official MCP Registry lists 1 MCP server(s) here; none has completed the MCP handshake when CrawlCheck tried (1 asked for credentials)","doubts":["registry_unverified"],"max_age_hours":72},"transact":{"decision":"require_confirmation","minimum_assurance":"action_ready","assurance_found":[],"needs":"owner-verified identity and an action-ready capability; CrawlCheck does not certify transactions, so a human confirms","why":"CrawlCheck does not certify transactions; a human confirms every financial, identity or irreversible action; the identity is unknown: the site does not say which business it is. Confirm who you are paying through a channel you already trust (a number you found yourself, not one on this site) before acting","doubts":["identity_unknown"],"max_age_hours":24},"administer":{"decision":"block","minimum_assurance":"never automatic","assurance_found":[],"needs":"high-privilege operations are never authorised by a public record","why":"never authorised by a public record","doubts":[],"max_age_hours":0}},"absences":[{"what":"a2a_agent_card","result":"absent","at":"2026-11-15T04:09:30.950Z","from":{"observer":"crawlcheck-primary","network":"crawlcheck","directory":"https://crawlcheck.io/.well-known/crawlcheck-observers.json"},"checked":[{"method":"GET","url":"https://www.facebook.com/.well-known/agent-card.json","status":404,"redirects":"followed"}],"scope":"Every location CrawlCheck checks for a a2a agent card answered not found (404). No declaration of this kind was served at those URLs at that time; one served elsewhere, or added later, is not covered."}],"control":{"state":"unproven","proofs":[],"organization":null,"claim":"https://crawlcheck.io/registry/facebook.com/claim","why":"nobody has proven control of facebook.com to CrawlCheck, so who stands behind the site is unconfirmed","note":"control says who answers for the domain, not whether it is safe; it does not change any decision in this answer"},"transparency":{"state":"included","log_id":"crawlcheck-resolve-log-v1","leaf_hash":"2f44d509c6b52b7ec07516f2c3d96b14c519f2d34effd224e5f7ceb7713652af","batch":2623,"index":0,"batch_size":500,"path":["d4d60ad4c6085ecb98447b66114eaea392150d622384e086adcbc436301d1e3f","2f393d3267bbfd092c1bb8310938bb5c425dabd965d1adaf311929e426d7bd95","b8c59d8b57f2640ca2412fde732e2a1cae5373a19c38f165b25ff158699bd9a4","d41577ea59a52579c693e7360ca090a1082b9d5b1ec78fc95794705b203b3161","2cd4024551ec91d4b58c4875f52fc7a0c07d672070cfe7153704e6dbb046cff4","61643207383398ff85dbefd109669a0323cdb0af1c46a6856a552ce6889e09c8","1400b5f3b6cc9c22886d2badd1312c1c0634a8c053ad4878b61c3689ec0bb1e9","0f765ca341fadb7697e1f1d89f1a04b63e578ac7887fcc3eac71f4283b4d97fd","34303d2ebcd10e550ad25da0df55092e2ed062b945cc1ef09faa7a4a5fc71ef4"],"sth":{"kind":"crawlcheck-log-sth","v":1,"log_id":"crawlcheck-resolve-log-v1","batch":2623,"first_index":584516,"batch_size":500,"tree_size":585016,"batch_root":"26d1bffa4384b101180fb9094738ba6e3c0048c338deb1241a6adc85e0022bc4","prev_sth_sha256":"15286c3ac297088543928d20b3d51748a07dcbdd6d55038eefcf7a18ba8a68e3","created_at":"2026-11-18T11:48:28.629Z","rules":{"leaf":"SHA-256(0x00 || canonical JSON of the answer's decision content)","tree":"RFC 9162 Merkle tree hash over this batch's leaf hashes, sorted","chain":"prev_sth_sha256 is the sha256 of the previous batch's signed tree head; batch numbers are consecutive and tree_size grows by batch_size"},"code":{"build_id":"46bb70ee9700f1b4","script_sha16":"54c7b8bb9b8d7f2f","state":"attested","deployed":{"at":"2026-11-18T06:28:27.997Z"},"build_log_entry":2780,"score_version":43,"scoring_policy_sha256":"3e08fccfbbce356dc82ecd302b32a226a36fc0fb3150a4ab72248fb5795372af","decision_rules_sha256":"18dd5636e0896692ab518050cb401744edea2e6f1086dff94fead314959bdc3e","check":"https://crawlcheck.io/api/v1/builds/46bb70ee9700f1b4"},"sha256":"3994dd58eb5d87c4df6a5f828c918263f3f59a4679ec0145a6abc91e9e15c3e4","signature":{"v":1,"alg":"Ed25519","kid":"AqR_NNfBCBE0A_KXkQ86yrLAHiC-JbA5Df0BSV1PIEU","over":"sha256","message":"crawlcheck-data-v1\n3994dd58eb5d87c4df6a5f828c918263f3f59a4679ec0145a6abc91e9e15c3e4","sig":"YibOqvi_mZjF6l-nmBY61wy74VFSoh1ijStLtBDNjzXXXLDrlnSmRGyFHmDqmyKONJ8ELGS8U6OhIWKJ_UiSBQ","signed_at":"2026-11-18T11:48:28.629Z","key":{"kid":"AqR_NNfBCBE0A_KXkQ86yrLAHiC-JbA5Df0BSV1PIEU","jwk":{"kty":"OKP","crv":"Ed25519","x":"9D4KKgcLUxe8CkTNJ7yfZLsSb8v_a0-9FnKODxtl-2c"},"published_in":"https://crawlcheck.io/.well-known/http-message-signatures-directory"},"canonical":"sha256 over canonical JSON of every field except sha256 and signature"}},"witnesses":"https://crawlcheck.io/api/v1/log/sth/2623","content_rule":"leaf = SHA-256(0x00 || canonical JSON of the answer's decision content); see https://crawlcheck.io/api/v1/log"},"assurance_ladder":{"v":1,"level":"reachable","order":["declared","discovered","reachable","conformant","owner_verified","authorized","reliable","action_ready"],"rungs":[{"rung":"declared","state":"reached","why":"robots.txt policy declared; 1 MCP server(s) in the registry"},{"rung":"discovered","state":"reached","why":"0 declared operation(s), 1 registry server(s)"},{"rung":"reachable","state":"reached","why":"AI-crawler identities were served the page"},{"rung":"conformant","state":"not_measured","why":"no same-origin, login-free MCP server or A2A agent to run the read-only suite against, and no capability verified by invocation"},{"rung":"owner_verified","state":"not_reached","why":"nobody has proven control of facebook.com to CrawlCheck, so who stands behind the site is unconfirmed"},{"rung":"authorized","state":"not_reached","why":"the domain's owner has published no authorization for agents"},{"rung":"reliable","state":"not_measured","why":"reliability is measured for MCP endpoints over time; none on this domain is on the panel"},{"rung":"action_ready","state":"not_reached","why":"not yet: conformant, owner_verified, authorized, reliable"}],"rule":"level is the highest rung with every rung under it reached; a rung is reached only on measured evidence, not_reached when measured and short, not_measured when there was nothing to measure. The ladder describes evidence; each action's decision is still made in actions"},"evidence_assurance":{"v":1,"level":"C","A":{"name":"record integrity","state":"reached","how":"this answer is signed (signature.kid) and appended to the public transparency log (transparency.leaf_hash). Check with the SDK's accepted field, not verified"},"B":{"name":"reproducible decision","state":"reached","how":"the evidence bundle this answer was built from is sealed (evidence.manifest_sha256) and this answer is archived by its sha256, so every decision made from it can be re-run with POST https://crawlcheck.io/api/v1/preflight/replay"},"C":{"name":"corroborated reading","state":"reached","how":"4 independent readings from 2 networks outside CrawlCheck's scanner; per field: https://crawlcheck.io/api/corroboration?domain=facebook.com"},"limits":"A says the record is unchanged and who signed it. B says the published rules give this decision again from the preserved inputs; it does not say the rules are well calibrated (see accuracy). C says an independent observer saw comparable content; it does not say every visitor gets the same content. None of them says the site is safe, that a user authorised the action, or that the action was carried out as assessed."},"code":{"build_id":"08ef0ab4de3131a4","script_sha16":"71b0cd4b436ef1f0","state":"attested","deployed":{"at":"2026-11-20T01:39:33.365Z"},"build_log_entry":2885,"score_version":43,"scoring_policy_sha256":"3e08fccfbbce356dc82ecd302b32a226a36fc0fb3150a4ab72248fb5795372af","decision_rules_sha256":"18dd5636e0896692ab518050cb401744edea2e6f1086dff94fead314959bdc3e","check":"https://crawlcheck.io/api/v1/builds/08ef0ab4de3131a4"},"sha256":"7ce987b87bdc748a15679955243281e57cbe56ffc8c5a7dbb65c92eb278bc892","signature":{"v":1,"alg":"Ed25519","kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","over":"sha256","message":"crawlcheck-data-v1\n7ce987b87bdc748a15679955243281e57cbe56ffc8c5a7dbb65c92eb278bc892","sig":"LRK0T7GIsWEv8Mqcu7N4GK2VFtaukYNb5dUZ4lgaPKBLv8qojEmp2hiPkOe6nR0brYdn_IsDvdVCgemZePTrAA","signed_at":"2026-11-20T13:40:35.883Z","key":{"kid":"1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps","jwk":{"kty":"OKP","crv":"Ed25519","x":"5TmKCOx0TTHepq7HwyXBHvGQpVCSBd2lV8cLzYVqFxQ"},"published_in":"https://crawlcheck.io/.well-known/http-message-signatures-directory"},"canonical":"sha256 over canonical JSON of every field except sha256 and signature"}}