Sitemap directive
A line in robots.txt, written Sitemap: followed by a full URL, that tells any crawler where the site's sitemap lives. CrawlCheck reads robots.txt and reports when no line declares a sitemap, because crawlers then have to find it by guessing or by being told elsewhere.
What Sitemap directive does not show
It points to the file; it does not show the sitemap is reachable, valid or up to date.
Terms this definition uses
Related terms in Crawling and indexing
How a crawler discovers, schedules, fetches and selects pages, and why a fetched page is not an indexed one.
Crawler identity · Product token · Identification string · Crawl frontier · Crawl queue · Crawl scheduler · Seed URL · URL discovery · Link extraction · Recrawl interval · Crawl frequency · Host politeness · Crawl rate · Crawl-delay · Robots group merging · Longest-match rule · Wildcard rule · End-anchor rule · Robots cache · Robots parse error · Robots redirect · Robots unavailable · Sitemap · Sitemap URL set · Sitemap lastmod · Sitemap hreflang · Image sitemap · Video sitemap · News sitemap · Canonical cluster · Duplicate cluster · Index selection · Crawl demand · Rendering queue · Rendered HTML · Indexing pipeline · Index freshness · Index lag · Deindexing · Removal request · URL inspection · Crawl anomaly · Bot spoofing · Shared IP range · Web Bot Auth
Questions about Sitemap directive
What is Sitemap directive?
A line in robots.txt, written Sitemap: followed by a full URL, that tells any crawler where the site's sitemap lives. CrawlCheck reads robots.txt and reports when no line declares a sitemap, because crawlers then have to find it by guessing or by being told elsewhere.
What does Sitemap directive not show or guarantee?
It points to the file; it does not show the sitemap is reachable, valid or up to date.
Which area of the glossary does Sitemap directive belong to?
Crawling and indexing: How a crawler discovers, schedules, fetches and selects pages, and why a fetched page is not an indexed one.
See it in the full glossary · 772 terms across 19 areas.
Free browser extension
See this on any site you visit, in one click.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.