Sitemap
A machine-readable list of canonical URLs and optional metadata offered for discovery.
What Sitemap does not show
Submission helps crawlers learn URLs but does not guarantee crawling, indexing, or ranking.
How CrawlCheck measures Sitemap
14 finding codes measure it. Each one’s share of all scans is computed from the live corpus when this page is served, not typed into it.
| Finding code | Share of scans | Guide |
|---|---|---|
NO_SITEMAP_FOUND | 9.9% | Do AI crawlers and agents respect robots.txt? |
SITEMAP_BLOCKED | 1.9% | Do AI crawlers and agents respect robots.txt? |
SITEMAP_IS_HTML | 0.4% | Do AI crawlers and agents respect robots.txt? |
SITEMAP_NO_LASTMOD | 7.5% | XML sitemap audit: check the file and every URL in it |
SITEMAP_ORIGIN_ERROR | 0.2% | XML sitemap audit: check the file and every URL in it |
SITEMAP_UNREADABLE | 0.5% | XML sitemap audit: check the file and every URL in it |
DECLARED_SITEMAP_BROKEN | 0.9% | Do AI crawlers and agents respect robots.txt? |
DECLARED_SITEMAP_REFUSED | 0.7% | XML sitemap audit: check the file and every URL in it |
SITEMAP_DECLARES_NOTHING | 1.8% | dataset |
SITEMAP_CHILD_BROKEN | 0.1% | dataset |
ROBOTS_SITEMAP_RELATIVE | 0.1% | dataset |
SITEMAP_LASTMOD_INVALID | 0% | dataset |
ROBOTS_NO_SITEMAP | 8.6% | Do AI crawlers and agents respect robots.txt? |
SITEMAP_URL_OFF_HOST | 0.1% | dataset |
Every code, its rule and its current rate: the dataset page. Scan a site to see which apply to it.
Where Sitemap comes up on this site
- XML sitemap audit: check the file and every URL in itA sitemap can return 200 , open in a browser, look orderly under the browser's XML stylesheet, and still be unusable.
- Canonical URL audit: do tags, redirects and sitemap agree?The failures that matter are elsewhere: two hostnames answering 200 for the same bytes, a sitemap listing a different form of the address, a template pointing a whole section's canonicals somewhere else, a domain that is only half an alias.
- Crawl budget: measure what a crawler reads in 50 fetchesA crawler finds pages two ways: by following links from pages it has read, starting at the homepage, or by reading your sitemap and queueing every address it lists.
- Release notes 29 Sep 2026: finding states, sitemap, trustIf your own sitemap has grown large, our sitemap audit guide covers when and how to split it.
- Sitemap lastmod: 103 pages, one date, why engines skip itA sitemap is a list of a site's pages, and the lastmod date beside each one tells a search engine or AI crawler which pages changed since it last looked.
Terms this definition uses
canonical · canonical URL · Offer
Definitions that name Sitemap
soft 404 on a machine file · sitemap origin error · case consistency · index coverage · canonical · sitemap index · orphan page · Sitemap URL set · Sitemap hreflang · Image sitemap · Video sitemap · News sitemap · changefreq and priority · service area · Sitemap directive
Related terms in Crawling and indexing
How a crawler discovers, schedules, fetches and selects pages, and why a fetched page is not an indexed one.
Sitemap directive · Crawler identity · Product token · Identification string · Crawl frontier · Crawl queue · Crawl scheduler · Seed URL · URL discovery · Link extraction · Recrawl interval · Crawl frequency · Host politeness · Crawl rate · Crawl-delay · Robots group merging · Longest-match rule · Wildcard rule · End-anchor rule · Robots cache · Robots parse error · Robots redirect · Robots unavailable · Sitemap URL set · Sitemap lastmod · Sitemap hreflang · Image sitemap · Video sitemap · News sitemap · Canonical cluster · Duplicate cluster · Index selection · Crawl demand · Rendering queue · Rendered HTML · Indexing pipeline · Index freshness · Index lag · Deindexing · Removal request · URL inspection · Crawl anomaly · Bot spoofing · Shared IP range · Web Bot Auth
Questions about Sitemap
What is Sitemap?
A machine-readable list of canonical URLs and optional metadata offered for discovery.
What does Sitemap not show or guarantee?
Submission helps crawlers learn URLs but does not guarantee crawling, indexing, or ranking.
How does CrawlCheck measure Sitemap?
Through the finding codes NO_SITEMAP_FOUND, SITEMAP_BLOCKED, SITEMAP_IS_HTML, SITEMAP_NO_LASTMOD, SITEMAP_ORIGIN_ERROR, SITEMAP_UNREADABLE, DECLARED_SITEMAP_BROKEN, DECLARED_SITEMAP_REFUSED, SITEMAP_DECLARES_NOTHING, SITEMAP_CHILD_BROKEN, ROBOTS_SITEMAP_RELATIVE, SITEMAP_LASTMOD_INVALID, ROBOTS_NO_SITEMAP, SITEMAP_URL_OFF_HOST, each with a stated rule, evidence bytes and a live share of scans on the dataset page.
Which area of the glossary does Sitemap belong to?
Crawling and indexing: How a crawler discovers, schedules, fetches and selects pages, and why a fetched page is not an indexed one.
← Robots unavailable · Sitemap URL set →
See it in the full glossary · 772 terms across 19 areas.
Free browser extension
See this on any site you visit, in one click.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.