Agent view · Findings · 2026-11-13
Booking links counted as missing sitemap pages
This is the response this server returns to a client that asks for Accept: text/markdown at https://crawlcheck.io/blog/booking-links-counted-as-missing-sitemap-pages: fetched when you opened this page, shown byte for byte. An assistant, a crawler that negotiates markdown, or anything reading our feed gets exactly this. The markdown leaves signed (RFC 9421, key in our key directory), so a crawler can prove the page it read is the page this server sent; the check is shown below.
| status | 200 |
|---|---|
| content-type | text/markdown; charset=utf-8 |
| vary | Accept |
| link | <https://crawlcheck.io/llms.txt>; rel="describedby"; type="text/plain", <https://crawlcheck.io/.well-known/api-catalog>; rel="service-desc"; type="application/linkset+json", <https://crawlcheck.io/blog/booking-links-counted-as-missing-sitemap-pages>; rel="cite-as", <https://crawlcheck.io/.well-known/http-message-signatures-directory>; rel="http-message-signatures-directory" |
| cache-control | public, max-age=900, stale-while-revalidate=2400 |
| bytes | 11,257 |
| words | 1,588 |
| sha-256 | 7ea280644914954b39fd69ecfd74a3ae76fc15ae10a46c7960e940947beb0567 |
| repr-digest | sha-256=:fqKAZEkUlUs5/Wns/XSjrnb8Fa4QpGx5YOlAlHvrBWc=: ← matches the body shown |
| signature-input | sig1=("@authority";req "@path";req "repr-digest" "content-type");created=1795181424;expires=1795185024;keyid="1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps";alg="ed25519";nonce="2KRgWFKiPJ8NBeMNSAI5oA";tag="crawlcheck-page" |
| signature | sig1=:gy+87g1U8e/ypEwJ0gODwSQkCPUG5UpTtU+FBk0erfpVKWbewfCzm58ZovyGvRpIvtR5Pl8jEocRaI8EX7EDBg==: |
| verified | yes — signature by key 1hHmE3fo1rg5Ig70… from https://crawlcheck.io/.well-known/http-message-signatures-directory (directory valid), valid for 3600 more s. Checked here exactly as a crawler would; redo it yourself: https://crawlcheck.io/api/tool/wba |
Findings · 2026-11-13 · By [VSNARY | Emmanuel Orta](/author) # Booking links counted as missing sitemap pages Our index alerts said five small sites had hundreds of new pages Google had never fetched. The sites had 15 to 55 pages each. The rest were booking links. On 11 November 2026 CrawlCheck's index alert reported that unfetched sitemap pages had jumped on five sites we run, for example from 19 to 501. It was false. The sites' sitemaps list 15 to 55 addresses. The extra rows were the same pages with booking and ad parameters that Google adds to links from a business listing, which appeared in search performance data and were counted as separate pages. We now drop those parameters before counting, and the alert counts sitemap addresses only. On this page - The alerts - Where the extra pages came from - Our second mistake - The fix - If you read index reports Unfetched pages reported, against the count before, per site Site A 501 reported · 19 before Site B 565 reported · 13 before Site C 357 reported · 12 before Site D 194 reported · 7 before Site E 110 reported · 3 before We run a weekly check that asks Google, page by page, whether it has fetched and indexed each address a site lists. If the number of unfetched pages jumps, it raises an alert. On 11 November it raised five in an afternoon. ## The alerts # | Site | Unfetched before | Unfetched reported | | --- | --- | --- | | A | 19 | 501 | | B | 13 | 565 | | C | 12 | 357 | | D | 7 | 194 | | E | 3 | 110 | These are small local business sites. Their sitemaps list between 15 and 55 addresses. A site with 55 pages cannot have 565 of them unfetched. ## Where the extra pages came from # The check built its list of addresses from two places: the sitemap, and the pages that appear in the site's search performance data. That second source included addresses like the home page followed by a long query string. They came from links on the business's Google listing, the booking and ad links, which carry tracking parameters. Each distinct parameter value looked like a distinct page. On one site, 494 of those rows had been added, and Google answered that 495 addresses were unknown to it. Which is true: those exact addresses are not pages, and Google does not index them. ## Our second mistake # We wrote the fix the same day and logged it as shipped. It had not reached production. Two more false alerts went out the next morning, one reporting a rise from 20 to 1,003, before we noticed and deployed it on 12 November. The record said done; the site said otherwise. We now read the live version before we mark a fix as live. ## The fix # Tracking parameters are removed before addresses are counted, so one page is one row. The alert itself now counts only addresses that came from the sitemap. Addresses seen only in performance data are kept for reference and do not raise an alert. ## If you read index reports # The same thing can inflate any report that mixes sitemap pages with pages seen in search data. If a small site suddenly shows hundreds of unknown or unindexed addresses, look at the addresses before you look for a cause. Parameters such as gsas, ahbb or rwg_token point to listing links, not to pages. Compare the count with your sitemap first; our [sitemap audit guide](/guides/xml-sitemap-audit) covers how. CrawlCheck’s dataset when this page loaded: 570,959 sites measured. Written by [Emmanuel Orta](/author) · VSNARY I build CrawlCheck and write everything published here, from measurements the verification layer took. Where a post got something wrong, the correction stays in the post. [Every post, and where I got it wrong →](/author) · [Follow on LinkedIn →](https://www.linkedin.com/in/emmanuelorta) Share this finding: [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fbooking-links-counted-as-missing-sitemap-pages%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial%26utm_campaign%3Dshare)[X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fbooking-links-counted-as-missing-sitemap-pages%3Futm_source%3Dx%26utm_medium%3Dsocial%26utm_campaign%3Dshare&text=Booking%20links%20counted%20as%20missing%20sitemap%20pages)[Email](mailto:?subject=Booking%20links%20counted%20as%20missing%20sitemap%20pages&body=https%3A%2F%2Fcrawlcheck.io%2Fblog%2Fbooking-links-counted-as-missing-sitemap-pages)[Follow Emmanuel on LinkedIn](https://www.linkedin.com/in/emmanuelorta) ## Questions this post answers ### Why did my site suddenly show hundreds of unindexed pages? Check the addresses. If they are the same few pages with long query strings, they are tracked links, such as booking or ad links from a business listing, counted as separate pages. ### What are gsas, ahbb and rwg_token in a URL? They are parameters Google adds to links from a business listing, such as booking and ad links. They identify the click, not a different page. ### Was anything actually wrong with the five sites? No. Their sitemaps list 15 to 55 addresses and those were unaffected. The alert was counting tracked versions of the same pages. ### What did CrawlCheck change? Tracking parameters are removed before addresses are counted, and the alert counts sitemap addresses only. Two false alerts went out before the fix reached production on 12 November 2026. ## Related findings [2026-11-06 Most of our 'referring pages' were forged Referer headers We counted any Referer header as a link click. Most were forged: our own addresses on control-panel ports, lookalike search domains, bare hostn](/blog/most-of-our-referring-pages-were-forged-referer-headers)[2026-10-22 Three sections said “not scored”. Our grade counted them. On 22 October 2026, three CrawlCheck report sections labelled not scored still counted in the grade. Who was affected, what we fix](/blog/three-report-sections-said-not-scored-the-grade-counted-them)[2026-08-20 Why caching only successes kept our scans slow Scans of small sites ran slow because a 'no data' answer was never remembered. Caching the miss cut a repeated 8.2 s lookup to 3 m](/blog/cache-your-misses-a-lookup-that-only-caches-on-success) The main product Found this on your own site? **We fix it for $100.** Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after. [Fix it all — $100](https://buy.stripe.com/4gM00j7OjawhdLEdyR2VG0v) [What it covers](/pricing#fix) · [scan first, it is free](/) Free browser extension Check this on any site you open, **without leaving the page.** CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account. [Add to Chrome — free](https://chromewebstore.google.com/detail/plgjengobhbdmbgckiiaaflgiepdbcpi?utm_source=crawlcheck.io&utm_medium=post) [What it does](/extension) Write for the library Measured something yourself? **Submit an article.** Send your finding, the data and the method. A person reads every submission and replies by email; accepted articles are published under your name with a link to your study. Free to submit. Articles are accepted on merit; no one can pay to have one published. [Submit an article](/submit-finding#send) [How it works](/submit-finding) · a reply within about a week Spotted a mistake or a bug on this page? [Report an error](/report-error?k=article). Confirmed errors are fixed and logged on the [corrections page](/corrections). ## Comments Comments are read before they appear. Nothing is published automatically, and no account is needed. An AI agent with a published signing key can post without the queue: it signs the request and the signature is checked against its own domain ([how](/api/comment/signed)). Those appear at once, marked *verified*. Send for review Writing about this? [Facts, live figures and marks](/press) — every number on that page is dated and traceable to a scan. [All findings](/blog) · [The dataset](/data) · [How the dataset works](/policy) We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io) #### How we measure, and our own machine files CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io). **Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing. **Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours. **Pages:** [Certificates](/certificates) · [Enterprise](/enterprise) · [MCP lockfile](/spec/mcp-lockfile) | **Registry:** [MCP: Operator](/registry/mcp/operator) · [MCP](/registry/mcp) · [AI policy](/registry/aipolicy) | **Blog:** [Booking links counted as missing sitemap pages](/blog/booking-links-counted-as-missing-sitemap-pages) | **Glossary:** [operator feed](/glossary/operator-feed) · [Sitemap lastmod](/glossary/sitemap-lastmod) · [Pillar page](/glossary/pillar-page) · [C2PA claim](/glossary/c2pa-claim) · [Dataset (schema.org)](/glossary/dataset-schema) [Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)
Same body as a file: /blog/booking-links-counted-as-missing-sitemap-pages.md · as data: /feed.json · what we tell agents site-wide: /llms.txt, /agents.md. To see any other site this way, from your own browser, after its JavaScript runs: the browser extension.