cache-buster
A random query parameter appended to a URL so that the request misses every cache keyed on the URL and reaches the origin.
What cache-buster does not show
It is a measurement tool, not a fix: it shows what the origin serves now, and it says nothing about what the edge is still serving to everyone who asks for the bare URL.
How CrawlCheck measures cache-buster
3 finding codes measure it. Each one’s share of all scans is computed from the live corpus when this page is served, not typed into it.
| Finding code | Share of scans | Guide |
|---|---|---|
MACHINE_FILE_CACHE_STALE | 0.4% | Does llms.txt work? What is known and how to test it |
MACHINE_FILE_CACHE_SPLIT | 0% | Stale cached robots.txt or llms.txt? Two fetches tell you |
ORIGIN_BLOCKED_BEHIND_CACHE | 0.2% | Is Cloudflare blocking AI crawlers on your site? |
Every code, its rule and its current rate: the dataset page. Scan a site to see which apply to it.
Where cache-buster comes up on this site
- Is Cloudflare blocking AI crawlers on your site?Then compare identities on the bare URL, never with a cache-buster, because a crawler never appends one: curl -s -o /dev/null -w "%{http_code} %{size_download}\n" -A "Mozilla/5.0" https://yoursite.com/ curl -s -o /dev/null -w "%{http_code}…
- How to check if AI crawlers can read your siteThen fetch it again with a cache-buster and compare.
- How to rank in ChatGPT: the part that is in your controlA stale cached copy means crawlers and browsers are told different things — and it hides from testing, because adding a cache-buster to the URL skips the cache.
- Do AI crawlers and agents respect robots.txt?A cache-buster asks the origin a fresh question and bypasses whatever the edge is holding, so the file comes back perfect and the real defect stays invisible.
- How to write an llms.txt and check it is actually servedFetch the bare URL with no cache-buster, because a crawler never adds one, and do it from outside your own network: your browser has its own cache, sends your session and renders an HTML error page convincingly. curl -sI https://example.com/llms.txt curl…
Terms this definition uses
Related terms in HTTP, edge and rendering
The delivery layer between an origin and a client: proxies, caches, redirects, status codes and the headers that decide what a crawler actually received.
Content-Language header · HTTP/3 · Alt-Svc · HTTPS DNS record · Link header · content negotiation · Markdown negotiation · Retry-After · Server-Timing · WWW-Authenticate · Allow header · Content-Digest · Origin server · Reverse proxy · CDN · Edge node · WAF · Bot score · TLS handshake · DNS resolution · DNS timeout · Connection timeout · Read timeout · Redirect chain · Redirect loop · 301 redirect · 302 redirect · 307 redirect · 308 redirect · 304 response · 404 response · 410 response · 429 response · 500 response · 502 response · 503 response · 504 response · HEAD request · GET request · Content type · MIME sniffing · Content encoding · Compression · Brotli · ETag · Last-Modified · If-None-Match · Cache hit · Cache miss · Cache key · Vary header · Stale response · Stale-while-revalidate · Edge redirect · Origin bypass · Response parity · Header parity · hreflang · canonical URL · HSTS · HSTS preload list · Content-Security-Policy · CSP nonce · edge cache · cache split · Cache-Control · Age header · render-blocking resource · payload text share · lazy loading · 404 and 410 status codes
Questions about cache-buster
What is cache-buster?
A random query parameter appended to a URL so that the request misses every cache keyed on the URL and reaches the origin.
What does cache-buster not show or guarantee?
It is a measurement tool, not a fix: it shows what the origin serves now, and it says nothing about what the edge is still serving to everyone who asks for the bare URL.
How does CrawlCheck measure cache-buster?
Through the finding codes MACHINE_FILE_CACHE_STALE, MACHINE_FILE_CACHE_SPLIT, ORIGIN_BLOCKED_BEHIND_CACHE, each with a stated rule, evidence bytes and a live share of scans on the dataset page.
Which area of the glossary does cache-buster belong to?
HTTP, edge and rendering: The delivery layer between an origin and a client: proxies, caches, redirects, status codes and the headers that decide what a crawler actually received.
← cache split · Cache-Control →
See it in the full glossary · 792 terms across 19 areas.
Free browser extension
See this on any site you visit, in one click.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.