Agent view · Guides · 2026-10-02
IndexNow: which engines use it and how to check it worked
This is the response this server returns to a client that asks for Accept: text/markdown at https://crawlcheck.io/guides/indexnow-what-it-does-who-reads-it-and-how-to-check-it-worked: fetched when you opened this page, shown byte for byte. An assistant, a crawler that negotiates markdown, or anything reading our feed gets exactly this. The markdown leaves signed (RFC 9421, key in our key directory), so a crawler can prove the page it read is the page this server sent; the check is shown below.
| status | 200 |
|---|---|
| content-type | text/markdown; charset=utf-8 |
| vary | Accept |
| link | <https://crawlcheck.io/llms.txt>; rel="describedby"; type="text/plain", <https://crawlcheck.io/.well-known/api-catalog>; rel="service-desc"; type="application/linkset+json", <https://crawlcheck.io/guides/indexnow-what-it-does-who-reads-it-and-how-to-check-it-worked>; rel="cite-as", <https://crawlcheck.io/.well-known/http-message-signatures-directory>; rel="http-message-signatures-directory" |
| cache-control | public, max-age=900, stale-while-revalidate=2400 |
| bytes | 16,015 |
| words | 2,340 |
| sha-256 | 3f5ca51ff971eac6bdbcb6e4bba3c6866713c342d482e80fb8320fd6c94dbde0 |
| repr-digest | sha-256=:P1ylH/lx6sa9vLbku6PGhmcTw0LUgugPuDIP1slNveA=: ← matches the body shown |
| signature-input | sig1=("@authority";req "@path";req "repr-digest" "content-type");created=1795311497;expires=1795315097;keyid="1hHmE3fo1rg5Ig70wuFtJ5wBc7G-sUTgUJkxyPWFRps";alg="ed25519";nonce="AnbiDwKR-Y5pIx8jnHZc4w";tag="crawlcheck-page" |
| signature | sig1=:VGQOxuqTUaR5IziZV0x37l27Ii7llX7Hc7Kao3MWqMoLkSB0TvYIGCO7HEjd0IyAI3wBGeUYIF1Rs1ZJlAdfCw==: |
| verified | yes — signature by key 1hHmE3fo1rg5Ig70… from https://crawlcheck.io/.well-known/http-message-signatures-directory (directory valid), valid for 3600 more s. Checked here exactly as a crawler would; redo it yourself: https://crawlcheck.io/api/tool/wba |
Guides · 2026-10-02 · By [VSNARY | Emmanuel Orta](/author) · 3 views
# IndexNow: which engines use it and how to check it worked
IndexNow lets a site tell search engines a URL changed instead of waiting to be recrawled. One submission reaches every participating engine. Which engines those are, the key file, the response codes, and what a 200 does and does not mean.
IndexNow is an open protocol for telling search engines that URLs were added, changed or deleted. A site hosts a key in a text file at its root and sends URLs with that key to an IndexNow endpoint; participating engines share submissions with each other, so one request reaches all of them. The registry at indexnow.org lists Bing, Yandex, Seznam, Naver, Yep, the Internet Archive and Amazonbot. Google does not participate. A 200 or 202 response means the submission was received, not that any URL was crawled or indexed.
Check your own site free
Check my site
About a minute. First scan free with your email, nothing installed.
On this page
- How the protocol works
- Who receives a submission
- Reading the response
- When to submit
- Checking that it is working
- Deletions and redirects
- Choosing an endpoint
- On a hosted platform
- What IndexNow does not do
Across every full scan CrawlCheck has run, 7.4% of sites published a sitemap in which no entry carries a modification date at all, leaving crawlers no change signal from the sitemap either.
Search engines learn that a page changed in one of two ways: they recrawl it on their own schedule, or the site tells them. Sitemaps with `lastmod` are the passive version of telling, and only when the dates are accurate, as [our own 103-page lastmod mistake](/blog/sitemap-lastmod-103-pages-one-date-why-engines-ignore-it) showed. [IndexNow](/glossary/indexnow) is the active one: when a [URL](/glossary/url) changes, the site sends it to an endpoint, and the engines that participate can fetch it soon after rather than whenever they next pass by.
## How the protocol works #
Generate a key, a string of 8 to 128 letters, digits and dashes, and publish it as a text file at the root of the host, for example `https://example.com/*key*.txt`, containing the key and nothing else. Then submit changed URLs. A single URL can go as a [GET request](/glossary/get-request) with `url` and `key` parameters; batches of up to 10,000 URLs go as a POST with a JSON body:
```
POST https://api.indexnow.org/indexnow
Content-Type: application/json; charset=utf-8
{"host":"example.com","key":"<key>","keyLocation":"https://example.com/<key>.txt",
"urlList":["https://example.com/guides/new-page","https://example.com/guides"]}
```
The receiving engine fetches the key file to confirm the submitter controls the host, then accepts the URLs. Engines share submissions, so posting to one endpoint is enough; posting the same URLs to every engine separately only adds load.
## Who receives a submission #
The protocol's own registry, `indexnow.org/searchengines.json`, lists the participants. As read for this guide it named Bing, Yandex, Seznam, Naver, Yep, the Internet Archive and Amazonbot. Google is not a participant, so for Google the sitemap and accurate `lastmod` values remain the mechanism, covered in [the sitemap audit guide](/guides/xml-sitemap-audit). Bing's participation matters beyond Bing itself, because some AI answer products have drawn on Bing's index.
## Reading the response #
| Status | Meaning | What to do |
| --- | --- | --- |
| 200 OK | URLs received | nothing; this is not a crawl or index confirmation |
| 202 Accepted | received, key validation pending | nothing; the key file is checked later |
| 400 Bad Request | malformed request | fix the JSON or parameters |
| 403 Forbidden | key not valid: file missing, or its content does not match | check the key file URL returns the key as plain text |
| 422 Unprocessable | URLs do not belong to the host, or key and host disagree | submit only URLs on the host named in the body |
| 429 Too Many Requests | submitting too often | batch changes and back off |
The protocol says a 4xx response must be corrected before retrying, while a 2xx or 5xx lets the notifier move on. Treat 200 as "delivered", never as "indexed". Whether a submitted URL is crawled, and whether it is indexed, is still each engine's decision.
## When to submit #
Submit when a URL is created, when its content meaningfully changes, and when it is deleted or redirected, so the engine can drop it. Do not resubmit unchanged URLs on a timer; engines can deprioritise hosts that submit noise. The practical approach is to record what each page looked like when you last submitted it and submit only the pages that changed since, plus the index page that lists them. A new page is then submitted once, and an unchanged page never again.
## Checking that it is working #
Three checks cover most failures. Fetch the key file from outside your network and confirm it returns 200, `text/plain`, and exactly the key, with no HTML wrapper; a CMS that routes unknown paths to a 404 page answered with status 200 breaks validation silently. Log every submission's status code and URL count rather than firing and forgetting. And in Bing Webmaster Tools, the IndexNow report shows the URLs Bing received from your host, which is the only first-party confirmation available. Many plugins and CDNs offer IndexNow as a switch; if two of them are enabled, the same URLs are submitted twice with two different keys.
## Deletions and redirects #
Submitting removed URLs is as useful as submitting new ones. When a page is deleted, make it return 404 or 410 first, then submit the URL; the engine fetches it, sees the status and drops it, instead of keeping the page in its index until its next scheduled crawl. When a page moves, put the redirect in place, then submit both the old and the new URL. Submitting a URL that still returns 200 with the old content achieves nothing, because the engine fetches what you told it about and sees no change.
The same applies to updates. Submit after the change is live and cached copies are purged, not when the edit is saved in the CMS. A submission that arrives before the [CDN](/glossary/cdn) serves the new version invites the engine to fetch the old one, and the next submission for the same URL may be treated as noise.
## Choosing an endpoint #
Any participating engine's endpoint, or the shared `api.indexnow.org`, delivers to all participants, so pick one and stay with it. Submitting the same batch to several endpoints multiplies requests for no benefit and makes rate limits more likely. Keep one key per host, and if a plugin already submits for you, do not add a second integration with its own key.
## On a hosted platform #
Whether you can use IndexNow at all depends on where the site is hosted, because the key file has to be served from your own domain. What each common platform does:
| Platform | What happens | What you do |
| --- | --- | --- |
| WordPress | Microsoft Bing's IndexNow plugin hosts the key file and submits on publish, update and delete. | Install the plugin; nothing else |
| Wix | Built in since September 2023. Wix submits changed pages itself. | Nothing; it is on |
| Shopify | Built in since 2025. Shopify submits store URLs itself. | Nothing; it is on |
| Cloudflare (any CMS) | Crawler Hints, one switch in the Cache settings, submits for you. It only works when traffic passes through Cloudflare's proxy; a DNS-only record is never seen. | Turn on Crawler Hints, keep the record proxied |
| Ghost | No built-in support. The open request is on Ghost's own forum. | Put the site behind Cloudflare with Crawler Hints, or call the API from a publish webhook (the key file must still answer at the root) |
| Substack, Medium | No way to add it: you cannot host the key file or run code on publish. | Nothing to set up; the platform decides how it tells engines |
Whichever route applies, the check is the same: Bing Webmaster Tools lists the URLs it received through IndexNow, so a platform that says it submits can be confirmed from there.
## What IndexNow does not do #
It does not make a page indexable that is blocked by [robots.txt](/glossary/robots-txt), marked `noindex`, or canonicalised elsewhere; the engine will fetch it and reach the same conclusion it would have reached on its own. It does not influence ranking. And it says nothing about AI crawlers that are not on the participant list. Its value is shortening the delay between publishing and being fetched, for the engines that take part. The status-code side of what a crawler then receives is in [what a crawler actually gets](/guides/http-status-codes-what-a-crawler-actually-gets).
Read live from the same counters the dataset page uses, at the moment this page was served. Bars are scaled to the largest value shown, not to 100%.
Written by
[Emmanuel Orta](/author) · VSNARY
I build CrawlCheck and write everything published here, from measurements the verification layer took. Where a post got something wrong, the correction stays in the post.
[Every post, and where I got it wrong →](/author) · [Follow on LinkedIn →](https://www.linkedin.com/in/emmanuelorta)
Share this finding: [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcrawlcheck.io%2Fguides%2Findexnow-what-it-does-who-reads-it-and-how-to-check-it-worked%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial%26utm_campaign%3Dshare)[X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fcrawlcheck.io%2Fguides%2Findexnow-what-it-does-who-reads-it-and-how-to-check-it-worked%3Futm_source%3Dx%26utm_medium%3Dsocial%26utm_campaign%3Dshare&text=IndexNow%3A%20which%20engines%20use%20it%20and%20how%20to%20check%20it%20worked)[Email](mailto:?subject=IndexNow%3A%20which%20engines%20use%20it%20and%20how%20to%20check%20it%20worked&body=https%3A%2F%2Fcrawlcheck.io%2Fguides%2Findexnow-what-it-does-who-reads-it-and-how-to-check-it-worked)[Follow Emmanuel on LinkedIn](https://www.linkedin.com/in/emmanuelorta)
## Related findings
[2026-10-18 How to check if your website supports HTTP/3 A site advertises HTTP/3 in its Alt-Svc header and its DNS HTTPS record. How to check both, what google.com and others show, and w](/guides/http-3-how-to-check-whether-your-site-offers-it)[2026-09-23 robots.txt status codes: what a 404, 403, 429 or 503 tells Google and AI crawlers What a 404, 403, 429 or 503 on robots.txt means to Google and other crawlers under RFC 9309: when crawling is allowed, paused or b](/guides/robots-txt-status-codes-what-404-403-and-503-tell-a-crawler)[2026-10-14 Sitemap lastmod: 103 pages, one date, why engines skip it 103 pages in our sitemap shared one lastmod though most had been rebuilt. Why Google and Bing ignore dates like that, and how to d](/blog/sitemap-lastmod-103-pages-one-date-why-engines-ignore-it)
The main product
Found this on your own site? **We fix it for $100.**
Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.
[Fix it all — $100](https://buy.stripe.com/4gM00j7OjawhdLEdyR2VG0v)
[What it covers](/pricing#fix) · [scan first, it is free](/)
Free browser extension
Check this on any site you open, **without leaving the page.**
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.
[Add to Chrome — free](https://chromewebstore.google.com/detail/plgjengobhbdmbgckiiaaflgiepdbcpi?utm_source=crawlcheck.io&utm_medium=post)
[What it does](/extension)
Write for the library
Measured something yourself? **Submit an article.**
Send your finding, the data and the method. A person reads every submission and replies by email; accepted articles are published under your name with a link to your study. Free to submit. Articles are accepted on merit; no one can pay to have one published.
[Submit an article](/submit-finding#send)
[How it works](/submit-finding) · a reply within about a week
Spotted a mistake or a bug on this page? [Report an error](/report-error?k=article). Confirmed errors are fixed and logged on the [corrections page](/corrections).
## Comments
Comments are read before they appear. Nothing is published automatically, and no account is needed. An AI agent with a published signing key can post without the queue: it signs the request and the signature is checked against its own domain ([how](/api/comment/signed)). Those appear at once, marked *verified*.
Send for review
Writing about this? [Facts, live figures and marks](/press) — every number on that page is dated and traceable to a scan.
[All guides](/guides) · [The dataset](/data) · [How the dataset works](/policy)
We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io)
#### How we measure, and our own machine files
CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io).
**Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing.
**Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours.
**Registry:** [MCP: Operator](/registry/mcp/operator) · [MCP](/registry/mcp) · [AI policy](/registry/aipolicy) | **Pages:** [IP ranges](/data/ip-ranges) · [Conduct: Googlebot](/agents/conduct/googlebot) | **Guides:** [How to check an MCP server before an agent connects](/guides/how-to-check-an-mcp-server-before-an-agent-connects) | **Glossary:** [entity map](/glossary/entity-map) · [Parent-child retrieval](/glossary/parent-child-retrieval) · [GET request](/glossary/get-request) · [Inbound link](/glossary/inbound-link) · [user-triggered fetch exemption](/glossary/user-triggered-fetch-exemption) · [RIS citation format](/glossary/ris-format)
[Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)
Same body as a file: /guides/indexnow-what-it-does-who-reads-it-and-how-to-check-it-worked.md · as data: /feed.json · what we tell agents site-wide: /llms.txt, /agents.md. To see any other site this way, from your own browser, after its JavaScript runs: the browser extension.