CrawlCheck
Googlebot simulator · free, no signup

Googlebot simulator: see your page the way Googlebot gets it

In one sentence

A Googlebot simulator requests a page with Googlebot's user-agent string and shows what the server sent back, so it can be compared with what a browser gets. It shows how a site treats a request that says it is Googlebot; only Google's own URL Inspection tool fetches from Google itself.

Type a domain. The free check requests your homepage as Googlebot and as Googlebot Smartphone, the copy Google indexes for most sites, and sets each response beside what a normal browser receives: status, redirects, size and words. It reads robots.txt the way Googlebot does and shows which words are in the HTML and which appear only after JavaScript runs.

First scan free with your email, then 3 credits ($3) a scan. Nothing installed, nothing changed on your site. The same scan as the homepage - about 30 read-only requests - and the same report, opened at the sections that answer this question.

How often a crawler gets a different page
A crawler served less than a browserCRAWLER_SERVED_LESS1.1%A challenge page served as 200CHALLENGE_SERVED_2000.3%The words need JavaScriptCONTENT_NEEDS_JAVASCRIPT2.1%robots.txt does not answer 200ROBOTS_NOT_2004.7%Share of 9,160 scans in the public dataset that carried the finding. Live; the same figures as /data.

Is your site one of them? Check it free → All 82 rates →

Measured

What the scan measures for this

Googlebot and Googlebot Smartphone

The status, redirects and bytes each Googlebot identity was served, beside a browser's copy. A challenge page, an error or a shorter page for either one is named.

See this section on a real report: Crawler permissions →

robots.txt for Googlebot

Which group in robots.txt applies to Googlebot, whether the page is allowed, whether robots.txt itself answers 200, and whether a sitemap is declared.

See this section on a real report: Sitemap →

Words before JavaScript

How many words are in the HTML Googlebot first receives and how many appear only once scripts run. Google renders scripts in a later step; most AI crawlers never run them.

See this section on a real report: Page weight & text →

Indexable

Status, canonical and any noindex in the page or its headers: whether the copy Googlebot received can be indexed at all.

See this section on a real report: Indexability →

For agents

What an agent is told about the same site

An agent does not read the report. It asks one endpoint before it reads, cites, connects or transacts, and gets a signed answer built from the same measurement.

1 · See it

The agent view

The answer an agent receives about any domain, rendered for a person: each action’s decision and the reason.

Agent view

2 · Call it

One endpoint

GET /api/v1/resolve?domain= returns the signed answer; preflight returns a decision and a receipt for one action.

QuickstartResolve spec

3 · Check it

Without trusting us

Every answer is Ed25519-signed and logged; it verifies offline with the published key.

Signing keysProof

Workflow

The workflow

  1. Run the free checkNo account and nothing installed. About twenty seconds, read-only.
  2. Compare the Googlebot rows with the browser rowA different status, a redirect only Googlebot follows, or far fewer words is the first thing to fix.
  3. Confirm a refusal with GoogleThe check sends Googlebot's user agent from CrawlCheck's address. A firewall that verifies Googlebot by IP can answer the real crawler differently, so confirm a refusal with URL Inspection in Search Console.
  4. Re-check after the fixThe second scan is the dated evidence that the change reached the crawler.
Free

Free tools for this

Each one is a single request to the domain you name, answered as text or a download. Put the domain on the end of the URL, or use the form on the tools page.

Reading

Read next

Start from what you are checking

You cannot pay to improve a score. A licence adds the lists behind the counts and the record over time - never a better grade.

Questions about this page

QWhat is a Googlebot simulator?
A tool that requests a page with Googlebot's user-agent string and shows what the server sent back. It answers how your site treats a request that says it is Googlebot, which is where challenge pages, cloaking and mobile differences show up.
QDoes it show exactly what Google sees?
It shows what your server sends to a request carrying Googlebot's name from an outside address. A firewall or bot rule that checks Google's IP ranges can answer the real Googlebot differently. When the result shows a refusal, confirm it with URL Inspection in Google Search Console, which fetches from Google.
QWhy check Googlebot Smartphone separately?
Google indexes most sites from the copy Googlebot Smartphone fetches. A site can serve the desktop crawler a full page and the smartphone crawler a redirect, an interstitial or less text, and the mobile copy is the one that counts.
QDoes Googlebot run JavaScript?
Yes, but in a later step than the first fetch. Words already in the HTML are read at once. Most AI crawlers do not run JavaScript at all, so text that needs it can be missing from AI answers even when Google indexes it.
QIs the Googlebot simulator free?
Yes. The check, the grade and the fix list are free with no signup.