CrawlCheck Skip to content

Menu

For humans

Check and fix

Register your site

Free. Every agent that asks is told who runs it

Free A–F report, every problem named

Checked twice a day from $29 a month, with an alert when a check finds a regression

We do the fixes for $100 and prove them

Executive report

A two-page PDF per site, monthly or weekly

The scan, step by step, and what each grade means

Free tools

Every check, no account

Your homepage as Chrome and as each AI crawler

robots.txt for AI crawlers

What each crawler is allowed, and what it got

Read the way crawlers ask for it

Is your business record right where agents look?

Compare two sites

Side by side, same measurement

Googlebot check

Is this IP from your logs really Google?

Entity map validator

entitymap.json against your structured data

Fix it on your platform

Browser extension

Grades as you browse

iPhone and Android

Register your site

Prove who runs it.

Registered sites answer every agent that asks: who runs this, what it declares, verified by a fetch. Free.

Datasets

For agents

Build

MCP, curl, Python, OpenAI Agents, LangChain, CrewAI, Vercel AI

Every endpoint, with a worked example

npm and PyPI, with the offline verifier

An approved server allowed, a rewritten tool refused

CrawlCheck in CI

Fail a pull request that breaks crawler access

Preflight rules and tests in your repository

Ask before acting

The signed answer about any domain, for a person

Resolve protocol

What the signed answer holds, how to verify it

Capability registry

What sites say agents may do, and what was verified

How each named agent behaves on sites

MCP tool changes

Servers whose tools changed after approval

Enterprise pilot

30 days of signed preflight for your agents

Verify it yourself

Verify a record

Check any answer on your own machine

The Ed25519 keys every answer is signed with

Every finding the measurement can raise

Canonical numbers

The definition and every public count, with sources and as_of

Run the suite against your own implementation

Proof and timestamps

How every record is sealed and anchored

Scans, signing, seals and a signed incident log

Before you act

One signed answer, no key.

Read, cite, connect or transact: ask first, verify the receipt offline.

Start in a minute

Research

Live data

AI crawler census

The top 1,000 sites, read daily, open CC BY data

AI crawler radar

What the top 10,000 sites say and serve

Agent Traffic Index

Verified and forged agent requests, monthly

Decisions, scans and crawler visits, counted

Agent safety benchmark

Do agents stop before acting?

Every city a scan was started from

Crawlers and vendors

Who sends each agent and what it reads

Crawlers compared

GPTBot vs ClaudeBot and more, two at a time

SEO and AI tools, measured

The vendors that measure sites, measured by the same rules

What registered sites declare, verified by a fetch

Certified sites

Every live machine-readability certificate

Evidence for disputes

A dated, signed record of what a site served

Reports and method

Four recurring reports, each signed with a feed

Exclusive findings

Measured only here

How we read the data

The rules, what we never cite, how to report an error

Six lessons and a certificate

MCP tool changes

How often public MCP servers change the tools agents approved

AI audit tools, Round 2

74 tools against 63 planted faults

AI audit tools, tested

Twelve tools against ten planted faults

Submit a finding

Your study, published with credit

Everything we got wrong, dated

The census

What the top sites serve AI crawlers.

Read daily from our own network, open CC BY data, every figure with its denominator.

Read the census

Library Pricing Sign in

Glossary · Accessibility and provenance

Captions

Synchronized text alternatives for speech and relevant non-speech audio in media.

What Captions does not show

Automatically generated captions require accuracy review; presence alone is not adequacy.

Where Captions comes up on this site

YouTube SEO case study: 81 videos rewritten, measuredThe channel today Run it on your own channel The YouTube check is free for a single video and reads chapters, captions, description and language.

Terms this definition uses

Accuracy · Review

Definitions that name Captions

Layout-aware parsing · Audio description

Related terms in Accessibility and provenance

What a page owes a keyboard, a screen reader and a fact-checker: the accessibility basics and the C2PA vocabulary for proving where a file came from.

Keyboard accessibility · Focus order · Focus indicator · ARIA · Accessible description · Color contrast · Reduced motion · Audio description · Status message · Content Credential · C2PA assertion · C2PA claim · Claim signature · Manifest store · Content binding · Hard binding · Soft binding · Ingredient · Trust list · landmark · contrast ratio

Questions about Captions

What is Captions?

Synchronized text alternatives for speech and relevant non-speech audio in media.

What does Captions not show or guarantee?

Automatically generated captions require accuracy review; presence alone is not adequacy.

Which area of the glossary does Captions belong to?

Accessibility and provenance: What a page owes a keyboard, a screen reader and a fact-checker: the accessibility basics and the C2PA vocabulary for proving where a file came from.

← Reduced motion · Audio description →

See it in the full glossary · 772 terms across 19 areas.

Free browser extension

See this on any site you visit, in one click.

CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.

Add to Chrome — free