Findings · 2026-08-20 · By VSNARY | Emmanuel Orta
AI visibility scores measure an API, not what anyone sees in ChatGPT
The answers these tools record are real. The label on top of them — “your visibility in AI search” — describes something that was never measured.
An AI-visibility tool asks the Gemini API who is the best tree service in Denver, records what comes back, counts how often your brand appears, and reports a percentage. Every step of that is real. The model genuinely produced those words. Nothing is fabricated.
What is false is the sentence wrapped around the number: here is your visibility in AI search. The measurement is real; the claim about what it represents is not. Three substitutions sit between the two.
1. The surface: an API is not the app
The OpenAI API is not ChatGPT. ChatGPT performs retrieval, calls tools, and carries personalisation and memory that the raw API does not reproduce. Google's AI Overviews is a different system again from the Gemini API — different retrieval, different ranking, different presentation, sitting inside a search results page rather than a chat window.
Measuring one and reporting the other is the substitution nobody names out loud. It is not fraud; there is no public API for the consumer surfaces, so every vendor in this category measures the thing they can reach and labels it as the thing you asked about.
2. The sample: the prompt list is somebody's guess
Even a perfect measurement only covers the prompts somebody chose to type. Nobody has the distribution of what buyers actually ask an assistant — not the vendors, not the model providers, not us. A share-of-voice percentage is a statistic over a hand-picked prompt list, presented as a property of your brand.
Change the list and the number changes. That is not a flaw in the arithmetic. It is what the arithmetic was always describing.
3. The moment: one answer is one draw
Ask the same model the same question twice and you can get different answers. Ask it next week, after a model update, and the difference is larger. A single run tells you almost nothing on its own. The churn between runs is the signal — how stable your presence is across dated repetitions — and it is the part most tools do not report, because it makes the headline number look unsteady.
The test that separates a measurement from a label
Ask what would have to be true for the number to be wrong, and how you would find out. If there is no answer, it is not a measurement. A percentage you cannot check is a claim about a thing nobody measured.
What survives that test is narrow and unglamorous: name the model, name the mode (grounded or from weights), name the date, report presence or absence rather than a rank, and publish the churn. Then a reader can disagree with you on evidence.
Where this leaves us, including the part that is ours
Our own answer ledger is not exempt. It queries an API too. When it produces figures, they will carry the model name, the mode and the date on every row, and they will describe what that model said on that day — not what a person saw. Today it produces nothing at all, because the pipeline is not running, and we would rather say that than publish a number we cannot defend.
The half of this problem that is directly observable is the audit. What your site serves to a crawler is not a proxy or a sample: it is the bytes, fetched under fifteen client identities, verified against the ranges each operator publishes. The causes are measurable. Only the effects are estimated — and a tool that tells you which is which is worth more than one that reports a confident percentage for both.
Every figure above came out of this scanner.
Point it at your own domain and see the same measurements, free.
Questions this post answers
Do AI visibility tools measure what people see in ChatGPT?
No. They query provider APIs, and an API is not the consumer app. ChatGPT adds retrieval, tools and personalisation the raw API does not reproduce, and AI Overviews is a different system again.
Are AI visibility scores fake?
The recorded answers are real model output. What is unsupported is the label placed on them, because the surface, the prompt sample and the moment all differ from what the number claims to describe.
What makes an AI visibility number checkable?
Naming the model, the mode, and the date, reporting presence or absence rather than a rank, and publishing the churn between dated runs so a reader can see how stable the result is.