Findings · 2026-08-21 · By VSNARY | Emmanuel Orta · 0 views

LLMs ignored our schema and still identified us: what actually got read

We publish a four-node JSON-LD graph. Two unrelated tools that turn pages into text for language models both dropped the whole of <head>, and with it every node. Both still named the company correctly. Then we measured our own ten sites: 87% of our structured data sits where neither of them looks.

We ran our own homepage through a commercial scraping API — the kind that sits between a website and a language model, turning pages into the markdown an assistant is actually handed. It is a good product and it did its job. The output was clean, well-structured markdown at 41% of the token cost of the HTML.

Then we read the HTML it returned alongside that markdown. It began:

<!DOCTYPE html><html lang="en"><body>

No <head>.

Our homepage carries exactly one JSON-LD block. That block carries four nodes: WebSite, SoftwareApplication, Organization and FAQPage. All four live in <head>. None in <body>. We checked rather than assuming:

Everything we publish about what this company is, what the product does and what it costs — the machine-readable version of it — was removed before the content reached a model.

Then we checked a second reader, and it did the same thing #

One pipeline is a vendor quirk. So we sent the same page through a second, unrelated one — a public URL-to-markdown reader widely used as a drop-in for agent fetching. It returned 9,742 bytes of clean markdown. We counted the same strings in its output that exist on our live page:

Our entire JSON-LD block is 2,944 bytes and none of it survived either transformation. Note the 16: that is the raw count of "@type" occurrences, which is larger than the four top-level nodes because the FAQPage nests a Question and an Answer per entry. Four nodes, sixteen type declarations, zero delivered.

Two independent implementations, built by different companies for different customers, made the same call. That is the part worth noticing. It is not a bug in either product — it is what boilerplate stripping means, and <head> is boilerplate by every reasonable definition.

It is still n=2. Two readers is enough to show the behaviour is not one vendor’s quirk. It is not enough to claim a rate across the web, and we have not measured what share of model-facing traffic arrives through an intermediary rather than a first-party crawler. We are not going to pretend otherwise.

Why the distinction has teeth #

A growing share of what reaches a language model does not arrive from that model’s own crawler. It arrives through an intermediary: a scraping API, a reader endpoint, a retrieval ingestion job. Each one applies its own transformation before the model sees anything, and neither your rank tracker nor your schema validator models that step. The validator reads your markup and confirms it is well-formed — which ours is. The markup was never the problem.

Which means a fact that exists only in your JSON-LD is a fact with a delivery condition attached. If your phone number, your opening hours or your service area appear in structured data and nowhere in visible prose, whether a model can state them depends on which pipeline fetched you.

The redundancy rule #

The practical response is not to abandon structured data. It is to stop treating it as the sole home of anything you need answered. Every fact you would be unhappy for a model to get wrong should exist in both places: declared in schema for the readers that parse it, and stated in visible prose for the readers that only ever see text.

That is the same discipline as the name-address-phone check this scanner already runs, arrived at from the opposite direction. That check exists because schema and visible page disagreeing is a defect. This one says the visible page must carry the fact at all.

The part that stops this being a complaint #

The pipeline still got us right.

Its structured extraction returned the company name correctly and a description that accurately summarised the product. It did that from the prose and from the og: and twitter: meta tags, which it captured separately into a metadata object rather than as part of the document.

So the schema was not needed here. It was simply not consulted.

That distinction matters more than either of the loud versions of this story. “Structured data is dead” is wrong: search engines read it, answer engines read it, and rich results depend on it. “This tool is broken” is also wrong: it produced an accurate summary and a clean document, which is what it exists to do. The accurate version is narrower. Publishing schema and having a model see your schema are two different claims, and the second depends on which pipeline is standing in between.

So we measured our own estate #

One page is an anecdote. We ran the same check across ten sites we operate — a SaaS product, an e-commerce store and eight local service businesses — asking one question: if a pipeline strips <head>, how much of our structured data disappears?

Thirty-nine of forty-five nodes — 87% — sit in <head>.

The exception is the interesting row. One tree service site carries all four of its nodes in <body>, because of how its templating happens to emit them. Nobody decided that. It was not a strategy. That site’s LocalBusiness, FAQPage and BreadcrumbList nodes survive a head-strip for no reason other than an accident of where a plugin writes its output.

Our own flagship — the site for the product that measures this category — is in the vulnerable group. All four nodes, all in <head>, all discarded.

Why head is where everyone puts it #

Because that is what the documentation shows, and because for twenty years the consumer of structured data was a search crawler that parsed the whole document. Google’s guidance places JSON-LD in <head> in most examples. Every CMS plugin follows. To a conforming parser it genuinely does not matter — JSON-LD is valid in either position and a full-document parser reads both identically.

That assumption held while the reader was a browser or a search crawler. It is less safe when the reader is a text-extraction layer whose job is to throw away everything that is not content — and <head> is, by definition, not content. Boilerplate stripping is the correct default for a summarisation pipeline. The schema is collateral.

What we are not going to tell you to do #

We are not going to tell you to move your schema into <body>. We considered it, and the reasons not to are stronger:

• It is valid, but unusual, and unusual markup is how you acquire a different problem later.
• The pipelines that matter most for search still read the whole document. You would be optimising for one consumer at the cost of convention.
• We measured one pipeline. We have not established that head-stripping is common, and generalising from a sample of one is the exact error we have written about before — the one where a 57% block rate turned out to be a single machine wearing seven crawler names.

What we will say is narrower and defensible. If your identity exists only in your schema, it is fragile. Whatever the pipeline, the prose is what survives. If your homepage does not state in plain sentences what you do, where you are and what it costs, you are relying entirely on the layer a text extractor is most likely to drop.

The og: and twitter: tags earn their keep. In this case they were the only structured signal that survived, because the extractor collected them deliberately rather than as part of the document body.

And test the extraction, not the markup. A schema validator confirms your JSON-LD is valid. It cannot tell you whether the thing on the other end ever received it. That is the same failure shape as a robots.txt returning HTTP 200 with a challenge page inside it: the artefact is fine, the validator is happy, and the machine at the far end got something else.

The thing we shipped that this pipeline did not use #

One more measurement, because it cuts against our own work. This site serves native markdown to anything that asks for it. Send Accept: text/markdown and you get markdown; send anything else and you get HTML. Vary: Accept is set so a cache cannot mix them up.

The pipeline did not ask. It sent Accept: text/html, took the HTML, and converted it on its own side — arriving at 41% of the token count, within a point of the 40% our endpoint would have handed it directly.

That is not a failure on their part. Content negotiation for markdown is a new convention and almost nothing implements it yet. But it is worth writing down that we built the thing, it works, it produces the same result the client computed for itself, and the client had no reason to know it existed.

Both observations point the same way. The machine layer is not one thing you either have or lack. It is a set of separate channels — schema, meta tags, prose, negotiated representations — and each consumer reads a different subset. The only way to know which subset reached a given reader is to look at what that reader actually received. Which is the premise of this project, and we just found a case where our own site failed it.

Method #

The pipeline output was read from its own playground, including the raw HTML it returns alongside the markdown. Its response reported a cache hit with a timestamp several hours old, so we re-checked the live page independently and relied on the cached copy only for the head-stripping behaviour, which is a property of the extractor rather than of the page.

Schema node counts were taken by fetching each homepage with an ordinary browser user-agent, splitting the document at </head>, extracting every application/ld+json block, parsing it, and counting nodes — expanding @graph where present, so a single block containing four nodes counts as four rather than one. Byte counts for the two representations came from identical requests differing only in the Accept header. Counts are homepages only; interior pages carry different graphs and are not measured here.

Questions this post answers

If LLMs ignore JSON-LD, why publish it?

Because the pipelines that discarded it still identified the entity from the visible text, the title, and the consistent naming across pages, which is the same identity the schema declares. Schema is the machine-checkable version of facts that must also be on the page; when the two agree, every reader gets the same answer.

Where does entity identity live on a page?

In the repeated, consistent statements: the name in the title and heading, the same phone and address everywhere, the same profile links, and structured data that says the same thing. A resolver triangulates; schema is one of the sources.

Related findings

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works