Findings · 2026-10-14 · By VSNARY | Emmanuel Orta · 0 views

Our llms.txt failed our own prompt-injection check

Our scanner reads llms.txt and agents.md for wording that tells an AI model what to hide, whom to obey or what to recommend. On 14 October it flagged one sentence in our own llms.txt. The sentence was harmless, and the rule was too broad. We fixed both.

An llms.txt file and an agents.md file are written to be read by AI models, not by people. That makes them the obvious place to put an instruction a model might follow, which is what indirect prompt injection means: text on a site that a model reads as a command. CrawlCheck's scan reads those files, plus hidden page text, HTML comments and very long image descriptions, and flags wording addressed to a model. On 14 October 2026 it flagged us.

What the check looks for #

Four kinds of wording, the same four we described when we found agents.md files that their owners did not write:

A match is a prompt to go and read the line, not proof of an attack. Most matches are old habits or careless wording. The point of the check is that a person who owns the site may never have read the file, and a model reads every word of it.

Where the check reads, and why hidden text counts #

The same patterns run over every surface a model reads and a person usually does not: llms.txt, agents.md, page elements hidden from view, HTML comments and unusually long image descriptions. A visitor never sees any of these, so nobody notices when they say something odd. A model fetching the page reads them as plainly as the visible text. That gap between what the owner sees and what the model reads is the whole reason the check exists, and it applies to our own files exactly as it applies to anyone else's.

The sentence that tripped it #

Our llms.txt explains what a CrawlCheck cite decision covers: whether a page may be read and attributed, under what rights, from whom, and when it was observed. It ended with this sentence:

"It never says the content is true."

That is a description of what the decision does not do. It is not addressed to a model, and no model would obey it. But the suppression rule was written to catch never say, and as written it also matched never says, because it did not require the word to end there. The instruction section of our own report scored 50, and the site's grade fell from A/99 to A/97.

Two fixes, in this order #

First, the sentence. We rewrote it as "It does not vouch that the content is true." Same meaning, no ambiguity for any reader, human or model. The self-scan returned to A/99 with no findings. That fix shipped at 12:53 PM Mountain Time on 14 October.

Then, the rule. Changing our own text made our grade right but left every other site with the same false positive. A rule that cannot tell an instruction from a description is wrong, whoever it fires on. The suppression and steering patterns now match only the command form of the verb, with nothing attached to its end:

Because it changes what is scored, the change is a new scoring revision, 32. A finding that disappears from a site's report for this reason is labelled as a rule change, not as something the owner fixed.

Why we kept the rule at all #

Narrowing a rule is easy to overdo. The imperative forms are exactly what real suppression looks like: when we read every tool description in the official MCP Registry, four servers told the model to keep something from the user. A check that ignored "do not mention" or "never reveal" to avoid embarrassing anyone would miss those. So the rule still fires on the commands and no longer fires on descriptions of what something never does.

What this does not change #

• The check still reports where each match was found and quotes the line, so an owner can read it rather than trust a count.
• A sentence written as a command is still flagged even when its intent is innocent, such as a note to human editors written as "never say". Rewrite it, or dispute the finding with the line quoted.
• Override and role patterns are unchanged. They had no comparable false positives in our own files.

Check your own files #

1. Fetch them as a crawler would: curl -s https://example.com/llms.txt and curl -s https://example.com/agents.md.
2. Read every sentence as if it were addressed to an assistant with access to a customer's account. If a line tells that assistant what to hide, whom to obey or what to recommend, decide whether you meant it.
3. Rewrite descriptive sentences that read like commands. Models read both, and clear wording costs nothing.
4. Run the free scan, which reads both files and any hidden text on the page.

Questions this post answers

Can an llms.txt file contain a prompt injection?

Yes. llms.txt and agents.md are written for AI models to read, so any sentence in them addressed to a model, such as telling it to hide something or recommend something, is an instruction the model may follow. CrawlCheck's scan reads both files and flags that wording.

What wording does CrawlCheck's instruction check flag?

Four kinds of wording addressed to a model: override ("ignore previous instructions"), role ("you are an assistant"), suppression ("do not mention", "never reveal") and steering ("always recommend"). Since scoring revision 32, only the command form of the verb is matched: "never say", not "never says".

Why did CrawlCheck flag its own llms.txt?

The file said a cite decision "never says the content is true". The suppression rule matched "never say" inside "never says". The sentence was reworded and the rule was narrowed to imperative forms on 14 October 2026.

Is a flagged sentence in llms.txt proof of an attack?

No. A match is a prompt to read the line. Most are careless wording or old habits. The check exists because site owners often never read these files, while models read every word.

What should I do if CrawlCheck flags a harmless sentence?

Rewrite it so it cannot read as a command, which helps models too, or dispute the finding on the disputes page with the line quoted.

Related findings

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works