Findings · 2026-09-01 · By VSNARY | Emmanuel Orta
llms.txt returned 200 and was a login wall: why status codes lie about machine files
The same page that answered a nonexistent path answered /llms.txt. One layer of our scanner noticed; the other called it present.
A large community site answers /llms.txt with HTTP 200. The body is 8,401 bytes of HTML: a script that submits a form on load, a favicon link, a page title, and nothing a language model could use. It is the same shell the site returns for a path that cannot exist. It is a login wall wearing a status code.
Two parts of our scanner looked at that response on the same scan. The host-file layer, which asks whether a machine file is live, read the content type, saw text/html, and said no: an llms.txt is text by definition, and this was not text. The older checks layer, which records what each path answered, saw the 200 and marked the file ok. Two instruments, one response, two verdicts.
The contradiction is what got reported, not the file
We did not find this by reading the site. We found it because the scanner audits its own output. One of its invariants, FETCHER_DISAGREEMENT, fires when two of our own fields describe the same path differently, and it fired: checks says live (HTTP 200, 8,401 B) while the host-file layer says not live. A violation there is never a finding about the site. It is a bug in the scanner, and it means one of the two layers is lying.
The checks layer was the liar. It had learned the soft-404 rule for sitemaps weeks earlier: when the bait path answers 200 with HTML and a sitemap path answers with the same HTML, the sitemap is absent, not broken. The rule had never been extended to llms.txt. So a sitemap at that address was correctly called absent while the llms.txt beside it, identical bytes, was called present.
What a 200 is allowed to mean
A status code is a claim by the server about the request, not about the content. For a machine file the content is the whole point, so the test has to be on the body. The rule that now runs: an llms.txt that answers 200 with HTML is absent. Whether it is a catch-all page, a challenge, or a login form does not matter; none of them is the file. The checks layer marks it absent, the NO_LLMS_TXT finding fires, and both layers agree again.
This is the same shape as the robots.txt that answered 200 with a challenge page, and the same lesson: an absence and a refusal look identical unless you check the content type. What was new here is that the disagreement between two of our own readings was the alarm, which is exactly what the self-audit exists to do.
How to check your own
Fetch the file and look at the content type, then at the first line of the body:
curl -s -D - https://yoursite.com/llms.txt -o /tmp/llms.txt | grep -i content-type
head -c 200 /tmp/llms.txtIt should say text/plain or text/markdown, and the first characters should be a heading or a sentence, not <!DOCTYPE html>. If your platform answers every unknown path with your homepage, it answers /llms.txt that way too, and every checker that trusts the status code will tell you the file exists. The guide to writing and verifying an llms.txt covers the rest of the shape.
Every figure above came out of this scanner.
Point it at your own domain and see the same measurements, free.
Questions this post answers
Can an llms.txt be served as HTML?
No. The convention is plain text or markdown. A 200 that carries HTML at /llms.txt is the site's catch-all page, a challenge, or a login form, and none of those is the file. Check the content type, not the status.
Why did two parts of the scanner disagree?
The host-file layer tests content type and treats HTML as not live; the checks layer had learned that rule for sitemaps but not for llms.txt. The self-audit flags any two of the scanner's own fields that describe one path differently, which is how the gap was found.