Findings · 2026-08-19 · By VSNARY | Emmanuel Orta
When a scanner grades a firewall instead of your website
We published a failing grade for a competitor. The grade was wrong, and the way it was wrong is the most useful thing this scanner has taught us.
For several hours, this site published a report grading a competitor 15 of 100 - an F, with findings like 'no sitemap' and 'robots.txt not served'. All of it was false. From an ordinary residential connection, their robots.txt is a normal 251-byte file that declares a sitemap. What our scanner had measured was their hosting firewall challenging our datacentre address on every path, at HTTP 200, in identical ~200-byte responses.We re-ran the scan and got the same 15. That felt like confirmation. It was not: a deterministic block reproduces perfectly, so two identical readings from one vantage point are one reading. Nothing about running it twice tested the thing that was actually wrong.The tell was hiding in plain sight: the site answered a URL that cannot exist exactly the way it answered its homepage. No real website does that. A site that returns the same bytes for /pricing and /this-path-was-generated-at-random is not describing itself - something in front of it is speaking on its behalf.That check now runs on every scan. A randomised, nonexistent path is fetched alongside the homepage; if they match in the way challenge walls match, the scan stops. The report says the site refused measurement - level 'refused', no grade, no score - because a refusal is not a bad site. It is a fact about our vantage point, and publishing it as their defect would be exactly the error we made.We deleted the false report and the corpus record behind it. The corrected behaviour is public: a refused site carries no number at all, and the finding names our limit, not their failure. If your security layer challenges datacentre traffic, our scanner will tell you that it was refused - and it will not pretend the refusal was your website.The lesson generalises past this scanner. Any tool that fetches your site from cloud infrastructure can be fed a challenge page and grade it as content - most will. Ask your auditor one question: how do you know what you measured was my website?Live, as you read this: the corpus now holds 98 domains across 773 scans. The figures in this piece were measured on the date above; this line is not.
Every figure above came out of this scanner.
Point it at your own domain and see the same measurements, free.
Questions this post answers
Can a scanner grade a firewall instead of a website?
Yes. If the origin refuses the scanner's address, every check fails for one reason and the resulting grade describes the refusal, not the site.
How should a scanner handle being refused?
By reporting the refusal as a refusal. A grade computed from blocked requests is a measurement of access, and publishing it as a quality score is an error.
What did this change here?
Refused scans are now identified and are not published as failing grades.