Findings · 2026-11-06 · By VSNARY | Emmanuel Orta · 3 views
Most of our 'referring pages' were forged Referer headers
The owner panel that lists who links to a site was reporting its own IP addresses, parked lookalike domains and a radiology college. None of them had linked to anything. How a referrer is forged, and the rule that now counts one.
A Referer header is a claim, not a record. Until 6 November 2026 CrawlCheck counted every one it received as a referring page. Workers Logs showed the top entries were forgeries: crawlcheck.io's own Cloudflare addresses on ports 2082 and 2086, sent by a scanner on Google Cloud probing for control panels; search-engine lookalikes such as bing.io and google.io that do not resolve; and real organisations' hostnames with no page path. A referral now counts only when the request also carries the browser signals a real click produces: Sec-Fetch-Site cross-site, Sec-Fetch-Mode navigate and an Accept-Language header, on a GET.
The owner dashboard has a panel called referring pages. It is meant to answer a simple question: which outside pages are sending people here? On 6 November 2026 we read it against the raw request logs and found that most of what it listed had never linked to us at all.
What was in the panel #
| Entry | What it really was | How we know |
|---|---|---|
| 104.21.64.145:2082 and 172.67.151.148:2086 | crawlcheck.io's own Cloudflare addresses, with cPanel port numbers appended | Sent by two Google Cloud addresses using a Chrome 70 user agent while requesting /aab9, /aaa9 and /getcmd, which are control-panel probe paths |
| bing.io, google.io, yandex.io, corp.google.com | Lookalike search-engine hostnames | None resolve in DNS, or resolve to a parking page. A browser cannot arrive from a page that does not exist |
| storeopinion-ca.today, myfordbenefitsguide.co | Spam domains | Do not resolve |
| aocr.org | The American Osteopathic College of Radiology | 882 'visits' with no path at all. A real click carries the page it came from |
| toolgrep.com/p/crawlcheck, huggingface.co, kagi.com | Real | toolgrep lists CrawlCheck (35 mentions); Hugging Face hosts our datasets; Kagi is a search engine |
Two more entries were our own alternate domains, which redirect to us and are not referrals in any useful sense.
Why the forgeries exist #
A Referer header is text the client chooses to send. Scanners set it for two reasons. Some set it to the target's own address to look like an internal navigation and slip past rules that trust same-site traffic. Some set it to a search engine so that a log reader sees 'traffic from Google' and does not look closer. The cPanel ports are the tell: CrawlCheck was looking for a hosting control panel on ports 2082 and 2086 and wrote that target into the header it sent along.
The radiology college is the stranger case. Its hostname arrived 882 times with no path. We did not find a page there that links to us, and a click from a page carries that page's URL. The likeliest explanation is a referrer-spam tool cycling through a list of real domains. Whatever the cause, nothing on that site sent us a visitor.
The rule that now counts a referral #
A browser following a link from another site sends more than a Referer. It sends Sec-Fetch-Site: cross-site, Sec-Fetch-Mode: navigate, an Accept-Language header and uses GET. Scanners that forge a Referer almost never forge the fetch-metadata headers, because those are set by the browser and most tools do not know they exist.
A referral now counts only when all of those are present. In addition, a Referer that is a bare IP address is dropped, hosts confirmed forged were removed from the stored history, our own alternate domains are classified as our own, and Kagi was added to the list of search engines.
After the change the panel lists le-atelier-noir, nutritionome, toolgrep, a GreenGeeks help-desk article and Hugging Face. The engines list no longer includes google.io or yandex.io.
How to check your own referrer report #
Most analytics tools apply some version of this filter already, which is why a Google Analytics referral list looks cleaner than a raw server log. If you read logs or run your own counter, three checks remove most of the noise: drop any Referer that is an IP address or your own host; drop any whose hostname does not resolve; and if your edge gives you fetch-metadata headers, require cross-site navigate. We had written about how few real arrivals carry a referrer at all, and the forged ones were a large share of the few that did.
It is not the first count of ours inflated by requests that only looked like the thing being counted. Forged crawler user agents did it to our crawler counts, and headless page loads did it to our engagement figures until this week. The pattern is the same each time: a header is a claim, and a claim is counted only once something the sender cannot easily fake agrees with it.
Share this finding:LinkedInXEmailFollow Emmanuel on LinkedIn
Questions this post answers
Can a Referer header be faked?
Yes. It is plain text the client chooses to send. Scanners set it to the target's own address, to a search engine, or to a random real domain so that a log reader trusts the request or misreads where traffic came from.
How do you tell a real referral from a forged one?
A browser following a link sends Sec-Fetch-Site cross-site, Sec-Fetch-Mode navigate and an Accept-Language header on a GET. Most tools that forge a Referer do not set those, because the browser sets them automatically.
Why were CrawlCheck's own IP addresses in its referrer list?
A scanner on Google Cloud probed for a hosting control panel on ports 2082 and 2086 and wrote that target into the Referer it sent. CrawlCheck counted the header as a referring page.
Did the American Osteopathic College of Radiology link to CrawlCheck?
No. Its hostname arrived 882 times with no page path, which a real click always carries. No page on that site links here. It is most likely a referrer-spam tool cycling through real domain names.
What should I do about forged referrers in my own logs?
Drop Referer values that are IP addresses or your own host, drop hostnames that do not resolve, and if your edge exposes fetch-metadata headers, count a referral only when they show a cross-site navigation.
Related findings
The main product
Found this on your own site? We fix it for $100.
Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.
Free browser extension
Check this on any site you open, without leaving the page.
CrawlCheck for Chrome grades every site you visit A–F and shows, from your own browser, what search engines and AI assistants are actually served. Free, no account.
Write for the library
Measured something yourself? Submit an article.
Send your finding, the data and the method. A person reads every submission and replies by email; accepted articles are published under your name with a link to your study. Free to submit, and placement is never for sale.
How it works · a reply within about a week
Spotted a mistake or a bug on this page? Report an error. Confirmed errors are fixed and logged on the corrections page.
Comments
Comments are read before they appear. Nothing is published automatically, and no account is needed. An AI agent with a published signing key can post without the queue: it signs the request and the signature is checked against its own domain (how). Those appear at once, marked verified.
Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.