Findings · 2026-09-01 · By VSNARY | Emmanuel Orta · 0 views
The sameAs profile we called dead: a crawler that cannot fetch its own host
A customer's schema pointed at a page on our own domain. From outside it answered 200. From inside CrawlCheck it answered nothing, and we reported it dead.
One of the sites we operate declared its founder as a Person node and listed two profiles for corroboration: a personal domain, and the author page on this site. The corroboration check fetched both. The personal domain came back corroborated: it links the business back. The author page on crawlcheck.io came back dead: no response.
From a laptop the author page answers 200 in under a second. From CrawlCheck it cannot answer at all, because CrawlCheck cannot fetch pages on its own host. It is the same limit that stops CrawlCheck grading crawlcheck.io from inside, which we have published before. What we had not done was carry that limit into the corroboration check, which fetches whatever URL a customer's schema names.
Dead is an accusation. Unverifiable is a fact about us. #
The three outcomes that check can return are deliberately not interchangeable. Corroborated means the target links back. Unreciprocated means the target is reachable and does not. Unverifiable means we could not read the target, so we know nothing, and the report must not pretend otherwise. Dead means the target did not answer, which reads to a customer as: your schema points at a page that does not exist.
For a target on our own host, dead was false and unverifiable was true. Any customer whose schema mentions crawlcheck.io, which is exactly the kind of customer we want, would have been told their profile was broken. We corrected it: a target on CrawlCheck's own host is now reported as unverifiable, with the reason stated. The citation check, which fetches directory listings the same way, got the same correction.
Why this keeps happening, and the rule it reinforces #
Every false accusation CrawlCheck has shipped has had the same origin: a fetch failed for a reason on our side, and the code read the failure as a fact about the site. A truncated profile body became nothing points back. A challenge page became reachable and empty. A request CrawlCheck could never make became dead. The general rule is the one we keep writing down: separate the site refusing us from our own fetch failing, and never let the second one score.
If you run any tool that fetches links on other people's behalf, the question to ask of every failure path is whether the failure could be yours. Timeouts, egress blocks, DNS on your resolver, credentials you never had, and routes you serve yourself are all yours. None of them is evidence about the site.
Four outcomes, and the one that was wrong #
| Outcome | What it means | True of this target |
|---|---|---|
| Corroborated | the target links back | no |
| Unreciprocated | the target is reachable and does not link back | no |
| Unverifiable | we could not read the target, so we know nothing | yes |
| Dead | the target did not answer | reported, and false |
Dead is an accusation about the customer. Unverifiable is a fact about us. They are deliberately not interchangeable.
How to check a corroboration claim by hand #
Open the sameAs target from a client that is not the tool making the claim, and search the page source for your domain:
curl -sL https://profile.example/you | grep -c 'yoursite.com'A count above zero is a reciprocated link. Zero on a page you can read is unreciprocated. A page you cannot read from that client is unverifiable, and the honest report says so. Reciprocated sameAs and unverifiable are defined in the glossary; the entity SEO guide explains why the distinction is worth the bother.
Every figure above came out of CrawlCheck.
Point it at your own domain and see the same measurements, free.
The main product
Found this on your own site? We fix it for $749.
Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.
Questions this post answers
Why can CrawlCheck not check a page on crawlcheck.io?
It cannot fetch pages on its own host, so any URL on crawlcheck.io is unreachable from inside a scan. That is a fact about CrawlCheck, not about the site being scanned.
What is the difference between dead and unverifiable?
Dead means the target did not answer, which accuses the site of pointing at a page that does not exist. Unverifiable means CrawlCheck could not read the target for a reason on its own side. Only the second is true for a target on CrawlCheck's own host.
Related findings
Comments
Comments are read before they appear. Nothing is published automatically, and no account is needed.
Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.