CrawlCheck

Findings · 2026-09-23 · By · 0 views

Internal link audit: 2,260 links checked, 4 went nowhere

Every sitemap page answered 200. One link on the page that measures us pointed at a profile that has never existed, and three in-page links on non-local reports pointed at sections that were never drawn.

All 924 pages in crawlcheck.io's sitemap returned 200. Across 2,260 distinct internal link targets, six answered 400, all of them API endpoints requested without their required parameter, which is correct. One link was broken: the instrument self-check page linked to a directory profile for CrawlCheck that does not exist, because the site is deliberately kept out of its own ranking. A second pass on a report found three in-page anchors pointing at sections not drawn for that kind of site. All four were fixed the same evening.

A site that audits other sites should be able to show its own internal links work. We had not checked all of them at once, so on 23 September we did: every page the sitemap declares, every link on every one of those pages, and then the in-page links on a report.

The method #

The inventory came from the sitemap, 924 URLs. Each was fetched and its status recorded. Every href on every fetched page that pointed inside the site was collected and de-duplicated, which gave 2,260 distinct targets. Targets already in the sitemap were known; the remaining 1,339 were requested one by one. Anything that did not answer with a 2xx or a redirect was listed.

That is deliberately simple. It follows the same order as the internal link audit guide published alongside this post, and it can be run by anyone with a sitemap and a list of URLs.

CheckCountResult
Sitemap pages fetched924924 answered 200
Distinct internal link targets2,260collected from those 924 pages
Targets outside the sitemap, requested1,3397 non-2xx
Of those, API endpoints without parameters6400, correct
Real broken page link1404, fixed
In-page anchors on one report383 with no target, fixed

Six 400s that were not broken #

Six links answered 400: the comparison tool's endpoint, two fix generators, and three file tools. Each is an API endpoint that needs a domain or a pair of domains in the request, and each is linked from documentation or a tools page as an address, not as something to click without a parameter. A 400 there says the request was incomplete, which it was. Counting them as broken would be the error the rest of this site warns against: judging a status code without asking what the request was.

The page /directory/self-check measures crawlcheck.io with the same scanner and score version as every vendor in the directory. It is kept out of the ranking on purpose, so the benchmark never contains its author. It draws its one row with the same table code the directory uses, and that code links every row's name to the vendor's profile at /directory/<name>.

CrawlCheck has no profile there, because it is not in the list the profiles are built from. So the one row on the page that measures us linked to a page that has never existed. The fix makes that row link to the latest report on crawlcheck.io, which is the evidence the row summarises, and the old address now redirects permanently to the self-check page for anyone who has it.

The cause is worth naming because it recurs. Shared rendering code carries an assumption about the data it is given. Here the assumption was that every row is a directory member. The self-check page passed a row that is deliberately not one, and the link was built anyway.

Every report opens with the three stages an answer engine goes through, reach, read and quote, and lists the sections under each as links that jump to that section further down. On a report for a site that is not a local business, three of those sections are not drawn at all: local presence, claim consistency and entity corroboration. They were still listed as links, marked n/a, and clicking one went nowhere.

The report already knows which sections it measured; the list is built from the same set. The fix draws an unmeasured row as plain text instead of a link. The contrasting case was checked before calling it done: on a local business report, the three rows are still links and all three sections are there.

The anchor we reported missing that was there #

During the audit we first listed a fourth dead anchor, #site, on the whole-site row. It was not dead. The first check looked for a section element carrying that id; the id sits on a different kind of element. A query for any element with the id found it at once. The finding was withdrawn before anything was changed.

This is the same mistake as grepping a class name against raw HTML and finding it inside a stylesheet: a check shaped more narrowly than the thing it claims to test. The rule we apply to findings about other sites, confirm the accusation by hand on what was actually served, applied here to a finding about our own.

What it adds up to #

Four dead links in 2,260 is a small number, and none was on a page a search engine sends much traffic to. The value of running the check was less the count than the causes: a shared renderer trusting its input, and a list that linked to things it had not drawn. Both are template defects, which means each one was on every page of its kind, not just the page where it was found.

The check is now part of the routine after any change to shared page code. The status of each file a crawler reads first is covered by the robots.txt status guide, and the rest of the method is in the internal link guide linked above.

Every figure above came out of this scanner.

Point it at your own domain and see the same measurements, free.

Scan a domain — free

The main product

Found this on your own site? We fix it for $749.

Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.

Questions this post answers

How many internal links on crawlcheck.io were broken?

One page link and three in-page anchors out of 2,260 distinct link targets and 38 anchors on the report checked. All 924 sitemap pages returned 200.

Why weren't the six 400 responses counted as broken?

They are API endpoints that require a domain parameter. A request without it is incomplete, and a 400 is the correct answer to an incomplete request.

What caused the broken link on the self-check page?

The shared directory table links every row to a vendor profile. CrawlCheck is deliberately kept out of the directory, so its row linked to a profile that does not exist.

What were the dead in-page anchors?

Three section links on reports for non-local sites pointed at sections that are only drawn for local businesses. They now render as plain text when the section is not measured.

How can I run the same audit on my site?

List your sitemap URLs, fetch each, collect every internal href, request the ones not in the sitemap, then check that every in-page link has a matching id in the rendered page. The companion guide walks through each step.

Related findings

How anything measured in this article was measured15client identitiesone second, one address5machine filesapex and www114named agentsresolved from robots.txt24sections scoredreach, read, quoteHow anything measured here was measured15 client identities5 machine files114 named agents24 sections scoredone second, one addressapex and wwwresolved from robots.txtreach, read, quote
No account, nothing installed, and the same sequence on every domain — which is what makes one scan comparable to another. Run it on your own site.

Comments

Comments are read before they appear. Nothing is published automatically, and no account is needed.

Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.

All findings · The dataset · How the dataset works