CrawlCheck

Findings · 2026-08-21 · By

Our own analytics was adding 650 ms to every request, and a comment kept it there

A first-party beacon reported that 70% of our LCP was time-to-first-byte. The cause was three telemetry writes awaited before routing — next to a comment explaining why they could not be moved. The comment was wrong.

We shipped a first-party performance beacon so we could see real field data for sites the Chrome UX Report will never cover. The first real reading it produced was about our own homepage, and it was not flattering.

LCP1,440 ms
TTFB1,014 ms (needs-improvement)
LCP sub-phases [TTFB, load delay, load duration, render delay][1014, 0, 0, 427]

Seventy percent of Largest Contentful Paint was time-to-first-byte. Nothing was slow about the page. The server had simply not started answering yet.

This is a static-ish page served from an edge worker. A second of TTFB is not a rendering problem, an image problem or a font problem. It is the server thinking before it speaks.

The natural experiment that located it in one request

Before profiling anything we noticed the codebase already contained a controlled experiment, because two routes sat on opposite sides of one block of code.

RouteWork doneTTFB
/v.jsserve a static string56 ms
/api/buildserve a smaller static object700 ms

The route doing less work took twelve times longer. The only difference between them was which side of the telemetry block they were routed on. That is a 650 ms fixed tax, and every route except one was paying it.

What the code was doing

Three recorders ran on every request: one logging the visit, one classifying the user-agent against known crawler ranges, and one writing an aggregate counter. All three did network-backed writes. All three were awaited before any routing decision was made.

So the order of operations for every single request, including a crawler asking for robots.txt, was: write three records, wait for all three, then work out what was being asked for.

The fix is one of the oldest in the platform: ctx.waitUntil() hands the runtime a promise to finish after the response has been sent. The response goes out immediately and the writes complete on their own time.

The comment that kept it alive

This is the part worth writing down, because the bug was not hard. It had a note next to it explaining why it could not be fixed:

ctx is not available in this signature

The signature was async fetch(request, env, ctx).

ctx was the third argument. It had been there the whole time. Somebody wrote that comment while looking at a different version of the function, or wrote it from memory, and after that nobody re-checked — because a comment explaining why something is impossible is read as a decision that has already been made, not as a claim to verify.

A wrong comment is worse than no comment. No comment invites you to look. A confident wrong comment tells you not to bother, and it kept a 650 ms tax on every request alive for as long as anybody believed it.

After

RouteBeforeAfter
/api/build720 ms60 ms
/ (homepage)~1,014 ms85 ms
/policy63 ms

Roughly eight to twelve times faster on TTFB, sitewide, from moving three calls into a function that already existed. Re-measured a day later across four routes and four samples each, the numbers hold between 49 ms and 76 ms.

Verifying that we had not just deleted the telemetry

Making something fast by quietly breaking it is the obvious failure here, and “the requests got faster” is not evidence that the writes still happen. ctx.waitUntil defers work; it can also silently drop it if the promise is never handed over correctly.

So we sent three requests identifying as a named AI crawler, spaced 2.5 seconds apart, and read the counter before and after. It went up by exactly three. The recording still works; it just no longer happens while the visitor waits.

What transfers

Method

TTFB figures are time_starttransfer from an ordinary HTTP client with a cache-busting query parameter, taken from outside the network serving the site. Before-figures were taken while the awaited version was live; after-figures immediately following the deploy and again a day later. The field reading came from our own beacon on a real browser session, reporting the standard web-vitals attribution breakdown.

Every figure above came out of this scanner.

Point it at your own domain and see the same measurements, free.

Scan a domain — free

Related findings

All findings · The dataset · How the dataset works