Findings · 2026-10-14 · By VSNARY | Emmanuel Orta · 0 views
Cron job ran a minute late; our directory froze for 20 hours
Our directory of SEO and AI tools promises every card is under a day old. For about 20 hours on 13 and 14 October its index did not rebuild at all, and eight cards linked to reports that no longer existed. Nothing was logged, because the job checked the clock instead of asking which schedule had fired.
From 5:06 PM MDT on 13 October 2026, the index behind CrawlCheck's directory of SEO and AI tools stopped rebuilding. The rebuild ran inside an hourly job scheduled for minute 2, and the code also required the clock to read minute 2; the platform's invocation log showed every run of that schedule starting at minute 3, so the job exited quietly every hour. Eight directory cards pointed at reports that no longer existed. The fix decides by the schedule that fired, not by the clock, and logs a failed rebuild. The index was rebuilt about 20 hours later.
The directory lists the companies that sell SEO and AI-visibility tools, each measured by the same scan as any other site. It promises that every card is under a day old and says on the page when its index was last rebuilt. For about 20 hours on 13 and 14 October 2026 that promise was not kept, and the page said so to anyone who read the date. We did not read it. A link sweep found the problem; this is what it was.
What a visitor saw #
Most cards looked normal, because most vendors had not changed. Eight did not: clicking them opened a report page that answered 404. Seven of the eight pointed at older reports that had been replaced by newer scans of the same vendor, which the stale index had never picked up. The eighth was a separate defect, covered below. The "Index last rebuilt" line under the directory read 23:06 UTC on 13 October the whole time.
The cause: the clock instead of the schedule #
The directory is maintained by a scheduled job. CrawlCheck runs on Cloudflare Workers, whose cron triggers start the job on a schedule, and several pieces of hourly work share the same invocation. To keep the directory rebuild to one of those schedules, minute 2 of each hour, the code also checked the clock: run the rebuild only if the current minute is 2.
The platform's own invocation log for 14 October showed every run of that schedule starting at minute 3. The clock check was false every hour, the rebuild was skipped, and the job finished successfully, because skipping was a legitimate branch. There was no error to log. Cloudflare's documentation for the scheduled handler passes the job two facts that do not drift: controller.cron, the schedule that started it, and controller.scheduledTime, when it was scheduled to run. The actual start time can differ from both. We read the one value that could.
Timeline #
| Mountain Time | What happened |
|---|---|
| 13 Oct, 5:06 PM | Last automatic rebuild of the directory index |
| 14 Oct, about noon | A sweep of every page and all 4,433 internal links finds eight directory cards returning 404 |
| 14 Oct, 12:41 PM | Rebuild failures now recorded in the error log; index rebuilt by hand: 387 rows, 376 measured, all 341 report links answer 200 |
| 14 Oct, 1:10 PM | Cause found: the 1:02 PM run had skipped the rebuild on the clock check |
| 14 Oct, 1:11 PM | Fix deployed: the rebuild runs when its schedule fires, whatever the clock says |
| 14 Oct, 2:13 PM | First rebuild with no help, inside the 2 PM run |
The eighth card: a write that failed quietly #
One card had a different cause. When a vendor is re-scanned, two things are stored: the report, and a pointer from the vendor's domain to that report. On 14 October our storage returned errors for a while. In one case the pointer was written and the report was not, and the code that saved the report ignored the failure. The card then pointed at a report that did not exist. The save is now retried once and a failure is logged, and the directory rebuild repairs a card whose report is missing by falling back to the copy kept with the domain.
What changed #
| Defect | Before | After |
|---|---|---|
| Which run rebuilds the index | The run that starts when the clock reads minute 2 | The run started by the minute-2 schedule |
| A rebuild that fails or does nothing | Printed to a console nobody reads | Recorded in the error log with its cause |
| A report that fails to save | Failure ignored; pointer already moved | Retried once, then logged; the rebuild repairs the card |
What we have not fixed #
The rebuild shares its hourly run with other work, and the whole run is limited to 15 minutes; the directory pass alone takes about ten. A run stopped at the limit ends without writing anything to our log, so that failure would again be silent from the inside. What makes it visible is the line the directory already prints: when the index was last rebuilt. The directory's own question-and-answer section states the promise that line is checked against. If it ever reads more than a few hours old, the directory is stale, and anyone can see it without asking us.
If you run scheduled jobs #
- Branch on the schedule, not the clock. If several schedules share one handler, decide what to do from the schedule that fired (
controller.cronon Cloudflare Workers, the job name in most other schedulers), never from the current minute. - Treat "did nothing" as a result worth logging. A job that skips its work and exits cleanly looks identical to a job that worked.
- Publish when the work last happened. A visible timestamp turns a silent failure into a visible one. Ours was there; we had not been reading it.
- Never move a pointer before the thing it points at is saved. Or, if you must, check the save and undo the pointer when it fails.
This is one of the corrections we publish when CrawlCheck's own output was wrong. All of them are listed together, each with the defect and the fix.
Every figure above came out of CrawlCheck.
Point it at your own domain and see the same measurements, free.
The main product
Found this on your own site? We fix it for $749.
Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.
Questions this post answers
Why would a cron job silently skip its work?
When the job checks the clock to decide what to do and the platform starts it later than the scheduled minute. The check fails, the job takes its skip branch and exits successfully, so nothing is logged. Branching on the schedule that fired avoids it.
Do Cloudflare Workers cron triggers start exactly on the minute?
Not necessarily. On 14 October 2026 CrawlCheck's invocation log showed every run of a minute-2 schedule starting at minute 3. Cloudflare's scheduled handler receives controller.cron and controller.scheduledTime, which do not depend on when the run actually started.
How should a scheduled job decide which task to run?
From the schedule that triggered it: controller.cron on Cloudflare Workers, or the job name in most other schedulers. Never from the current time.
How long was the CrawlCheck directory out of date?
About 20 hours: the index was last rebuilt automatically at 5:06 PM MDT on 13 October 2026 and rebuilt by hand at 12:41 PM MDT on 14 October. The fix shipped at 1:11 PM and the first unaided rebuild ran at 2:13 PM.
How can I tell whether the CrawlCheck directory is current?
The directory page prints when its index was last rebuilt. It rebuilds every hour, so a time more than a few hours old means it is stale.
Related findings
Comments
Comments are read before they appear. Nothing is published automatically, and no account is needed.
Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.