fix(cron): grace window must defer a fire, not drop it - #20
Merged
Conversation
The catch-up sweep skipped any fire inside the 90s grace window on the reasoning that "node-cron has not yet had its chance". On a post-sleep wake that is backwards: node-cron's minute-tick was suspended through that minute and never replays it, which is the whole reason catch-up exists. The sweep then wrote writeCheckpoint(now) unconditionally, pushing lastCheck past the skipped fire so the next sweep rejected it via `prevFire <= lastCheck`. Net: a 90s-wide blind spot immediately before every wake sweep. Recurring jobs hide it (they fire again next interval); a date-pinned one-shot loses its fire until the schedule next comes round. Observed: manamurah-ayam-hub-verdict-ONESHOT (`30 9 14 7 *`, MYT) was due 2026-07-14T01:30:00Z. The host swept at 01:30:59Z -- 59s < 90s grace -- so it was skipped, then buried. Jobs at 01:22 and 01:40 replayed normally. Its next natural fire would have been July 2027. computeMissedFires now reports `deferred` alongside replay/tooOld, and checkpointFloor() clamps the persisted checkpoint to just before the earliest deferred fire. The fire is reconsidered next sweep, where the run-log dedup makes the real call: fired on time -> vetoed (this is what guards the 2026-06-07 sitrep double-fire, and is unchanged); slept through -> replayed once. Every skip path is now replayed, deferred, or warned -- none silent. Tests drive two consecutive sweeps and pin both failure modes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B849gvnP1gpazo2U4R9hPN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
The catch-up sweep skipped any fire inside the 90 s grace window on the reasoning that "node-cron has not yet had its chance." On a post-sleep wake that reasoning is backwards: node-cron's minute-tick was suspended through that minute and never replays it — which is the entire reason this module exists.
The sweep then wrote
writeCheckpoint(now)unconditionally in itsfinally, pushinglastCheckpast the skipped fire. The next sweep rejected it viaprevFire <= lastCheck. The fire was gone.Net: a 90-second-wide blind spot immediately before every wake sweep. Recurring jobs hide it (they just fire again next interval); a date-pinned one-shot loses its fire until the schedule next comes round.
Observed in production
manamurah-ayam-hub-verdict-ONESHOT(30 9 14 7 *, Asia/Kuala_Lumpur) was due2026-07-14T01:30:00Z. The host woke and swept at 01:30:59Z — 59 s < 90 s grace — so it was skipped, then buried. Jobs on either side of it replayed normally:Its next natural fire would have been July 2027. Found because the one-shot was a deliberate re-check that never happened.
The fix
computeMissedFiresnow returnsdeferred: DeferredFire[]alongsidereplay/tooOld, andcheckpointFloor(now, deferred)clamps the persisted checkpoint tomin(now, earliestDeferred - 1).The fire is reconsidered on the next sweep, by which point the grace has passed and the run-log dedup makes the real decision:
The grace window keeps its anti-race role but can no longer destroy a fire. Every skip path in this module is now replayed, deferred, or warned — none silent.
Tests
catchup.test.tsdrives two consecutive sweeps and pins both failure modes:holds the checkpoint behind the deferred fire instead of advancing to nowreplays the fire on the next sweep when the host slept through it← the dropdoes NOT replay when node-cron fired it on time (no double-fire)← the June regressioncheckpointFloor— earliest-wins, never advances pastnowVerified against the real production values (actual schedule + the actual 01:30:59Z wake from
gateway.log): the old checkpoint buries the fire on both sweeps; the new one replays it. 798/798 tests pass,tsc --noEmitclean. Built and running on the live gateway (boot54cfa5d3).🤖 Generated with Claude Code
https://claude.ai/code/session_01B849gvnP1gpazo2U4R9hPN