Age the Memory metrics window on the monotonic clock - #873
Conversation
Review price tag🟢 95 effective lines — about 11–29 min of focused review (based on 200–500 lines/hour). This is within the range where reviewers find the most issues per line, and small changes usually receive feedback the fastest. Spread across many files. This PR changes 10 files — more than about 9 in 10 PRs touch. Each extra file is another piece of context a reviewer has to load and hold at once. Why these numbers?These minutes are what careful defect-finding costs at 200–500 lines/hour — the rate review studies report, not how long a skim takes. "Effective lines" already exclude generated files and lockfiles. Treat the rates and the 200/400 thresholds as guardrails, not laws. |
| def increment | ||
| @buckets[current_bucket] += 1 | ||
| # | ||
| # @note Times must be nondecreasing; FIFO eviction assumes chronological inserts. |
There was a problem hiding this comment.
Sharp observation. We use UTC time which never moves backwards, yet NTP adjustments could cause time move in any direction. Normally this happens by slowing/speeding up time, but bigger leaps are possible too.
This is harmless, when happens forward - e.g. when you open a laptop after hibernation but breaks the window if it happens in an opposite direction, until window expire.
This started as an effort to reduce allocations, but you pointed out here that maybe we should not use wall clock time here at all and the better feat would be to use monotonic seconds instead
@OursCodeur how do you feel about this change?
There was a problem hiding this comment.
Agreed, eviction should run on elapsed time, backwards step appends an older-keyed bucket behind newer ones, slide_window! never reaches it, and it outlives the window.
I'd use monotonic_time for insertion and eviction (w/ float ms -> s at the boundary) and keep current_time for last_success_at and Failure#occurred_at. Redis unchanged, monotonic epochs don't compare across hosts.
AFAIK CLOCK_MONOTONIC stops during suspend, so stale events survive hibernation. If active-time behavior is acceptable I can do it as a follow-up PR with a backward-step regression spec.
There was a problem hiding this comment.
Hey, do you think the current pr still makes sense if we gonna change wall clock time to monotonic time? It seems like the shape before the change fits more here so the SlidingWindow abstraction controls the clock used.
Also maybe the #sum_in_window need to change to accept window size rather then absolute time.
def sum_in_window: (Integer window_size) -> IntegerThere was a problem hiding this comment.
Yeah sounds right. Clock injected into SlidingWindow, duration-based sum_in_window, a backward-step regression spec, and a Timecop shim so specs and features can age the monotonic window. Or a fresh PR if you'd rather close this one, your call.
There was a problem hiding this comment.
I think we con repurpose this pr, it's fine. Great we figured it out and found a real problem
There was a problem hiding this comment.
Had it stashed in advance ;) force-pushed the monotonic shape and updated the description.
d021031 to
b8ae054
Compare
Closes #779
SlidingWindownow owns the clock domain: buckets key on monotonic seconds andsum_in_window(window_size)takes a duration, so callers cannot feed wall timestamps into the window.WindowMetrics#build_metrics_snapshotloses its wall read;record_successandrecord_failureare unchanged from develop.Reason : Wall clock can corrupt FIFO eviction: a backward NTP step inserts an older-keyed bucket behind newer ones where
slide_window!never reaches it, so it outlives the window. Monotonic keys make the nondecreasing invariant hold by construction.Re: the original goal, each record op reads the wall clock exactly once, now pinned by specs. The internal monotonic reads are allocation-free and cheap (144ns vs 345ns for
Time.now.utc).Benchs:
Tests:
A regression spec pins that events expire by elapsed time when the wall clock steps backwards. The suites set
Timecop.mock_process_clock = trueso travel and freeze move the monotonic lane; the "outside the running window" shared examples use timecop's relative form (Timecop.freeze(-window_size - 10)) because freezing to aTimetarget deliberately leaves the monotonic clock unshifted.system_clock_spec's monotonic example no longer freezes: with the process clock mocked, a frozen assertion compares the mock to itself.sliding_window_specdrives the injected clock to absolute seconds instead of nesting Timecop blocks.