Make subscription status durable and report data freshness separately#883
Merged
Conversation
The public status endpoint reported running feeds as STOPPED because the subscription status was a write-once Redis flag, defaulted to STOPPED when absent, that the restart path removed and a leader restart wiped — fully decoupled from whether the feed was actually being polled. - Durable desired-state: SubscriptionRegistry.clearInMemory() preserves the Redis status on leader (re)start; FeedUpdater.createSubscriptions honors an explicit STOPPED so an enabled-but-stopped feed is not auto-resubscribed by a new leader (SubscriptionRegistry.isStopped / FeedUpdater.shouldSubscribe). - Restart/stop divergence fix: restartSubscription removes only the in-memory id (removeSubscriptionId), never HDEL'ing the durable status; setup failures keep STARTING; maybeRetrySubscription guards the retry against reviving stopped feeds or creating duplicate pollers. - Report freshness separately: new FeedFreshnessService (overdue logic extracted from MetricUpdater, which now delegates) exposes isLive/lastUpdated; PublicFeedProviderStatus gains dataFresh + lastUpdated. subscriptionStatus now means durable intent, dataFresh means actual data flow.
Surface the new dataFresh / lastUpdated fields from the status endpoint as separate "Data Fresh" (Fresh/Stale chip) and "Last Updated" (relative time) columns, so subscription status (durable intent) and actual data flow are visible side by side.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #883 +/- ##
============================================
+ Coverage 80.19% 80.31% +0.11%
- Complexity 1540 1566 +26
============================================
Files 206 207 +1
Lines 5741 5801 +60
Branches 377 388 +11
============================================
+ Hits 4604 4659 +55
- Misses 919 922 +3
- Partials 218 220 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
createSubscription's setup-failure branch redundantly re-set STARTING on the async retry path, racing with the synchronous STARTING set by the caller and making updateSubscriptionStatus(STARTING) verify counts non-deterministic (green locally, red in CI on testStartSubscription). The re-set is unnecessary: every caller sets STARTING before createSubscription and, since restart now keeps the durable status (removeSubscriptionId), nothing clears it. Remove it so the failure path leaves the status untouched, and assert that invariant (never removes / never marks STOPPED) instead.
assadriaz
pushed a commit
that referenced
this pull request
Jun 24, 2026
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The public status endpoint (
/status/feed-providers) reported actively-running feeds asSTOPPED. Confirmed in prod: 32 STARTED / 44 enabled-STOPPED, while the STOPPED feeds (boltoslo, oslobysykkel, getaround*, ryde*, dott*) were all serving data minutes old.Root cause: subscription status was a write-once Redis flag, defaulted to
STOPPEDwhen absent, that the restart path removed and a leader restart wiped — fully decoupled from whether the feed was actually being polled. Investigation ruled out multi-leader, Redis eviction (evicted_keys=0), and read-replica divergence; the raw leader map literally held only 33 entries with the broken feeds absent.Changes
1. Durable subscription desired-state (the "survives leader restart" requirement)
SubscriptionRegistry.clearInMemory()preserves the durable Redis status on leader (re)start;start()/stop()no longer wipe it.SubscriptionRegistry.isStopped()+FeedUpdater.shouldSubscribe()—createSubscriptionsskips anenabledprovider whose subscription is explicitlySTOPPED, so a stopped feed is not auto-resubscribed by a new leader.2. Restart/stop divergence fix
restartSubscriptionremoves only the in-memory id (removeSubscriptionId), never HDEL'ing the durable status → never momentarily absent.STARTING; newmaybeRetrySubscriptionguards the 60s retry against reviving stopped feeds or creating duplicate pollers.3. Report data freshness separately
FeedFreshnessService(overdue logic extracted fromMetricUpdater, which now delegates) exposesisLive/lastUpdated.PublicFeedProviderStatusgainsdataFresh(boolean) +lastUpdated(epoch seconds).subscriptionStatus= durable intent;dataFresh= actual data flow.Behaviour
subscriptionStatusdataFreshRollout
The DTO change is additive (backward-compatible). On first deploy the currently-broken feeds self-correct: they are absent in the durable map → treated as "should start" → resubscribed and persisted as
STARTED; a genuinely-stopped feed stays stopped.Testing
mvn prettier:checkclean. New/extended:FeedFreshnessServiceTest,PublicFeedProviderStatusMapperTest,SubscriptionRegistryTest,FeedUpdaterSubscriptionTest,MetricsUpdaterTest.tsc --noEmit,eslint,vite build, prettier — all clean.