fix: swap entrypoint handler atomically on configuration updates - #6
Open
jpka wants to merge 1 commit into
Open
Conversation
The entrypoint handler was seeded with an http.HandlerFunc in NewServer and replaced with an *http.ServeMux in switchConfigs. atomic.Value requires every Store to use the same concrete type, so the first configuration update panicked in the watcher goroutine. With no recover in that loop the goroutine died and every later update was dropped, leaving the handler chain serving the previous middleware until the process restarted -- the behaviour reported in jahmeergnlt#1. Changes: - Hold the handler in atomic.Pointer[http.Handler] behind a private setHandler, so the type invariant is enforced by the compiler rather than by convention. - Build the new chain before taking the lock, then publish the configuration and its handler in one critical section, so GetConfig never reports a state the live chain does not implement. - Build the chain from routers in sorted key order and skip empty or duplicate paths with a reported reason, instead of letting http.ServeMux panic on them. - Recover around each configuration switch so one bad snapshot cannot kill the watcher and strand the server on a stale chain. - Add concurrent, multi-provider and malformed-config tests, plus CI running go vet, gofmt and go test -race -count=3. Configuration snapshots continue to replace rather than merge, so a router withdrawn by a provider stops serving; TestRouteRemovedOnConfigReplace pins it. PR jahmeergnlt#5 identified the same root cause first and is credited below. Closes jahmeergnlt#1 Co-Authored-By: Wasim <wasim143mr@gmail.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
5 tasks
|
👀 We've notified the reward creators here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1
/claim #1
The bug
mainpanics on the first configuration update, every time:NewServerseeds the entrypoint with anhttp.HandlerFunc;switchConfigsthen stores an*http.ServeMux.atomic.Valuerequires everyStoreto use the same concrete type, so the second store panics. The panic lands in thewatchergoroutine, which has norecover, so the goroutine dies and every subsequent update is silently dropped — the handler chain keeps serving the previous middleware until the process restarts. That is the behaviour described in this issue.go test ./...onmainreproduces it 100% of the time.What changed
Type safety by construction. The handler now lives in
atomic.Pointer[http.Handler]behind a privatesetHandler. The invariant is enforced by the compiler instead of relying on every future call site remembering to wrap its value.Consistency across the swap. The new chain is built before the lock is taken, then the configuration and the handler built from it are published in a single critical section.
GetConfigcan never report a configuration that the live chain does not already implement.Deterministic, panic-free chain construction. Routers are visited in sorted key order — Go randomises map iteration, so which router won a contested path previously varied between builds of the same input. Empty and duplicate paths are now reported via
GetConfigErrors()and skipped, rather than reachinghttp.ServeMux.Handle, which panics on both.A watcher that cannot be killed. Each switch runs under a
recover, so one malformed snapshot cannot strand the server on a stale chain.Semantics note
Snapshots continue to replace rather than merge. Merging is tempting for multi-provider setups, but without per-provider ownership a router withdrawn by its provider can never be removed, and two providers claiming one path collide.
TestRouteRemovedOnConfigReplacepins the current behaviour so any future move to merging is a deliberate, tested decision.Tests
The existing
TestConcurrentConfigurationUpdatesis unchanged. Added:TestHandlerSwapAppliesAfterFirstConfigTestNoPartialMiddlewareApplicationTestConcurrentProviderUpdatesTestConfigAndHandlerAgreeAfterChurnTestRouteRemovedOnConfigReplaceTestWatcherSurvivesMalformedConfigTestBuildHandlerChainIsDeterministicTestDefaultHandler404BeforeConfigVerified locally on Go 1.26.5:
go vet,gofmt -l, andgo test ./... -race -count=3green across three independent runs (nine executions per test), no flakes.I also confirmed the regression tests genuinely fail when the fix is reverted. Worth noting: with the
recoverin place, reverting the atomic fix degrades the failure from a crash into precisely the symptom this issue reports — updates accepted, chain silently stale — and the tests still catch it.CI
Adds
.github/workflows/ci.yml:gofmt,go vet,go build, andgo test -race -count=3on Go 1.21 (the go.mod minimum) and current stable.-count=3because a single green run of a concurrency test is not evidence that a swap is atomic.Credit
#5 identified the same root cause first, and its author is credited as co-author on the commit. This PR differs in enforcing the invariant at compile time and in adding the concurrent and race-detector coverage the issue asks for.