Skip to content

fix: swap entrypoint handler atomically on configuration updates - #6

Open
jpka wants to merge 1 commit into
jahmeergnlt:mainfrom
jpka:fix/issue-1-atomic-config-swap
Open

fix: swap entrypoint handler atomically on configuration updates#6
jpka wants to merge 1 commit into
jahmeergnlt:mainfrom
jpka:fix/issue-1-atomic-config-swap

Conversation

@jpka

@jpka jpka commented Jul 30, 2026

Copy link
Copy Markdown

Closes #1

/claim #1

The bug

main panics on the first configuration update, every time:

panic: sync/atomic: store of inconsistently typed value into Value
  pkg/server.(*Server).switchConfigs   server.go:111
  pkg/server.(*Server).watcher         server.go:75

NewServer seeds the entrypoint with an http.HandlerFunc; switchConfigs then stores an *http.ServeMux. atomic.Value requires every Store to use the same concrete type, so the second store panics. The panic lands in the watcher goroutine, which has no recover, so the goroutine dies and every subsequent update is silently dropped — the handler chain keeps serving the previous middleware until the process restarts. That is the behaviour described in this issue.

go test ./... on main reproduces it 100% of the time.

What changed

Type safety by construction. The handler now lives in atomic.Pointer[http.Handler] behind a private setHandler. The invariant is enforced by the compiler instead of relying on every future call site remembering to wrap its value.

Consistency across the swap. The new chain is built before the lock is taken, then the configuration and the handler built from it are published in a single critical section. GetConfig can never report a configuration that the live chain does not already implement.

Deterministic, panic-free chain construction. Routers are visited in sorted key order — Go randomises map iteration, so which router won a contested path previously varied between builds of the same input. Empty and duplicate paths are now reported via GetConfigErrors() and skipped, rather than reaching http.ServeMux.Handle, which panics on both.

A watcher that cannot be killed. Each switch runs under a recover, so one malformed snapshot cannot strand the server on a stale chain.

Semantics note

Snapshots continue to replace rather than merge. Merging is tempting for multi-provider setups, but without per-provider ownership a router withdrawn by its provider can never be removed, and two providers claiming one path collide. TestRouteRemovedOnConfigReplace pins the current behaviour so any future move to merging is a deliberate, tested decision.

Tests

The existing TestConcurrentConfigurationUpdates is unchanged. Added:

Test Guards
TestHandlerSwapAppliesAfterFirstConfig The regression above — the first update must actually reach the chain
TestNoPartialMiddlewareApplication A response never mixes one snapshot's body with another's middleware header
TestConcurrentProviderUpdates Four providers pushing distinct snapshots under request load
TestConfigAndHandlerAgreeAfterChurn API and live chain agree once updates quiesce
TestRouteRemovedOnConfigReplace Withdrawn routes stop serving
TestWatcherSurvivesMalformedConfig Conflicting/empty paths are reported, and later updates still apply
TestBuildHandlerChainIsDeterministic Identical input always yields an identical chain
TestDefaultHandler404BeforeConfig Entrypoint is usable before any configuration arrives

Verified locally on Go 1.26.5: go vet, gofmt -l, and go test ./... -race -count=3 green across three independent runs (nine executions per test), no flakes.

I also confirmed the regression tests genuinely fail when the fix is reverted. Worth noting: with the recover in place, reverting the atomic fix degrades the failure from a crash into precisely the symptom this issue reports — updates accepted, chain silently stale — and the tests still catch it.

CI

Adds .github/workflows/ci.yml: gofmt, go vet, go build, and go test -race -count=3 on Go 1.21 (the go.mod minimum) and current stable. -count=3 because a single green run of a concurrency test is not evidence that a swap is atomic.

Credit

#5 identified the same root cause first, and its author is credited as co-author on the commit. This PR differs in enforcing the invariant at compile time and in adding the concurrent and race-detector coverage the issue asks for.

The entrypoint handler was seeded with an http.HandlerFunc in NewServer and
replaced with an *http.ServeMux in switchConfigs. atomic.Value requires every
Store to use the same concrete type, so the first configuration update panicked
in the watcher goroutine. With no recover in that loop the goroutine died and
every later update was dropped, leaving the handler chain serving the previous
middleware until the process restarted -- the behaviour reported in jahmeergnlt#1.

Changes:

- Hold the handler in atomic.Pointer[http.Handler] behind a private setHandler,
  so the type invariant is enforced by the compiler rather than by convention.
- Build the new chain before taking the lock, then publish the configuration and
  its handler in one critical section, so GetConfig never reports a state the
  live chain does not implement.
- Build the chain from routers in sorted key order and skip empty or duplicate
  paths with a reported reason, instead of letting http.ServeMux panic on them.
- Recover around each configuration switch so one bad snapshot cannot kill the
  watcher and strand the server on a stale chain.
- Add concurrent, multi-provider and malformed-config tests, plus CI running
  go vet, gofmt and go test -race -count=3.

Configuration snapshots continue to replace rather than merge, so a router
withdrawn by a provider stops serving; TestRouteRemovedOnConfigReplace pins it.

PR jahmeergnlt#5 identified the same root cause first and is credited below.

Closes jahmeergnlt#1

Co-Authored-By: Wasim <wasim143mr@gmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@opirebot

opirebot Bot commented Jul 30, 2026

Copy link
Copy Markdown

👀 We've notified the reward creators here.
Make sure your payment account is ready to receive the payment for your hard work 💪

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🎯 Fix Race Condition Causing Stale Middleware Chain During Concurrent Provider Updates

1 participant