fix: do not mutate the caller's exclude map in writeSubset, drop the global mutex - #21
Open
burruplambert wants to merge 1 commit into
Open
Conversation
…al mutex writeSubset wrote the Header-Order:/PHeader-Order: magic keys into the caller's exclude map. Callers such as Response.Write and httputil dump pass shared package-level maps, so that write both raced with concurrent header writes and permanently leaked the exclusions into every later write using the same map - an orderless write's output could depend on whether an ordered write had happened earlier in the process. The package-global RWMutex (added in 7abeb12 and 107c8e4) exists only to guard that mutation, at the cost of two global lock operations per header key on every sort, shared across all connections. Instead, add the magic keys to a small copy of the exclude map in the one path that needs them and never write to the caller's map. The mutex then guards nothing and is removed along with the per-key RLock/RUnlock in SortedKeyValues and SortedKeyValuesBy. Wire output is unchanged for every fresh-process input; the only behavioral change is that orderless writes no longer depend on the process's write history, which was unintended state leakage. Adds a test pinning that WriteSubset does not mutate the caller's map, a shared-map concurrency test for the race detector, and a parallel WriteSubset benchmark (the mutex cost only shows under concurrency: about -28% ns/op on 2 cores with it removed).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
writeSubsetwrites theHeader-Order:/PHeader-Order:magic keys into the caller's exclude map:Callers pass shared package-level maps -
respExcludeHeaderfromResponse.Write,reqWriteExcludeHeaderDumpfrom httputil - so that write both races with concurrent header writes and permanently leaks the two exclusions into every later write that uses the same map. An orderless write's output ends up depending on whether an ordered write happened earlier in the process.The package-global
RWMutexexists only to guard that mutation. It was added in 7abeb12 ("in order to avoid any conflicts in the exclude map when several tasks are run") and extended toSortedKeyValuesByin 107c8e4. The cost is two global lock operations per header key on every sort, shared across all connections.Changes
writeSubsetadds the magic keys to a small copy of the exclude map and never writes to the caller's mapRLock/RUnlockinSortedKeyValuesandSortedKeyValuesByare removed-raceBenchmarkHeaderWriteSubsetParallel, since the mutex cost only appears under concurrencyBehavior
Wire output is unchanged for every fresh-process input. The only behavioral difference is that an orderless write no longer depends on the process's write history, which was unintended state leakage rather than a feature.
Measurements
Single-threaded
BenchmarkHeaderWriteSubsetis about -6%; uncontended atomics are cheap. UnderBenchmarkHeaderWriteSubsetParallelon 2 cores it is -28% ns/op, and the gap widens with core count.Note
TestHeaderWritecase 9 fails on master, before and after this change: the order map is built from theHeader-Order:values verbatim while the comparator looks them up lowercased, so mixed-case order entries never match. Unrelated to this change and left alone, since fixing it changes wire behavior.