Skip to content

fix: do not mutate the caller's exclude map in writeSubset, drop the global mutex - #21

Open
burruplambert wants to merge 1 commit into
bogdanfinn:masterfrom
burruplambert:fix/writesubset-exclude-mutation
Open

fix: do not mutate the caller's exclude map in writeSubset, drop the global mutex#21
burruplambert wants to merge 1 commit into
bogdanfinn:masterfrom
burruplambert:fix/writesubset-exclude-mutation

Conversation

@burruplambert

Copy link
Copy Markdown

Summary

writeSubset writes the Header-Order:/PHeader-Order: magic keys into the caller's exclude map:

if exclude == nil {
    exclude = make(map[string]bool)
}
mutex.Lock()
exclude[HeaderOrderKey] = true
exclude[PHeaderOrderKey] = true
mutex.Unlock()

Callers pass shared package-level maps - respExcludeHeader from Response.Write, reqWriteExcludeHeaderDump from httputil - so that write both races with concurrent header writes and permanently leaks the two exclusions into every later write that uses the same map. An orderless write's output ends up depending on whether an ordered write happened earlier in the process.

The package-global RWMutex exists only to guard that mutation. It was added in 7abeb12 ("in order to avoid any conflicts in the exclude map when several tasks are run") and extended to SortedKeyValuesBy in 107c8e4. The cost is two global lock operations per header key on every sort, shared across all connections.

Changes

  • writeSubset adds the magic keys to a small copy of the exclude map and never writes to the caller's map
  • the mutex then guards nothing, so it and the per-key RLock/RUnlock in SortedKeyValues and SortedKeyValuesBy are removed
  • tests: the caller's map is not mutated, and concurrent writes sharing one exclude map are clean under -race
  • adds BenchmarkHeaderWriteSubsetParallel, since the mutex cost only appears under concurrency

Behavior

Wire output is unchanged for every fresh-process input. The only behavioral difference is that an orderless write no longer depends on the process's write history, which was unintended state leakage rather than a feature.

Measurements

Single-threaded BenchmarkHeaderWriteSubset is about -6%; uncontended atomics are cheap. Under BenchmarkHeaderWriteSubsetParallel on 2 cores it is -28% ns/op, and the gap widens with core count.

Note

TestHeaderWrite case 9 fails on master, before and after this change: the order map is built from the Header-Order: values verbatim while the comparator looks them up lowercased, so mixed-case order entries never match. Unrelated to this change and left alone, since fixing it changes wire behavior.

…al mutex

writeSubset wrote the Header-Order:/PHeader-Order: magic keys into the
caller's exclude map. Callers such as Response.Write and httputil dump
pass shared package-level maps, so that write both raced with
concurrent header writes and permanently leaked the exclusions into
every later write using the same map - an orderless write's output
could depend on whether an ordered write had happened earlier in the
process.

The package-global RWMutex (added in 7abeb12 and 107c8e4) exists only
to guard that mutation, at the cost of two global lock operations per
header key on every sort, shared across all connections.

Instead, add the magic keys to a small copy of the exclude map in the
one path that needs them and never write to the caller's map. The
mutex then guards nothing and is removed along with the per-key
RLock/RUnlock in SortedKeyValues and SortedKeyValuesBy.

Wire output is unchanged for every fresh-process input; the only
behavioral change is that orderless writes no longer depend on the
process's write history, which was unintended state leakage.

Adds a test pinning that WriteSubset does not mutate the caller's map,
a shared-map concurrency test for the race detector, and a parallel
WriteSubset benchmark (the mutex cost only shows under concurrency:
about -28% ns/op on 2 cores with it removed).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant