Skip to content

Optimize gRPC request processing: fast negotiation path and reduced per-request allocations#739

Open
He-Pin wants to merge 16 commits into
apache:mainfrom
He-Pin:optimize/fast-negotiation-uri-match
Open

Optimize gRPC request processing: fast negotiation path and reduced per-request allocations#739
He-Pin wants to merge 16 commits into
apache:mainfrom
He-Pin:optimize/fast-negotiation-uri-match

Conversation

@He-Pin

@He-Pin He-Pin commented Jun 21, 2026

Copy link
Copy Markdown
Member

Motivation

Profiling the gRPC unary request hot path revealed several allocation and CPU overhead sources:

  1. Protocol negotiation: GrpcProtocol.negotiate() performed redundant header scans via separate detect() + Codecs.detect() + Codecs.negotiate() calls per request
  2. Per-request allocations: Lambda objects, ByteString.slice wrappers, Option.map closures, and PartialFunction allocations in the unary handler path
  3. URI path matching: methodName used backtick pattern matching which could be slightly improved with guard-based matching

Modification

Protocol negotiation (3 commits)

  • Fast negotiation path: Single-pass header scan for native gRPC (application/grpc+proto), avoiding redundant detect() + Codecs.detect() + Codecs.negotiate() calls
  • Respect client compression preference: Writer codec respects grpc-accept-encoding header, using the client's preferred codec order
  • String comparison fix: Changed eq to == for proper string comparison in negotiate fast path
  • Pre-computed negotiation result: Cache NativeIdentityIdentity tuple to avoid per-request allocation
  • negotiated lambda elimination: Replace Option.map with direct match to avoid closure allocation

Unary hot path (6 commits)

  • Pre-create unary method lambdas: Generated handler pre-creates lambdas for unary methods at initialization, avoiding per-request lambda allocation
  • Native identity strict decoding: Optimized decodeIdentitySingleFrame to parse gRPC frame header directly without ByteReader allocation
  • Offset-aware deserialization: Added ProtobufFrameSerializer.deserialize(data, offset, length) to bypass ByteString.slice allocation (~2 wrapper objects per request)
  • methodName guard-based matching: Replaced backtick pattern matching with guard condition for slightly cleaner generated code
  • Inline exception handling: handleUnaryException inlines the error-to-response conversion, avoiding PartialFunction allocation via GrpcExceptionHandler.from()
  • encodeFrameData small message optimization: For messages ≤4KB, combine frame header + data into single array allocation

Result

Benchmarked with ghz (complex_proto, 1000 concurrency, 50 connections, 120s, SerialGC):

Metric Baseline (1.2.0) Optimized Improvement
Throughput (ForkJoinPool) 62,899 req/s 73,584 req/s +17.0%
P99 latency (ForkJoinPool) 32.46 ms 29.20 ms -10.0%
Avg latency (ForkJoinPool) 8.34 ms 6.77 ms -18.8%
P99 latency (AffinityPool) 24.17 ms -25.5%

JMH microbenchmarks confirm:

  • GrpcMarshallingBenchmark.unmarshallStrict: ~50M ops/s
  • ScalaUnaryHandlerBenchmark.generatedUnary: ~3.8M ops/s

Tests

  • sbt runtime / Test / testOnly GrpcProtocolSpec GrpcMarshallingSpec CodecsSpec GrpcResponseHeadersSpec
  • sbt codegen / Test / test
  • All tests pass

References

  • Profiling with async-profiler (CPU + allocation)
  • Benchmark suite: grpc_bench scala_pekko_bench

He-Pin added 2 commits June 21, 2026 02:39
…g matching

Motivation:
Per-request overhead in the hot path limits high-concurrency throughput.
Protocol negotiation and URI path matching allocate objects and scan
headers on every request, even when the values are predictable.

Modification:
- GrpcProtocol.negotiate(): Add a fast path for native gRPC that does a
  single header scan instead of calling Codecs.detect() and
  Codecs.negotiate() separately (which each scan all headers). Pre-compute
  common reader/writer combinations (Identity/Identity, Identity/Gzip) as
  static vals to avoid repeated object creation.
- Handler.scala.txt template: Replace Uri.Path pattern matching with
  direct string operations on request.uri.path.toString, avoiding
  allocation of Path/Slash/Segment objects per request.

Result:
Reduced per-request allocations and header scanning overhead in the
gRPC request processing hot path.

Tests:
- runtime / compile - passed
- Validated with local benchmark (ghz, complex_proto scenario)

References:
None - performance optimization
Motivation:
Flamegraph analysis showed ~14% of CPU time spent in Deflater.deflate
(gzip compression) for response encoding. For typical small protobuf
messages in gRPC benchmarks, gzip compression adds CPU overhead without
meaningful size reduction. The default codec preference order (Gzip
before Identity) caused the server to compress every response when the
client advertised gzip support, even for tiny messages.

Modification:
- Codecs.supportedCodecs: Change order to Seq(Identity, Gzip) so the
  negotiate() function prefers Identity when both codecs are supported
- GrpcProtocol.negotiate() fast path: Match the new preference order,
  choosing Identity when present in the accept-encoding list

This matches the behavior of Vert.x gRPC server which does not compress
responses by default.

Result:
- Eliminates ~14% CPU overhead from Deflater.deflate
- Improves 99% tail latency (33.75ms -> 32.33ms in local benchmark)
- Low concurrency: 55,357 -> 56,177 req/s (+1.5%)
- Aligns with Vert.x default behavior for fair comparison

Tests:
- runtime / compile - passed
- Validated with local benchmark (ghz, complex_proto scenario)

References:
None - performance optimization
@He-Pin He-Pin marked this pull request as draft June 21, 2026 16:00
He-Pin added a commit to He-Pin/pekko-grpc that referenced this pull request Jun 21, 2026
…ndleUnary

Motivation:
The ProtobufFrameSerializer fast path in handleUnary directly called
deserialize(data, offset, length) without checking if the data was
sufficient to contain a frame header. When the request entity was
empty (HttpEntity.empty), data.length - FrameHeaderSize produced a
negative length (-5), bypassing the MissingParameterException that
reader.decodeSingleFrame would normally throw. This caused the
ErrorReportingSpec test "should respond with an 'invalid argument'
gRPC error status when calling an method without a request body"
to fail because the expected INVALID_ARGUMENT status was not returned.

Modification:
Add a length check before the ProtobufFrameSerializer.deserialize call:
if data.length < FrameHeaderSize, throw MissingParameterException to
match the behavior of reader.decodeSingleFrame which validates the
frame header is present before attempting to parse it.

Result:
Empty request bodies now correctly return grpc-status: 3
(INVALID_ARGUMENT) regardless of whether the ProtobufFrameSerializer
fast path or the standard decodeSingleFrame path is used.

Tests:
sbt plugin-tester-scala / Test / testOnly example.myapp.helloworld.ErrorReportingSpec
All 2 tests passed.

References: Fixes CI failure in PR apache#739
@He-Pin He-Pin marked this pull request as ready for review June 22, 2026 09:00
@pjfanning

Copy link
Copy Markdown
Member

@He-Pin can the gzip change be moved to its own PR because it is a behaviour change - better for transparency?

@He-Pin

He-Pin commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

I agree, I think we can add an adaptive one for it,wdyt @pjfanning

@pjfanning

Copy link
Copy Markdown
Member

@He-Pin if you want to make the compression adaptive then that's ok by me.

@He-Pin

He-Pin commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

Let's do that on this PR. at $work, it's adaptive.

He-Pin added a commit that referenced this pull request Jun 23, 2026
…ndleUnary

Motivation:
The ProtobufFrameSerializer fast path in handleUnary directly called
deserialize(data, offset, length) without checking if the data was
sufficient to contain a frame header. When the request entity was
empty (HttpEntity.empty), data.length - FrameHeaderSize produced a
negative length (-5), bypassing the MissingParameterException that
reader.decodeSingleFrame would normally throw. This caused the
ErrorReportingSpec test "should respond with an 'invalid argument'
gRPC error status when calling an method without a request body"
to fail because the expected INVALID_ARGUMENT status was not returned.

Modification:
Add a length check before the ProtobufFrameSerializer.deserialize call:
if data.length < FrameHeaderSize, throw MissingParameterException to
match the behavior of reader.decodeSingleFrame which validates the
frame header is present before attempting to parse it.

Result:
Empty request bodies now correctly return grpc-status: 3
(INVALID_ARGUMENT) regardless of whether the ProtobufFrameSerializer
fast path or the standard decodeSingleFrame path is used.

Tests:
sbt plugin-tester-scala / Test / testOnly example.myapp.helloworld.ErrorReportingSpec
All 2 tests passed.

References: Fixes CI failure in PR #739
He-Pin added a commit that referenced this pull request Jun 23, 2026
Motivation:
The previous PR always used Identity (no compression) for native gRPC
responses to avoid CPU overhead on small messages. However, large
messages benefit significantly from gzip compression. An adaptive
approach that compresses only messages above a configurable threshold
provides the best of both worlds.

Modification:
- Add AdaptiveGzip codec that only compresses messages above a
  configurable threshold (default 1024 bytes). Messages below the
  threshold pass through uncompressed while the grpc-encoding header
  still advertises gzip.
- Add Codec.compressWithFlag() returning (ByteString, Boolean) to
  signal per-frame compression status accurately in the gRPC frame
  header (bit 0 of the 5-byte header). This fixes a correctness bug
  where streaming frames would incorrectly set the compression flag
  even for uncompressed data.
- Add pekko.grpc.server.compression-threshold config (default 1024).
  Generated Scala handlers read config once at handler creation and
  pass the threshold as Int to avoid per-request config overhead.
- Pre-compute NativeIdentityAdaptiveGzip negotiation result and cache
  the default AdaptiveGzip instance for zero per-request allocation
  on the common path.
- Add ProtobufFrameSerializer.serializedDataSize() for efficient
  size checking without double serialization.
- Add adaptive fast path in GrpcResponseHelpers for small messages
  using serializeDataFrame single-allocation path.

Result:
- Messages below threshold: zero compression CPU overhead, sent
  uncompressed with correct frame header flag.
- Messages above threshold: gzip compressed with correct frame header
  flag. Per gRPC spec, per-frame compression flag tells clients
  whether each frame is compressed.
- Default threshold of 1024 bytes aligns with industry consensus
  (Netty, Ktor, OneUptime gRPC guide) as the break-even point where
  gzip CPU cost is justified by bandwidth savings on protobuf data.
- JIT/GC optimized: zero per-request allocation on common path via
  pre-computed negotiation result and cached AdaptiveGzip singleton.

Tests:
- runtime / Test / test - 152 tests passed
- plugin-tester-scala / Test / testOnly ErrorReportingSpec - passed
- 30 new AdaptiveGzip tests covering codec behavior, boundary
  conditions, compressWithFlag, streaming frame header correctness,
  round-trip encode/decode, instance caching, and negotiate integration

References:
Refs #739
He-Pin added 10 commits June 24, 2026 01:00
…erhead

Motivation:
Flamegraph analysis showed ~27% of handler CPU time spent in Gzip
compression (Deflater.init, Deflater.deflate, GZIPOutputStream).
For typical small gRPC messages (under 1KB), gzip compression adds
significant CPU overhead without meaningful size reduction. The ghz
benchmark client sends grpc-accept-encoding: gzip (without identity),
causing the server to compress every response.

Modification:
- GrpcProtocol.negotiate(): For native gRPC, always prefer Identity
  writer codec for responses. The per-frame compression flag (bit 0
  of the 5-byte gRPC frame header) tells the client whether each frame
  is compressed, so clients correctly handle uncompressed frames even
  when they advertised gzip support.
- This enables the serializeDataFrame fast path in responseForSingleElement,
  which creates pre-framed uncompressed data in a single byte array
  allocation, avoiding the Gzip compress → encodeDataToResponse chain.
- ScalapbProtobufSerializer.deserialize(): Use CodedInputStream with
  byte array directly (via asByteBuffer.hasArray) to avoid ByteBuffer
  wrapper allocation.

Result:
- High concurrency (1000 conn): 72,027 -> 78,585 req/s (+9.1%)
- Combined with all optimizations: 59,177 -> 78,585 req/s (+32.8%)
- Gap to Vert.x (81,100) reduced from 18% to 3.1%
- Low concurrency: 42,959 -> 54,101 req/s (+25.9%, now 43% faster than Vert.x)

Tests:
- runtime / compile - passed
- Validated with local benchmark (ghz, complex_proto scenario)

References:
None - performance optimization
Motivation:
Flamegraph analysis showed handler lambda creation as a potential
allocation hotspot. For unary gRPC methods, the lambda
(e: Request) => implementation.method(e) was created inside the
handle function which is called per request.

Modification:
Move lambda creation outside the handle function by pre-creating
val handlers for each unary method. The lambdas are now created
once during service registration instead of per request.

Result:
Reduces per-request object allocation by eliminating lambda creation.
Benchmark shows similar performance (within margin of error) as JIT
already optimizes simple lambdas, but this is a good practice for
reducing GC pressure.

Tests:
- codegen / compile - passed
- runtime / compile - passed
- Benchmark verification with ghz (1000 concurrency, 50 connections)

References:
None - performance optimization
Motivation: The remaining gRPC benchmark gap is dominated by small per-request overheads. Native identity strict unary requests still used the generic ByteReader-based strict frame decoder, and the previous unary handler lambda optimization generated invalid Scala for keyword method names and Power API metadata.

Modification: Add a native identity strict frame decoder that parses the gRPC frame header directly while preserving malformed-frame error semantics. Remove unused fast-path response accept-encoding scanning and the dead NativeIdentityGzip tuple. Generate separate safe local handler names for unary Scala handlers, keep pre-created lambdas for non-Power APIs, and keep Power API lambdas request-scoped so metadata is in scope. Remove the accidentally committed Scala handler template backup.

Result: Strict native identity unary decoding avoids the generic ByteReader allocation path. Scala generated handlers compile for keyword method names such as Match, while Power API handlers continue to receive request metadata correctly.

Tests: scalafmt on changed Scala files; sbt plugin-tester-scala / Compile / compile; sbt runtime / Test / testOnly GrpcMarshallingSpec CodecsSpec; sbt benchmarks / Jmh / run quick GrpcMarshalling strict unmarshal benchmarks; git diff --check.

References: None - local performance optimization follow-up from OPTIMIZATION_HANDOFF.md
Motivation:
The unary gRPC hot path allocated unnecessary objects on every request:
1. handleUnary called unaryExceptionHandler which created a new
   PartialFunction via GrpcExceptionHandler.from() on every error path
   (up to 4 allocations per request in worst case)
2. encodeFrameData allocated a separate 5-byte header array plus a
   ByteStrings concatenation wrapper for every frame
3. Async recovery paths eagerly created exception handler PartialFunctions
   even when no exception occurred

Modification:
1. Add handleUnaryException that inlines the exception-to-response
   conversion, avoiding the intermediate PartialFunction allocation
   from GrpcExceptionHandler.from(). Directly checks isDefinedAt and
   falls back to defaultMapper.
2. Optimize encodeFrameData for small messages (<=4KB, covering the
   vast majority of gRPC unary payloads) by combining the 5-byte frame
   header and data into a single array allocation, producing one
   contiguous ArrayByteString instead of a ByteStrings(Header, Data)
   concatenation. Falls back to original approach for large messages.
3. Change async recovery exception handlers from eager val to lazy val
   so the PartialFunction is only allocated if recovery is actually
   needed.

Result:
Fewer object allocations per unary gRPC request on the hot path,
reducing GC pressure under high concurrency. encodeFrameData produces
a single contiguous ByteString for typical small messages, improving
downstream serialization efficiency.

Tests:
sbt runtime / Test / testOnly GrpcMarshallingSpec CodecsSpec GrpcResponseHelpersSpec
All 18 tests passed.

References: None - local performance optimization follow-up from OPTIMIZATION_HANDOFF.md
…llocation

Motivation:
On the unary gRPC strict-entity fast path, decodeIdentitySingleFrame
calls ByteString.slice(5, length) to strip the 5-byte gRPC frame header.
This creates a ByteString1 wrapper object plus intermediate ByteBuffer
wrappers when the sliced bytes reach the protobuf CodedInputStream.
These allocations happen on every request.

Modification:
Add deserialize(data, offset, length) to ProtobufFrameSerializer trait
with a default fallback to slice + deserialize. Override in
ScalapbProtobufSerializer to access the backing byte array directly
with the offset, bypassing all intermediate wrappers. Use this
offset-aware path in GrpcMarshalling.handleUnary for strict entities
when the serializer is a ProtobufFrameSerializer (which all generated
ScalaPB and Java protobuf serializers are).

Result:
Strict-entity unary deserialization avoids the ByteString1 wrapper
allocation from slice and the intermediate ByteBuffer wrappers from
asByteBuffer, going directly from the raw gRPC frame byte array to
CodedInputStream. JMH shows no statistically significant throughput
change in single-threaded benchmarks (within error bars), but the
reduced allocation count benefits GC under high concurrency.

Tests:
sbt runtime / Test / testOnly GrpcMarshallingSpec CodecsSpec GrpcResponseHelpersSpec
All 18 tests passed.

References: None - local performance optimization follow-up from OPTIMIZATION_HANDOFF.md
Motivation:
The Java DSL unmarshal for strict entities used the same
decodeSingleFrame path as before, creating a ByteString.slice wrapper
per request. The Scala DSL already had the offset-aware optimization.

Modification:
Apply the same ProtobufFrameSerializer.deserialize(data, offset, length)
fast path to the Java DSL unmarshal method for strict entities,
bypassing the ByteString.slice allocation.

Result:
Java DSL strict entity unmarshalling now also avoids the ByteString1
wrapper allocation on the hot path, consistent with the Scala DSL.

Tests:
sbt runtime / Test / testOnly GrpcMarshallingSpec
All 7 tests passed.

References: None - follow-up to offset-aware deserialization optimization
…tring

Motivation:
The generated methodName function called request.uri.path.toString on
every request, triggering a full recursive renderPath that allocates a
StringBuilder, renders each path segment, and creates a new String.
This appeared as a hotspot (30 CPU samples) in async-profiler under
high concurrency gRPC workloads.

Modification:
Replace the toString + substring approach with direct Uri.Path pattern
matching. The gRPC path format /{service}/{method} maps to
Path.Slash(Segment(service, Slash(Segment(method, Empty)))). Pattern
matching on this structure extracts the method name with zero string
allocations - only field accesses and a String.equals comparison.

Result:
Eliminates per-request renderPath + StringBuilder + String allocations
in the generated handler. Under complex_proto benchmark (1000c/50conn/
120s), average latency dropped from 7.62ms to 6.88ms (-9.7%) and P99
from 28.56ms to 24.80ms (-13.2%).

Tests:
sbt runtime / Test / testOnly GrpcMarshallingSpec
All tests passed.
sbt codegen / compile
Compiled successfully.

References: None - performance optimization from flamegraph analysis
Motivation:
The negotiated method used Option.map with a pattern-matching lambda,
which allocates a new Function1 object on every request. This appeared
as GrpcMarshalling$$$Lambda (48 CPU samples) in async-profiler.

Modification:
Replace Option.map with direct pattern matching on the negotiate
result. The match on Some/Failure/Success is allocation-free (only
type checks and field accesses via destructuring).

Result:
Eliminates one lambda allocation per gRPC request in the negotiated
method hot path.

Tests:
sbt runtime / Test / testOnly GrpcMarshallingSpec
All 7 tests passed.

References: None - performance optimization from flamegraph analysis
…ndleUnary

Motivation:
The ProtobufFrameSerializer fast path in handleUnary directly called
deserialize(data, offset, length) without checking if the data was
sufficient to contain a frame header. When the request entity was
empty (HttpEntity.empty), data.length - FrameHeaderSize produced a
negative length (-5), bypassing the MissingParameterException that
reader.decodeSingleFrame would normally throw. This caused the
ErrorReportingSpec test "should respond with an 'invalid argument'
gRPC error status when calling an method without a request body"
to fail because the expected INVALID_ARGUMENT status was not returned.

Modification:
Add a length check before the ProtobufFrameSerializer.deserialize call:
if data.length < FrameHeaderSize, throw MissingParameterException to
match the behavior of reader.decodeSingleFrame which validates the
frame header is present before attempting to parse it.

Result:
Empty request bodies now correctly return grpc-status: 3
(INVALID_ARGUMENT) regardless of whether the ProtobufFrameSerializer
fast path or the standard decodeSingleFrame path is used.

Tests:
sbt plugin-tester-scala / Test / testOnly example.myapp.helloworld.ErrorReportingSpec
All 2 tests passed.

References: Fixes CI failure in PR apache#739
Motivation:
`eq` is reference equality in Scala — it only returns true when both
operands are the exact same object. `mediaType.subType` is not
guaranteed to return interned strings, so `eq` can silently return
false for semantically equal values like "grpc+proto".

Modification:
Replace `eq` with `==` (value equality) in the `negotiate` fast path.

Result:
Content-type negotiation works correctly regardless of string interning.
@He-Pin He-Pin force-pushed the optimize/fast-negotiation-uri-match branch from 1054ad2 to 75444a2 Compare June 23, 2026 17:02
@He-Pin He-Pin requested a review from pjfanning June 23, 2026 17:11
@He-Pin He-Pin changed the title Optimize gRPC request processing: fast negotiation path, Identity codec, and reduced per-request allocations Optimize gRPC request processing: fast negotiation path and reduced per-request allocations Jun 23, 2026
He-Pin added 2 commits June 24, 2026 01:17
Motivation:
Review identified redundant method calls, dead code branches, and
a missing safety check in the strict deserialization path.

Modification:
- GrpcProtocolNative: inline compressed-bit check and use ByteString.drop
  instead of Identity.uncompress + slice, eliminating a method call and
  redundant bounds checking on the identity decode hot path
- GrpcMarshalling (scaladsl): merge duplicate catch branches
  (MissingParameterException is already matched by NonFatal), reducing
  bytecode branch count for cleaner JIT compilation
- GrpcMarshalling (javadsl): add frame header length check before
  offset-aware deserialization in strict path, matching scaladsl behavior
- Handler template: remove dead if(powerApis) conditional inside
  @if(!powerApis) block in pre-created lambda definitions

Result:
Fewer allocations and method calls on the hot path, cleaner bytecode
for JIT, and consistent error handling between Java and Scala DSLs.
@He-Pin

He-Pin commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

@pjfanning I have removed the gzip change from this PR

He-Pin added 2 commits June 24, 2026 01:39
…review findings

Motivation:
The negotiate() fast path hardcoded writerCodec to Identity, ignoring
the client's grpc-accept-encoding header. This was a behavioral
regression from the original Codecs.negotiate() which respected client
compression preferences.

Modification:
- negotiate() now scans grpc-accept-encoding and selects writer codec
  respecting client preference order (matching Codecs.negotiate behavior)
- Separate negative frame length check from trailing data check with
  clearer error messages in decodeIdentitySingleFrame
- Fix misleading Codecs.supportedCodecs comment: order does not affect
  codec selection, client preference does
- Add GrpcProtocolSpec with tests for negotiate fast path behavior
…Java API fallback

Motivation:
Second review identified that the ProtobufFrameSerializer fast path in
handleUnary skipped all frame header validation that decodeIdentitySingleFrame
carefully performs. A test assertion was also incorrect, and the negotiate
fast path did not handle pure Java API requests.

Modification:
- Add compressed-flag check, frame length validation, and declared-length
  bounds checking to ProtobufFrameSerializer fast path in both scaladsl
  and javadsl handleUnary, matching decodeIdentitySingleFrame validation
- Fix GrpcProtocolSpec test: unknown grpc-encoding returns failed reader,
  not None (matching Codecs.detect behavior)
- negotiate() fast path delegates to slowNegotiateOpt for non-Scala-API
  requests, preserving Codecs.extractHeaders Java API header handling

Result:
Frame validation is consistent between fast path and standard path.
Pure Java API requests are handled correctly through the slow path.
val supportedCodecs = immutable.Seq(Gzip, Identity)
// Codecs supported by this server. The selection order is determined by the
// client's grpc-accept-encoding header preference, not by this sequence order.
val supportedCodecs = immutable.Seq(Identity, Gzip)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this gzip change be handled only in the new separate PR?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will rebate, I think I will continues it this weekends


override protected def reader(codec: Codec): GrpcProtocolReader =
AbstractGrpcProtocol.reader(codec, decodeFrame)
if (codec eq Identity)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this identity codec change, can it be done in the gzip PR instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants