Release results that are discarded after cancellation - #731
Open
agorbachenko wants to merge 1 commit into
Open
Conversation
agorbachenko
marked this pull request as draft
July 26, 2026 10:50
agorbachenko
force-pushed
the
fix/cancelled-row-stream-wedges-connection
branch
from
July 26, 2026 10:54
e82c6a2 to
46e47be
Compare
`Operators.discardOnCancel` drops results that arrive after the result stream was cancelled. `PostgresqlResult` is reference-counted and drains its remaining messages when released, so a discarded result that is not released leaves its messages unconsumed. The `Conversation` stays at the head of the conversation queue without demand, the client stops requesting from the transport, and neither this nor any subsequent conversation on that connection can complete. Pooled connections are handed back in that state, so the next borrower waits forever for a response that the server has already sent. The discard context propagates upstream only, so the existing handler above the mapping is not visible to `discardOnCancel`. Moving it below `discardOnCancel` covers both the rows discarded within `WindowPredicate` and the discarded results. Registering a second handler instead would compose the two (`Operators.discardLocalAdapter` chains them) and release the rows discarded upstream twice. [resolves pgjdbc#661]
agorbachenko
force-pushed
the
fix/cancelled-row-stream-wedges-connection
branch
from
July 26, 2026 11:17
46e47be to
a28cb98
Compare
agorbachenko
marked this pull request as ready for review
July 26, 2026 12:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make sure that:
Issue description
Resolves #661, and the connection leak reported in r2dbc/r2dbc-pool#198.
PostgresqlStatement.execute()decorates theFlux<PostgresqlResult>withOperators.discardOnCancel. After a cancellation that operator keeps the subscription and drops any further result throughOperators.onDiscard.PostgresqlResultis reference-counted and drains its remaining messages indeallocate(), so a discarded result has to be released to be drained.The
doOnDiscard(ReferenceCounted.class, ReferenceCountUtil::release)that is already present is registered above themap(…), so it is not part ofdiscardOnCancel's subscriber context and never applies to the mapped results. A result that is cancelled before it is consumed is therefore dropped without being released, and itswindowUntilwindow is never drained.From that point the connection is unusable:
Conversationstays at the head ofBackendMessageSubscriber.conversationswithdemand == 0,drainLoop()leaves at the head because the head has no demand (intentional, High CPU usage when cancelled query #242),demandMore()cannot request from the transport whilehasBufferedItems()istrue,State of the message subscriber once wedged, read from a running reproducer:
Note
sinkCancelled=false: the client never observes a cancellation, it only sees a consumer that stopped requesting. That is why the investigations in r2dbc/r2dbc-pool#198 could not pin this on reactor-pool or onusingWhen— at or below the client this is indistinguishable from a slow consumer, so no change keyed on cancellation insideReactorNettyClientcould have fixed it.With a pool the symptom is a leaked connection: the cancelling borrower releases it, and the next borrower's query never completes even though
pg_stat_activityshows the backendidlewith the response fully sent.The handler is moved rather than duplicated.
Operators.discardLocalAdaptercomposes handlers (safeConsumer.andThen(consumer)), so registering a second one would give every operator upstream of both registrations both handlers, and the rowsFluxWindowPredicatediscards would be released twice. Measured over the 940 pre-existing tests inPostgresCancelIntegrationTests: 0Error in discard hook/IllegalReferenceCountExceptionunpatched, 80 with a second handler added, 0 with the handler moved. Since the discard context propagates upstream only, one registration belowdiscardOnCancelcovers both concerns —WindowPredicatestill sees it, and nowdiscardOnCanceldoes too.New Public APIs
None.
Additional context
The new test,
PostgresCancelIntegrationTests#shouldReuseConnectionWhenResultIsDiscardedAfterCancel, cancels a streaming response without consuming the result and then reuses the connection. It is deterministic — no pool and no race involved: 5/5 repetitions fail before the change (the second query times out), 5/5 pass after.Also verified:
-Dtest='*IntegrationTests': 1267 tests, 0 failures, 11 skipped, including the 946 existing cancellation repetitions,zipWitha failingMono), upgraded to Spring Boot 4.0.7 / reactor-core 3.8.6 / reactor-pool 1.2.6 / Java 25: wedges after about three queries on both 1.1.2.RELEASE andmain, and survives 127 queries with no stall and no leak reports with this change,main, and passes 150 iterations × 2 response sizes × 2 scenarios with this change.Notes:
flatMapdeadlock discussed at the end of ReactorNettyClient stucked on cancelled Conversation if that Conversation has more than 256 rows (size of reactor.bufferSize.small) #661. That one is a consumer that never requests; this one is a cancellation that loses the obligation to drain.FluxDiscardOnCancelSubscriber.onNextcould releaseReferenceCounteditems it drops instead of relying on a context handler. I went with the smaller, local change.windowUntilusage,DefaultPostgresqlReplicationConnection.startReplication(), is not affected. That chain has nodiscardOnCancel, so a cancellation propagates to the exchange normally and the conversation terminates; andPostgresReplicationStreamis not reference-counted, so there is nothing a discard handler could release.LogicalDecodeIntegrationTestspasses with-Dio.netty.leakDetection.level=paranoidand no discard-hook errors.