Skip to content

fix(triple): release server DATA discarded before decoder handoff - #16585

Merged
EarthChen merged 4 commits into
apache:3.3from
chaoyoung:codex/triple-server-input-ownership
Nov 18, 2026
Merged

EarthChen merged 4 commits into
apache:3.3from
chaoyoung:codex/triple-server-input-ownership

Conversation

@chaoyoung

@chaoyoung chaoyoung commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #16584

What changes were proposed in this pull request?

Triple server DATA rejected or silently discarded before decoder execution can retain its reference-counted input even after stream closure. Register close-once input wrappers before executor submission and route explicit rejection through the existing message error cleanup.

The final design uses a registry, a volatile terminal flag and one atomic state per input: QUEUED -> DELIVERING -> HANDED_OFF, or CLOSED. Close/remote cancellation claims only QUEUED inputs. A started callback closes its own DELIVERING input in finally if it never handed off. At handoff, registry ownership ends before the message listener runs, so decoder-owned buffered input is not closed by the terminal drain. Closing the original message is guarded once. Registration rechecks the terminal flag after insertion to cover a concurrent drain.

The shared lifecycle lock and in-flight counter have been removed. The class javadoc explains the ownership transitions and the registration race. Original frame id, end-stream status and original-message close behavior are retained. The FinalFragment-preserving adapter stays in the WebSocket listener; the generic HTTP/gRPC listener has no optional WebSocket dependency. SerializingExecutor behavior and unrelated task APIs are unchanged.

Why are the changes needed?

A decode task is not a reliable owner until it runs. Saturation may reject it, while backing-executor shutdown may silently discard it. Decoder cleanup cannot release input it never received; a submission-site catch alone does not cover silent discard or queued input.

How was this patch tested?

Base: 3.3 at 1cf125c. The initial unchanged-base regression failed four of five tests with an unreleased reference; the accepted-partial control passed. Subsequent human review independently reproduced the expanded fixture failures and confirmed the fix.

Before the per-input refactor, the current 18 ownership tests passed. Five additional invocations brought the fixture to 23: two terminal-close/remote-cancel cases failed because an unrelated queued body waited for the preparing callback; the error-handler failure, preparation failure and registration-versus-close controls passed. After the refactor, all 23 pass. The original buffered-A/read-in-B cancellation, accepted-partial, handoff-before-decode, close-exception, WebSocket-fragment and exactly-once controls remain.

mvn -q -pl dubbo-rpc/dubbo-rpc-triple -am -Pjacoco \
  '-Dtest=org.apache.dubbo.rpc.protocol.tri.**.*Test,org.apache.dubbo.rpc.TriRpcStatusTest,org.apache.dubbo.rpc.StatusRpcExceptionTest,org.apache.dubbo.rpc.stub.StubInvocationUtilTest' \
  -Dsurefire.failIfNoSpecifiedTests=false -Dspotless.action=check test jacoco:report
mvn -q -pl dubbo-rpc/dubbo-rpc-triple -am -DskipTests -Dspotless.action=check validate
git diff --check

Before pushing c698bf4: Triple protocol package 491 tests, zero failures/errors, two existing disabled WriteQueueTest methods; ownership 23/23 pass. Formatting, whitespace checks and independent code review pass. Real Netty/Tomcat WebSocket sample clients pass 16 tests each on JDK 21 and JDK 25. Those sample runs use checkout core classes with released 3.3.6 Spring Boot bootstrap dependencies and no registry; they do not replace the upstream Nacos/dependency matrix. No snapshot artifacts were installed locally.

Local JaCoCo covers all added executable lines in the generic listener and WebSocket adapter. The base default onDataReceived forwarding line is unexercised, and two short-circuit conditions remain partially branch-covered; full patch branch coverage is not claimed. Updated-head GitHub CI is separate from these local results.

A final test-inventory audit found three module test classes outside the protocol package. The selector above was extended to include TriRpcStatusTest, StatusRpcExceptionTest and StubInvocationUtilTest. This additional complete-module verification, run after the push without further code changes, passes 518 tests with zero failures/errors and the same two existing disabled methods. Every annotated test class in the module has a corresponding report; the earlier 491 count refers specifically to the protocol-package run.

Scope

The ownership fixture uses real executors, a gRPC streaming decoder and response-observer lifecycle, with application routing isolated and an H2 channel test double. It is not a full TCP gRPC/load benchmark. Accepted input retains the existing decoder lifecycle, including decoder-specific termination/handoff races; this PR does not synchronize all decoder read/close behavior. Client timeout handling (#16415 / #16418), general executor redesign and HTTP/3 validation are separate scope. No configuration change is required.

AI assistance

AI assistance was used for the implementation, tests, independent review and description. The protocol-package and WebSocket checks were run before pushing; the additional complete-module check is identified above.

@codecov-commenter

codecov-commenter commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.65217% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.14%. Comparing base (1cf125c) to head (c698bf4).
⚠️ Report is 3 commits behind head on 3.3.

Files with missing lines Patch % Lines
...h12/http2/GenericHttp2ServerTransportListener.java 95.65% 0 Missing and 2 partials ⚠️
...tocol/tri/h12/AbstractServerTransportListener.java 93.75% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##                3.3   #16585      +/-   ##
============================================
+ Coverage     60.99%   61.14%   +0.14%     
- Complexity    11756    11762       +6     
============================================
  Files          1953     1953              
  Lines         89286    89348      +62     
  Branches      13478    13486       +8     
============================================
+ Hits          54462    54628     +166     
+ Misses        29220    29103     -117     
- Partials       5604     5617      +13     
Flag Coverage Δ
integration-tests-java21 32.10% <57.97%> (+<0.01%) ⬆️
integration-tests-java8 32.23% <57.97%> (+0.01%) ⬆️
samples-tests-java21 32.10% <0.00%> (-0.02%) ⬇️
samples-tests-java8 29.74% <0.00%> (-0.07%) ⬇️
unit-tests-java11 59.42% <95.65%> (+0.16%) ⬆️
unit-tests-java17 58.86% <95.65%> (+0.16%) ⬆️
unit-tests-java21 58.90% <95.65%> (+0.18%) ⬆️
unit-tests-java25 58.82% <95.65%> (+0.13%) ⬆️
unit-tests-java8 59.40% <95.58%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chaoyoung

chaoyoung commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

WebSocket regression fixed and locally revalidated

Commit 2abcce2 fixes the first CI run's real WebSocket regression: the pending-input wrapper hid FinalFragment metadata. The marker-preserving adapter now lives in the WebSocket listener, so the generic HTTP/gRPC listener does not acquire a mandatory dependency on the optional WebSocket module. Original-message close-once ownership is unchanged.

Before pushing this update, I ran:

  • Two new fragment regressions: original PR fails; corrected implementation passes. Added a close-IOException/exact-once cleanup test.
  • Full Triple package: 483 tests, zero failures/errors, two pre-existing @Disabled WriteQueueTest cases.
  • Real Netty and Tomcat WebSocket sample: original PR 16/16 failures; corrected implementation 16/16 passing on both JDK 21 and JDK 25. These local runs use no registry, checkout core classes, and released 3.3.6 Spring Boot bootstrap dependencies. They do not reproduce the exact CI Nacos/dependency matrix.
  • Local JaCoCo covers all added executable lines in GenericHttp2ServerTransportListener and the WebSocket adapter; three short-circuit condition lines remain partially branch-covered. This is not a claim of full PR branch coverage.
  • Spotless, git diff whitespace check, and independent review pass. No snapshot artifacts were installed into the local Maven repository.

Updated GitHub CI remains pending; the previous failed run must not be treated as passed.

@zrlw
zrlw requested a balanced review from Copilot November 16, 2026 09:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

4 open issues
What changed in this PR

Fixes server-side HTTP/2 DATA body ownership so reference-counted inputs aren’t leaked when executor submission is rejected or discarded before decoder handoff.

Changes:

  • Add transport-level pending-input registry and lifecycle gating to release DATA bodies on close/cancel before decoder takes ownership.
  • Extend server transport listener flow with onDataReceived(...) / onDataMessage(...) hooks and explicit RejectedExecutionException handling.
  • Add comprehensive ownership/concurrency regression tests, plus WebSocket final-fragment coverage.
File Description
dubbo-rpc/​dubbo-rpc-triple/​src/​main/​java/​org/​apache/​dubbo/​rpc/​protocol/​tri/​h12/​AbstractServerTransportListener.java Adds onDataReceived/onDataMessage hooks and catches executor rejection for DATA delivery.
dubbo-rpc/​dubbo-rpc-triple/​src/​main/​java/​org/​apache/​dubbo/​rpc/​protocol/​tri/​h12/​http2/​GenericHttp2ServerTransportListener.java Implements pending-input ownership registry and transport-driven cleanup on close/cancel.
dubbo-rpc/​dubbo-rpc-triple/​src/​main/​java/​org/​apache/​dubbo/​rpc/​protocol/​tri/​websocket/​DefaultWebSocketServerTransportListener.java Wraps FinalFragment inputs so fragment metadata survives pending-input wrapping.
dubbo-rpc/​dubbo-rpc-triple/​src/​test/​java/​org/​apache/​dubbo/​rpc/​protocol/​tri/​h12/​http2/​GenericHttp2ServerTransportListenerTest.java New regression suite validating ref-count ownership across rejection/discard/close/cancel and WebSocket fragments.

Review effort: Lite

}
});
} catch (RejectedExecutionException e) {
onError(message, e);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in f643263. The rejection path now invokes onDataFinally(message) in a finally block around onError(message, e), so finalization is preserved even if the error handler throws. The rejection regression failed with 0 finalizations before the change and now asserts exactly 1, alongside exactly-once body release. The full Triple test package and real WebSocket checks passed before pushing; details are in the validation follow-up.

Comment on lines +241 to +249
synchronized (inputLock) {
if (inputClosed) {
return;
}
// The decoder owns accepted input, including any bytes buffered for later requests.
// Only inputs that have not crossed this boundary belong to the transport registry.
pendingInputs.remove((PendingInput) message.getBody());
}
super.onDataMessage(message, listener);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added listenerFailureBeforeConsumptionReleasesInputOnce to check this path directly. It also passes on the previous implementation: a callback that throws before consuming the body reaches the existing onError(message, ...) cleanup and releases the original body exactly once. I am retaining removal before the callback. The decoder can already own buffered input while the callback is still running; delaying removal until callback success would let concurrent transport cancellation close decoder-owned input. The accepted-partial and read-in-progress cancellation controls cover that distinction. The registry therefore tracks only inputs that have not been handed to the message listener, rather than claiming ownership until a successful callback return.

Comment on lines +300 to +315
private void closeInputs() {
synchronized (inputLock) {
inputClosed = true;
if (processingInputs == 0) {
closePendingInputs();
}
}
}

private void closePendingInputs() {
// An already-started callback may still be preparing its DATA message. Defer
// pre-delivery cleanup until it returns; accepted input belongs to the decoder.
for (PendingInput input : pendingInputs) {
closeInput(input);
}
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in f643263. Pending inputs are now detached into a snapshot under inputLock and closed after releasing the lock. Cleanup still waits until all started DATA callbacks leave the handoff, so it does not close a body being prepared for the decoder. The new closeCallbackCanWaitForConcurrentLateData regression times out on the prior lock-held implementation and passes with this change. I also added a red/green regression for unchecked close failures: each input is closed independently, so a RuntimeException from one close does not abandon the remaining detached inputs. Repeated close still releases each original body only once.

Comment on lines +102 to +119
@Override
protected PendingInput newPendingInput(Http2InputMessage message) {
return message.getBody() instanceof org.apache.dubbo.remoting.websocket.FinalFragment
? new PendingFragmentInput(message)
: super.newPendingInput(message);
}

private final class PendingFragmentInput extends PendingInput
implements org.apache.dubbo.remoting.websocket.FinalFragment {
private PendingFragmentInput(Http2InputMessage message) {
super(message);
}

@Override
public boolean isFinalFragment() {
return ((org.apache.dubbo.remoting.websocket.FinalFragment) in).isFinalFragment();
}
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in f643263: FinalFragment is imported once and the repeated fully qualified references are removed. The marker-preserving adapter remains confined to the WebSocket-specific listener; the generic HTTP/gRPC listener still has no dependency on the optional WebSocket module. Formatting checks and the real Netty/Tomcat WebSocket sample tests passed before pushing.

@chaoyoung

Copy link
Copy Markdown
Contributor Author

Follow-up in f643263 addresses the review comments after local validation:

  • Finalize rejected DATA in a finally block around error handling.
  • Detach pending inputs under the lifecycle lock, then close outside it. Keep the existing deferral while DATA callbacks are handing input to the decoder. Isolate per-input close exceptions so one unchecked failure cannot abandon the remaining snapshot.
  • Use the FinalFragment import in the WebSocket adapter.
  • Retain handoff-before-callback ownership: a new pre-consumption callback-failure control already passes on the previous code through existing error cleanup. Moving removal after callback success would let transport cleanup close decoder-owned buffered input during cancellation. The inline reply explains the boundary.

Before pushing, the rejection-finalization and lock-held-close tests reproduced their failures; the snapshot close-failure test also failed before the extra fix. Final validation on OpenJDK 25:

mvn -q -pl dubbo-rpc/dubbo-rpc-triple -am -Pjacoco \
  '-Dtest=org.apache.dubbo.rpc.protocol.tri.**.*Test,GenericHttp2ServerTransportListenerTest' \
  -Dsurefire.failIfNoSpecifiedTests=false -Dspotless.action=apply test jacoco:report
mvn -q -pl dubbo-rpc/dubbo-rpc-triple -am -DskipTests -Dspotless.action=check validate
git diff --check

Triple reports 486 tests, 0 failures, 0 errors, 2 existing disabled WriteQueueTest methods; the ownership test class has 18 passing tests. Formatting and whitespace checks passed. Independent review found no remaining blocking issue.

The real dubbo-samples Triple WebSocket Netty/Tomcat clients also pass 16 tests each on JDK 21 and JDK 25, rerun after the final fix. These runs use the checkout core classes with released 3.3.6 Spring Boot bootstrap dependencies and no registry; they do not replace the upstream Nacos/dependency matrix or whole-repository CI. The previous head CI was green; new-head CI results are separate from these local checks.

All four review threads have replies and remain available for reviewer confirmation. Could a maintainer review this update when available? AI assistance was used for the follow-up and validation.

@EarthChen EarthChen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed report and the thorough test suite. I reproduced the leak on the base commit 1cf125c421 (8 of 18 tests fail with refcount 1) and confirmed the fix turns them all green, plus a full dubbo-rpc-triple module run.

Before this moves to a final review, I'd like to explore whether the ownership machinery can be simplified. As written it is hard to review: a registry set, a lifecycle lock, a closed flag and an in-flight counter interact across four methods, and it is not obvious from the code which parts are load-bearing.

One data point: grpc-java has the same class of bug (grpc/grpc-java#12890, fix PR grpc/grpc-java#12891, still unmerged — the maintainers want it extended into the full failure-notification design tracked in grpc/grpc-java#636). Their fix is a per-call-site catch (RejectedExecutionException) that closes the message producer and rethrows. That is sufficient there because (a) their SerializingExecutor always rethrows — the silent shutdown-drop is a dubbo-specific addition from #15122 — and (b) their deframer runs on the transport thread, so the executor task only ever holds one assembled message. Dubbo decodes inside the executor task, so a submission-site catch cannot cover the queued-data windows (I verified the rejection-only variant still fails 7 of these 18 tests); some registry remains necessary here.

That said, I prototyped a leaner variant that passes this same 18-test suite plus a full module run: replace the lock and the counter with a per-input state machine (QUEUED → DELIVERING → HANDED_OFF / CLOSED, one AtomicInteger per input, no shared lock):

stateDiagram-v2
    [*] --> QUEUED: onDataReceived() registers input
    QUEUED --> DELIVERING: task starts (CAS)
    QUEUED --> CLOSED: close() / cancelByRemote()
    DELIVERING --> HANDED_OFF: handoff to decoder (CAS)
    DELIVERING --> CLOSED: task finally, not handed off
    HANDED_OFF --> [*]: decoder owns input, transport never touches it
    CLOSED --> [*]: released once
Loading
close() / cancelByRemote()
  inputClosed = true
  for each input still QUEUED: CAS -> CLOSED, close body once

executor task starts       : CAS QUEUED -> DELIVERING      (lose -> input was closed, skip)
handoff to message listener: CAS DELIVERING -> HANDED_OFF, leave registry
task finally               : CAS DELIVERING -> CLOSED      (lose -> decoder or drain owns it)

The line count is about the same, but the concurrency reasoning becomes local to each input, and the event-loop path no longer takes any lock. One behavioral nuance: the global lock in the current design guarantees that no delivery can start after close() returns; the per-input version has a benign race where a delivery that already passed its checks completes into a closing stream — the same behavior as before this PR, and the decoder/stream teardown releases it.

Could you either consider this direction, or add a short design note (class javadoc) explaining why the registry, the lock, the closed flag and the counter each need to exist? As a reviewer it is currently hard to tell which parts are load-bearing.

(For context: the rejection-only #16588 was closed in favor of this PR.)

@chaoyoung

chaoyoung commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor Author

Following the design feedback in #16585 (review), c698bf4 adopts the per-input state approach. The registry remains necessary for silent discard, while the shared lifecycle lock and in-flight counter are gone. Class javadoc now explains QUEUED / DELIVERING / HANDED_OFF / CLOSED ownership and registration racing with the terminal drain.

Terminal cleanup claims only QUEUED inputs. Each started callback closes its own still-DELIVERING input in finally. Handoff removes transport ownership before invoking the listener; decoder-owned buffered inputs and original-message close-once behavior are preserved. A handoff that already passed its terminal check retains the existing decoder lifecycle; this update does not claim to fix every decoder-specific termination race.

Before the refactor, the expanded 23-test fixture had exactly two failures: queued DATA was not released promptly on close/cancel while another callback was preparing. Afterward all 23 pass, including the original buffered-input/read-in-progress and post-handoff controls, plus registration-versus-close, preparation failure and error-handler failure. Before pushing, the Triple protocol package ran 491 tests with 0 failures/errors and 2 existing disabled methods. A final inventory audit then added TriRpcStatusTest, StatusRpcExceptionTest and StubInvocationUtilTest, which sit outside that package: complete-module verification after the push passes 518 tests, 0 failures/errors, with the same 2 disabled methods and no further code changes. Every annotated module test class has a corresponding report. Real Netty/Tomcat WebSocket clients pass 16 tests each on JDK 21 and 25 with the same disclosed no-registry/sample-bootstrap setup. Formatting, whitespace checks and independent review passed before pushing.

The PR body now describes the final design and test scope. Local coverage covers the added generic-listener executable lines; one default base-hook forwarding line and two partial short-circuit conditions remain disclosed rather than claiming 100% branch coverage. New-head CI will be checked separately. Please review this update when available.

@EarthChen EarthChen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified on base 1cf125c421 vs head c698bf42e2 (JDK 17):

  • Base + this PR's test suite (only adapted to drop the newPendingInput hook that does not exist on base): Tests run: 23, Failures: 11 — exactly the claimed symptom: bodies still at refCnt 1 after executor rejection, silent discard by SerializingExecutor on a shut-down executor, queued bodies not released on close/cancel while another callback is preparing, and loss of FinalFragment metadata after wrapping. Red at base, green at head (23/23); full dubbo-rpc-triple run: 518 tests, 0 failures, 2 pre-existing @Disabled.
  • Mechanism confirmed at the receiving call site: AbstractServerTransportListener.onData closes the body only inside the executor task, so a RejectedExecutionException at submission — or the silent task drop in SerializingExecutor.schedule when the backing executor is shut down (introduced by #15122) — leaves the refcounted ByteBufInputStream body (over frame.content() in the HTTP/2 and HTTP/3 codecs) unreleased.
  • The per-input QUEUED/DELIVERING/HANDED_OFF/CLOSED design addresses the earlier review feedback: no shared lifecycle lock on the event-loop path, terminal cleanup claims only QUEUED inputs, and each started callback closes its own DELIVERING input in finally; close-once is enforced by getAndSet(CLOSED)/CAS before closing the original message. The remaining race windows are disclosed in the class javadoc and match pre-PR behavior.
  • Submission-site RejectedExecutionException handling now finalizes DATA via onError + onDataFinally; for the generic HTTP/2 listener this converts a raw REE propagation into a stream-level error response, consistent with the existing in-task error path.
  • Sibling sites: HTTP/1 and servlet DATA use non-refcounted byte-array bodies and inherit the rejection catch through the base class without needing the registry; the WebSocket adapter preserves FinalFragment so fragment accumulation keeps working. Client-side DATA handling is out of scope here and could be a follow-up if a similar window exists.

Looks good overall — approving.

@EarthChen
EarthChen merged commit ae8b2cb into apache:3.3 Nov 18, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Triple server leaks inbound DATA when decode tasks are rejected or silently discarded

4 participants