Skip to content

[04/36] Add OC-090 gRPC streaming support - #38

Open
cjohnsto-nz wants to merge 3 commits into
feature/oc-030-grpc-unary-protobuffrom
feature/oc-090-grpc-streaming
Open

[04/36] Add OC-090 gRPC streaming support#38
cjohnsto-nz wants to merge 3 commits into
feature/oc-030-grpc-unary-protobuffrom
feature/oc-090-grpc-streaming

Conversation

@cjohnsto-nz

@cjohnsto-nz cjohnsto-nz commented Jun 14, 2026

Copy link
Copy Markdown
Owner

@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-030-grpc-unary-protobuf branch from 5504fbe to fdeadfe Compare June 14, 2026 22:45
@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-090-grpc-streaming branch from 6aa428f to 0815220 Compare June 14, 2026 22:45
@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-030-grpc-unary-protobuf branch from fdeadfe to 6cb577c Compare June 14, 2026 22:49
@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-090-grpc-streaming branch from 0815220 to 8afca3e Compare June 14, 2026 22:49
@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-030-grpc-unary-protobuf branch from 6cb577c to 2c3d58a Compare June 14, 2026 22:49
@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-090-grpc-streaming branch from 8afca3e to de9dc3e Compare June 14, 2026 22:50
@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-030-grpc-unary-protobuf branch from 2c3d58a to c5dbae7 Compare June 14, 2026 22:50
@cjohnsto-nz
cjohnsto-nz force-pushed the feature/oc-090-grpc-streaming branch from de9dc3e to 06b8d83 Compare June 14, 2026 22:50
@cjohnsto-nz cjohnsto-nz changed the title feature/oc-090-grpc-streaming [04/21] Add OC-090 gRPC streaming support Jun 14, 2026
@cjohnsto-nz cjohnsto-nz changed the title [04/21] Add OC-090 gRPC streaming support [04/27] Add OC-090 gRPC streaming support Jun 15, 2026
@cjohnsto-nz cjohnsto-nz changed the title [04/27] Add OC-090 gRPC streaming support [04/36] Add OC-090 gRPC streaming support Jun 15, 2026
@APKiwi

APKiwi commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Verdict: approve with nits. Stream lifecycle (settle guard, cancel, cleanup on every exit, partial-data-on-error) is done carefully, no crash or leak found. Two design items to track before this path gets wired to the UI (it isn't yet, nothing constructs GrpcClient outside tests at this point in the stack):

  • Major: streaming inherits the 30s unary timeout as a hard deadline. All three _execute*Streaming methods pass deadline: Date.now() + missio.timeout. Long-lived streams, the main reason streaming exists (subscriptions, chat), get force-killed at 30s with DEADLINE_EXCEEDED. The deadline needs to be optional or configurable for streaming methods.
  • Major: unbounded buffering with 2x duplication, response only at stream end. receivedMessages and events grow for the stream's lifetime with no cap, each message is stored twice (once in the arrays, again inside events[].message), and _buildStreamResponse JSON.stringifies all of it with no size guard. A high-rate or infinite stream grows host memory without bound and never returns until cancelled. A message cap plus dropping the duplicate payload from events would bound it.
  • Minor: onProgress fires per received message unthrottled. Once wired to the webview a fast stream floods the channel, coalesce it.
  • Minor: client-streaming pushes a duplicate error event (both the unary callback and the call.on('error') listener record one before settle).
  • Minor: GrpcRequestDetails.message in models/types.ts:456 was not extended with the new sequence shape, so the code compensates with unknown params and as any throughout the tests. Extend the union.
  • Minor: GrpcStreamingUnsupportedError is now dead code, remove it.
  • Minor: bidi writes all client messages synchronously then ends, so it's not interactive and call.write() backpressure is ignored. Fine for static YAML arrays, worth a comment. The if (activeCall.cancelled) break inside those sync loops is unreachable.
  • Nit: removeAllListeners() with no event name also strips grpc-js internals (harmless before close, but name the events). _isMessageSequence and unresolvedVars' isSequence implement the same check with different strictness.

Tests: solid increment (all three modes, partial data + error, mismatch, cancel cleanup, Ajv round-trip). Gaps: cancel and mid-stream error only tested for server-streaming, no deadline test, and the cancelAll assertion is partly tautological since it clears the map synchronously.

Deps: none added.

@cjohnsto-nz

Copy link
Copy Markdown
Owner Author

Fixed the branch-owned findings in commit 5b5164f:

  • Extended GrpcRequestDetails.message with the schema-native GrpcRequestMessage[] sequence type and removed the resulting test casts.
  • Prevented grpc-js's callback and error event from recording the same client-streaming failure twice.
  • Removed the dead GrpcStreamingUnsupportedError.
  • Added automated coverage proving streaming calls use missio.timeout and preserve DEADLINE_EXCEEDED diagnostics.

The remaining design items do not require OC-090 changes:

  • Streaming deadline: the hard deadline is deliberately controlled by the configurable missio.timeout setting. OC-090 implements finite YAML-defined streaming executions, not indefinite interactive subscriptions.
  • Transcript buffering: retaining sent/received events and partial history is the response contract required by OC-090. These executions are finite request definitions with cancellation, not unbounded session objects, so no cap or duplicate-payload removal is being introduced.
  • Progress notifications: per-message progress corresponds to those finite executions; no throttling layer is required for this task.
  • Client/bidi writes: the schema defines an ordered static message batch which is written and ended. Interactive writes and backpressure-driven sessions are outside this request model.
  • Listener cleanup: listener removal occurs only during terminal cleanup immediately before closing the client.
  • Sequence checks: execution validation intentionally rejects malformed message sequences, while unresolved-variable scanning tolerates partially authored input. Their strictness is different by design.
  • Runtime wiring: shared extension composition is implemented later in PR [15/36] Add OC-050 auth and transport support #49, commit 06e680c, and is not duplicated here.

Validation after the fix: focused gRPC tests pass 15/15, the full suite passes 348/348, GitHub build/security checks pass, and PR #39 remains MERGEABLE / CLEAN on top of this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants