Skip to content

[Rust] Add persistent stream proto - #761

Merged
elenagaljak-db merged 1 commit into
mainfrom
stack/eos-proto
Aug 24, 2026
Merged

[Rust] Add persistent stream proto#761
elenagaljak-db merged 1 commit into
mainfrom
stack/eos-proto

Conversation

@elenagaljak-db

@elenagaljak-db elenagaljak-db commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

What changes are proposed in this pull request?

This PR adds the in-development protobuf contract used by persistent Zerobus ingestion streams. It contains only the wire-level API.

The contract adds:

  • PersistentStream, a bidirectional RPC whose first message creates a durable stream or resumes one by stream_id.
  • ResumeIngestStreamRequest, including the protobuf descriptor and required record_type used to validate a resumed stream against its original configuration.
  • ResumeIngestStreamResponse.last_committed_offset, the server watermark from which the client continues at last_committed_offset + 1.
  • Persistent envelopes for create, resume, ingestion, acknowledgements, and server close signals.
  • The server-side RetireStream contract; a public Rust retirement API is intentionally deferred.

The shapes and field numbers match the current Shinkansen server contract. The PR also records this API-contract addition in rust/NEXT_CHANGELOG.md.

Stack

  1. [Rust] Add persistent stream proto #761 — protobuf contract
  2. [Rust] Add persistent gRPC transport #762 — gRPC transport and recovery engine
  3. [Rust] Expose persistent stream API #763 — public Rust API
  4. [Rust] Test persistent stream resume #764 — stateful mock and integration tests
  5. [Rust] Document persistent streams #765 — documentation and runnable example

How is this tested?

The protobuf is compiled by the Rust SDK build. The complete create, ingest, flush, and resume flow is exercised in #764.

Signed-off-by: elenagaljak-db <elena.galjak@databricks.com>
@elenagaljak-db
elenagaljak-db requested a review from a team August 20, 2026 14:51

@teodordelibasic-db teodordelibasic-db left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elenagaljak-db
elenagaljak-db added this pull request to the merge queue Aug 24, 2026
Merged via the queue into main with commit dffaafc Aug 24, 2026
61 checks passed
@elenagaljak-db
elenagaljak-db deleted the stack/eos-proto branch August 24, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants