Skip to content

Streaming partials for @Extractable - #4

Merged
asaptf merged 2 commits into
mainfrom
feat/streaming-partials
Aug 10, 2026
Merged

Streaming partials for @Extractable#4
asaptf merged 2 commits into
mainfrom
feat/streaming-partials

Conversation

@asaptf

@asaptf asaptf commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Closes the last open feature on the roadmap. A caller can now show fields filling in as the model produces them instead of waiting on a spinner.

let stream = Extract.stream(from: pdfURL, as: Receipt.self, using: session)
for try await update in stream {
    switch update {
    case .partial(let p):   // Receipt.Partial — every field optional
    case .final(let r):     // ExtractionResult<Receipt> — identical to `detailed`
    }
}

The rule this feature is built around

A partial never shows a value that a later snapshot contradicts within the same token. While the model is writing 473.00, the caller does not see 47 — a wrong total on screen is worse than an empty field. A scalar surfaces only once its JSON token is provably complete: closing quote for strings, a delimiter after a number, the full keyword for true/false/null.

This trades latency for truthfulness deliberately, and DECISIONS.md says so explicitly so it does not get "optimised" later.

What the assembler actually emits mid-stream, now locked down by exact assertions:

Buffer Snapshot
{"total":473 {}
{"total":473.00 {}
{"total":473.00} {"total":473.00}
{"name":"Ada","total":47 {"name":"Ada"}
{"lineItems":[{"sku":"A","qty":1 {"lineItems":[]}
{"lineItems":[{"sku":"A","qty":1} {"lineItems":[{"sku":"A","qty":1}]}

Arrays grow element by element as each element completes.

Honest limits, documented rather than papered over

  • Chunked documents emit no partials, only .final. Per-chunk snapshots are incoherent before the deterministic merge, and we have measurements showing what that merge is worth.
  • Signals, provenance, grounding and tables attach to .final only. A partial is a UI preview, not an evidenced result.
  • Repair retries are not streamed. Partials come from the first attempt; the stream still ends with .final, or throws the same errors as detailed.

Compatibility

Additive throughout. Extractable gains associatedtype Partial: Decodable & Sendable = Self, so hand-written conformances keep compiling; the macro generates a real Partial for @Extractable types. The ExtractionGenerating seam gains a streaming method with a default implementation that calls the existing generate once, so every existing test conformance and MockLanguageModel are untouched.

Verification

  • swift build, swift test190 tests; the pre-existing 177 pass unchanged
  • swift format lint --strict — silent
  • Macro expansion tests cover nested types and arrays

A first pass of the regression tests guarded their assertions behind if let snap { … }, which would have passed vacuously had the parser returned nil — including if it drifted toward emitting nothing at all. Those are the tests protecting the rule above, so they were rewritten to assert exact snapshots in both directions, and a missing case was added: a completed field alongside an incomplete one, which is the realistic mid-stream shape neither original test covered.

asaptf added 2 commits August 10, 2026 13:13
Add Extract.stream that yields ExtractionUpdate.partial snapshots as the
model produces JSON, then a terminal .final matching Extract.detailed.

Partial is an associatedtype on Extractable (default Self) so hand-written
conformances stay intact; the macro generates a nested Partial with every
field optional. Half-tokens are withheld — numbers need a delimiter,
strings a closing quote — trading latency for truthfulness.

Chunked documents emit only .final; repair retries are not streamed;
signals/tables attach to the final result only.
Replace vacuous if-let guards with exact expected snapshots so
half-token regressions cannot pass by emitting nothing. Cover the
realistic completed-field-then-incomplete-field shape.
@asaptf
asaptf merged commit 6221d66 into main Aug 10, 2026
2 of 3 checks passed
@asaptf
asaptf deleted the feat/streaming-partials branch August 10, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant