fix: two-pass extraction truncated on thinking-heavy models (qwen3.8) - #31
Merged
Conversation
…ncation The two-pass Ollama flow auto-sized Pass 1's context via _dynamic_num_ctx with only a 4096-token output reserve. Thinking-heavy models that emit long exhaustive extractions (qwen3.8:27b) hit the context window mid-list (done_reason: length) and silently truncated, ~halving coverage on larger meetings. Pass 1 now reserves 16384 output tokens; Pass 2 unchanged. _call_ollama_chat gains an output_reserve parameter. Verified via millet's real code path on EN/TR/DE meetings: qwen3.8 topic coverage on a 44 KB meeting rose 8 -> 20 (matching the cloud baseline), format 5/5, localized headers intact. Bump 0.15.0 -> 0.15.1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Ollama two-pass summarization flow auto-sizes Pass 1's context window via
_dynamic_num_ctx, which reserved only 4096 output tokens. Thinking-heavy models that emit long exhaustive extractions — notablyqwen3.8:27b— exhaust the window mid-extraction (done_reason: length) and silently truncate, roughly halving topic/action/question coverage on larger meetings.Discovered while evaluating qwen3.8 through millet's real code path (not a harness): an earlier ad-hoc test showed 22 topics on a 44 KB meeting, but the shipped pipeline produced only 8 — the extraction was being cut off.
Fix
_call_ollama_chatgains anoutput_reserveparameter (default 4096, unchanged behavior).output_reserve=16384; Pass 2 (small input) unchanged.done_reason: shipped config →length(truncated); fix →stop(complete).Evidence (measured through millet's real two-pass, qwen3.8:27b)
Pass-1 extraction on the 44 KB meeting grew ~1.8 KB → ~5.6 KB. Format 5/5 throughout; no hallucinated speakers.
Tests
TestDynamicNumCtx: output_reserve widens context + respects ceiling.output_reserve=16384, Pass 2 keeps 4096.tests/test_summarize*.py: 45 passed; ruff clean.Version 0.15.0 → 0.15.1.