Skip to content

sanitize_reply drops a fenced block even when it wraps plain prose #74

Description

@jamditis

Follow-up filed from the codex review on the #24 PR. Deferred rather than fixed in-PR because the fix needs a prose-vs-code heuristic, which is a materially larger change than #24 scoped.

What happens

sanitize_reply() drops fenced blocks wholesale (pipeline.py, _strip_markdown). When the model wraps an otherwise good natural-language answer in triple backticks, the whole answer goes and the member hears the empty-reply fallback:

sanitize_reply("```\nTwo plus two is four.\n```")  ->  "Sorry, I don't have an answer for that."

The model returned usable text and we speak an apology.

Why it was not fixed in #24

Dropping fenced content is the behaviour #24 asked for ("strip ... markdown that the VOICE_SYSTEM_PROMPT already asks the model to avoid"; the prompt says "no code blocks"). The same review pass that raised this also, in an earlier round, raised the opposite finding: an unterminated fence must be dropped with its contents so code is never spoken. Both cannot hold unconditionally, because the sanitizer cannot tell code from prose inside a fence.

The failure modes are not symmetric, so the current default is the safe one:

What a fix would need

A signal for "this fence holds prose, not code" that is cheap and hard to fool. Sketches, none obviously right:

  • Unwrap only when the fence has no language tag AND its content has no code-ish tokens (no ;{}()=, no leading indentation, no import/def/function).
  • Unwrap only when the fence is the entire reply and it is one or two sentences that end in terminal punctuation.
  • Leave it alone and treat this as the model's bug, since VOICE_SYSTEM_PROMPT already forbids fences.

Worth measuring how often a fenced-prose reply actually occurs before building any of them. If it is rare, the current fallback is the right answer and this issue closes as wontfix.

wake-20260716T1705-46da79

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions