Skip to content

sanitize_reply: markdown shapes that still reach Piper #76

Description

@jamditis

Observed against pipeline.py at 79a7741 (the #75 branch), by calling sanitize_reply() directly. Each line below is a real run, not a prediction.

These are all shapes #75 does not claim to cover. Filing rather than widening #75, which is already large.

input expected got
Here is code:\n rm -rf /\nDone code not spoken Here is code: rm -rf / Done
See <https://example.com/very/long/path> URL not spoken URL survives verbatim
[outer [inner] label](https://x.test/p) label only full URL survives
Write \label` for a link.` [label](url) spoken literally Write label for a link.
Use A → B. Press ↑ twice. arrows kept or transliterated Use A B. Press twice.
See [docs]\x00(https://x.test) URL not spoken URL survives (control byte is removed after markdown parsing)
\x9b31mred\x9b0m nothing spoken 31mred0m
ESC P ... ESC \ (DCS) whole sequence gone payload spoken

Two structural notes behind several of these:

  • Ordering. _strip_markdown() runs before _CONTROL_CHARS_RE and _EMOJI_RE, so a control byte or variation selector sitting inside markdown syntax hides that syntax from the parser and is then removed, leaving the syntax behind. Canonicalising before parsing would close the whole row-group at once.
  • Links run before code spans. _strip_links runs ahead of _iter_code_spans, so link syntax inside a code span is rewritten even though a span's content is supposed to be literal. The file already treats span content as literal for emphasis; links are the one pass that does not.

Worth a decision, not just a patch: _strip_markdown is now ~7 ordered passes whose comments are mostly about how each one breaks the others (fences before markers, markers looped, headings after fences, links before spans). Every gap above is "one more ordered pass". A real block/inline parse would make the ordering constraints disappear rather than accumulate. The failure modes here are UX (bad audio, or speech silently cut), not exfiltration — the text is the model's own reply to the user who asked — so this is a question of how much correctness is worth buying, not an urgent fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions