Observed against pipeline.py at 79a7741 (the #75 branch), by calling sanitize_reply() directly. Each line below is a real run, not a prediction.
These are all shapes #75 does not claim to cover. Filing rather than widening #75, which is already large.
| input |
expected |
got |
Here is code:\n rm -rf /\nDone |
code not spoken |
Here is code: rm -rf / Done |
See <https://example.com/very/long/path> |
URL not spoken |
URL survives verbatim |
[outer [inner] label](https://x.test/p) |
label only |
full URL survives |
Write \label` for a link.` |
[label](url) spoken literally |
Write label for a link. |
Use A → B. Press ↑ twice. |
arrows kept or transliterated |
Use A B. Press twice. |
See [docs]\x00(https://x.test) |
URL not spoken |
URL survives (control byte is removed after markdown parsing) |
\x9b31mred\x9b0m |
nothing spoken |
31mred0m |
ESC P ... ESC \ (DCS) |
whole sequence gone |
payload spoken |
Two structural notes behind several of these:
- Ordering.
_strip_markdown() runs before _CONTROL_CHARS_RE and _EMOJI_RE, so a control byte or variation selector sitting inside markdown syntax hides that syntax from the parser and is then removed, leaving the syntax behind. Canonicalising before parsing would close the whole row-group at once.
- Links run before code spans.
_strip_links runs ahead of _iter_code_spans, so link syntax inside a code span is rewritten even though a span's content is supposed to be literal. The file already treats span content as literal for emphasis; links are the one pass that does not.
Worth a decision, not just a patch: _strip_markdown is now ~7 ordered passes whose comments are mostly about how each one breaks the others (fences before markers, markers looped, headings after fences, links before spans). Every gap above is "one more ordered pass". A real block/inline parse would make the ordering constraints disappear rather than accumulate. The failure modes here are UX (bad audio, or speech silently cut), not exfiltration — the text is the model's own reply to the user who asked — so this is a question of how much correctness is worth buying, not an urgent fix.
Observed against
pipeline.pyat 79a7741 (the #75 branch), by callingsanitize_reply()directly. Each line below is a real run, not a prediction.These are all shapes #75 does not claim to cover. Filing rather than widening #75, which is already large.
Here is code:\n rm -rf /\nDoneHere is code: rm -rf / DoneSee <https://example.com/very/long/path>[outer [inner] label](https://x.test/p)Write \label` for a link.`[label](url)spoken literallyWrite label for a link.Use A → B. Press ↑ twice.Use A B. Press twice.See [docs]\x00(https://x.test)\x9b31mred\x9b0m31mred0mESC P ... ESC \(DCS)Two structural notes behind several of these:
_strip_markdown()runs before_CONTROL_CHARS_REand_EMOJI_RE, so a control byte or variation selector sitting inside markdown syntax hides that syntax from the parser and is then removed, leaving the syntax behind. Canonicalising before parsing would close the whole row-group at once._strip_linksruns ahead of_iter_code_spans, so link syntax inside a code span is rewritten even though a span's content is supposed to be literal. The file already treats span content as literal for emphasis; links are the one pass that does not.Worth a decision, not just a patch:
_strip_markdownis now ~7 ordered passes whose comments are mostly about how each one breaks the others (fences before markers, markers looped, headings after fences, links before spans). Every gap above is "one more ordered pass". A real block/inline parse would make the ordering constraints disappear rather than accumulate. The failure modes here are UX (bad audio, or speech silently cut), not exfiltration — the text is the model's own reply to the user who asked — so this is a question of how much correctness is worth buying, not an urgent fix.