Skip to content

Fix Windows console/file UTF-8 encoding crashes - #131

Open
Prysiaznyj wants to merge 1 commit into
browser-use:mainfrom
Prysiaznyj:fix/windows-utf8-and-srt-encoding
Open

Fix Windows console/file UTF-8 encoding crashes#131
Prysiaznyj wants to merge 1 commit into
browser-use:mainfrom
Prysiaznyj:fix/windows-utf8-and-srt-encoding

Conversation

@Prysiaznyj

@Prysiaznyj Prysiaznyj commented Aug 13, 2026

Copy link
Copy Markdown

Summary

  • Several helpers/*.py scripts print non-ASCII characters (arrows); on a stock Windows console (non-UTF-8 codepage, e.g. cp1252) this raises UnicodeEncodeError unless the caller happens to set PYTHONIOENCODING=utf-8 first. Reconfigure stdout to UTF-8 unconditionally at import time in render.py, grade.py, pack_transcripts.py, timeline_view.py, and transcribe_batch.py.
  • render.py's build_master_srt wrote master.srt without an explicit encoding, corrupting accented characters on Windows and breaking the downstream ffmpeg subtitles burn (Invalid UTF-8 in decoded subtitles text). Now written explicitly as UTF-8.
  • render.py's build_final_composite: after the encoding fix, ffmpeg's subtitles filter still failed on Windows (Invalid data found when processing input) because the absolute subtitles path used backslashes, colliding with the filter's own escaping syntax. Normalized to forward slashes before escaping the drive-letter colon.

Found and fixed while running the full transcribe → pack → render pipeline end-to-end on Windows against real footage.

Test plan

  • Ran pack_transcripts.py on a real transcript on Windows without PYTHONIOENCODING set — no crash, arrow prints correctly.
  • Ran the full render.py pipeline (extract → concat → subtitles burn) on Windows — master.srt accents intact, ffmpeg subtitles filter succeeds.
  • Not tested on macOS/Linux (the encoding issue is Windows-specific; reconfigure(encoding="utf-8") is a no-op there since stdout is already UTF-8).

Summary by cubic

Fixes Windows-specific Unicode and path issues that caused UnicodeEncodeError in helper output and subtitle burn failures. Previously: helpers printed non-ASCII to a non-UTF-8 console and master.srt used the default encoding/backslash paths; now: stdout is UTF-8, master.srt is UTF-8, and subtitle paths use forward slashes with an escaped drive colon.

  • Reconfigure stdout to UTF-8 at import in helpers/render.py, helpers/grade.py, helpers/pack_transcripts.py, helpers/timeline_view.py, and helpers/transcribe_batch.py. Callers no longer need PYTHONIOENCODING set. No-op on macOS/Linux.
  • Write master.srt with UTF-8 encoding to preserve accents and satisfy ffmpeg subtitle parsing.
  • Normalize subtitle path for ffmpeg by replacing backslashes with forward slashes and escaping the drive-letter colon to avoid filtergraph escaping conflicts on Windows.

Written for commit 4e7cc82. Summary will update on new commits.

Review in cubic

Two related issues hit on Windows, found while running the full
transcribe -> pack -> render pipeline from a stock Windows terminal:

- helpers/render.py, grade.py, pack_transcripts.py, timeline_view.py,
  transcribe_batch.py: several print() calls use non-ASCII characters
  (mostly arrows). Windows consoles default to a non-UTF-8 codepage
  (e.g. cp1252), which raises UnicodeEncodeError on those prints unless
  the caller happens to set PYTHONIOENCODING=utf-8 first. Reconfigure
  stdout to UTF-8 unconditionally at import time instead of relying on
  that env var being set.

- helpers/render.py build_master_srt: master.srt was written without an
  explicit encoding, so accented characters got mangled on platforms
  where the default text encoding isn't UTF-8 (Windows again). This also
  broke the downstream ffmpeg subtitles burn ("Invalid UTF-8 in decoded
  subtitles text"). Write it explicitly as UTF-8.

- helpers/render.py build_final_composite: after fixing the encoding
  above, ffmpeg's subtitles filter still failed on Windows
  ("Invalid data found when processing input") because the absolute
  subtitles path used backslashes, which collide with the filter's own
  escaping syntax. Normalize to forward slashes before escaping the
  drive-letter colon.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="helpers/grade.py">

<violation number="1" location="helpers/grade.py:40">
P3: `sys.stdout.reconfigure(encoding="utf-8")` runs unconditionally at import time and raises `AttributeError` if `sys.stdout` is not a live `TextIOWrapper` (e.g. stdout is `None` under `pythonw.exe`, or the module is imported while stdout is replaced by a non-filesystem stream such as `io.StringIO` under `contextlib.redirect_stdout`). These are standalone CLI scripts so normal console runs are unaffected, but guarding the call makes the fix robust to those environments instead of trading one crash (UnicodeEncodeError) for another (AttributeError). This applies identically to all four added calls in this batch.</violation>
</file>

<file name="helpers/render.py">

<violation number="1" location="helpers/render.py:34">
P3: `sys.stdout.reconfigure()` raises `AttributeError` when `sys.stdout` is not a `TextIOWrapper` (e.g., stdout redirected to a StringIO or a non-buffered wrapper in an embedded/hosted interpreter), and this call runs unconditionally at import time so it would abort the module before any fallback. In normal CLI usage stdout is a TextIOWrapper, so this is an edge case; guard it (or use the `errors`-safe reconfigure only when available) to keep the module importable in such environments.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread helpers/grade.py
# Windows consoles default to a non-UTF-8 codepage (e.g. cp1252), which raises
# UnicodeEncodeError on the arrows this module prints. Force UTF-8 stdout
# unconditionally rather than relying on PYTHONIOENCODING being set.
sys.stdout.reconfigure(encoding="utf-8")

@cubic-dev-ai cubic-dev-ai Bot Aug 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: sys.stdout.reconfigure(encoding="utf-8") runs unconditionally at import time and raises AttributeError if sys.stdout is not a live TextIOWrapper (e.g. stdout is None under pythonw.exe, or the module is imported while stdout is replaced by a non-filesystem stream such as io.StringIO under contextlib.redirect_stdout). These are standalone CLI scripts so normal console runs are unaffected, but guarding the call makes the fix robust to those environments instead of trading one crash (UnicodeEncodeError) for another (AttributeError). This applies identically to all four added calls in this batch.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At helpers/grade.py, line 40:

<comment>`sys.stdout.reconfigure(encoding="utf-8")` runs unconditionally at import time and raises `AttributeError` if `sys.stdout` is not a live `TextIOWrapper` (e.g. stdout is `None` under `pythonw.exe`, or the module is imported while stdout is replaced by a non-filesystem stream such as `io.StringIO` under `contextlib.redirect_stdout`). These are standalone CLI scripts so normal console runs are unaffected, but guarding the call makes the fix robust to those environments instead of trading one crash (UnicodeEncodeError) for another (AttributeError). This applies identically to all four added calls in this batch.</comment>

<file context>
@@ -34,6 +34,11 @@
+# Windows consoles default to a non-UTF-8 codepage (e.g. cp1252), which raises
+# UnicodeEncodeError on the arrows this module prints. Force UTF-8 stdout
+# unconditionally rather than relying on PYTHONIOENCODING being set.
+sys.stdout.reconfigure(encoding="utf-8")
+
 
</file context>
Suggested change
sys.stdout.reconfigure(encoding="utf-8")
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8")
Fix with cubic

Comment thread helpers/render.py
# Windows consoles default to a non-UTF-8 codepage (e.g. cp1252), which raises
# UnicodeEncodeError on the arrows/em-dashes this module prints. Force UTF-8
# stdout unconditionally rather than relying on PYTHONIOENCODING being set.
sys.stdout.reconfigure(encoding="utf-8")

@cubic-dev-ai cubic-dev-ai Bot Aug 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: sys.stdout.reconfigure() raises AttributeError when sys.stdout is not a TextIOWrapper (e.g., stdout redirected to a StringIO or a non-buffered wrapper in an embedded/hosted interpreter), and this call runs unconditionally at import time so it would abort the module before any fallback. In normal CLI usage stdout is a TextIOWrapper, so this is an edge case; guard it (or use the errors-safe reconfigure only when available) to keep the module importable in such environments.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At helpers/render.py, line 34:

<comment>`sys.stdout.reconfigure()` raises `AttributeError` when `sys.stdout` is not a `TextIOWrapper` (e.g., stdout redirected to a StringIO or a non-buffered wrapper in an embedded/hosted interpreter), and this call runs unconditionally at import time so it would abort the module before any fallback. In normal CLI usage stdout is a TextIOWrapper, so this is an edge case; guard it (or use the `errors`-safe reconfigure only when available) to keep the module importable in such environments.</comment>

<file context>
@@ -28,6 +28,11 @@
+# Windows consoles default to a non-UTF-8 codepage (e.g. cp1252), which raises
+# UnicodeEncodeError on the arrows/em-dashes this module prints. Force UTF-8
+# stdout unconditionally rather than relying on PYTHONIOENCODING being set.
+sys.stdout.reconfigure(encoding="utf-8")
+
 try:
</file context>
Suggested change
sys.stdout.reconfigure(encoding="utf-8")
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8")
Fix with cubic

DarkStyleee added a commit to DarkStyleee/video-use that referenced this pull request Aug 20, 2026
…font on Windows

Three small Windows failures, none of them covered by browser-use#131 or browser-use#128.

grade.py passed a temp file path straight into metadata=print:file=. Inside a
filter option value the drive colon reads as an option separator, so ffmpeg
reports "No option name near 'UsersRunner...'" and the analysis dies before a
single frame is sampled. The path is now escaped the way render.py already
escapes the subtitles path. Verified with --analyze on a real capture: the
filter it computes is byte-identical to the one from a run where the path
parses.

render.py and timeline_view.py read the transcript and the EDL with read_text(),
which decodes with the locale codepage. This does not raise on a non-ASCII
transcript, which would at least be visible. On cp1251 it returns mojibake:
"привет" comes back as "Рїсђрёрірес‚", and the subtitles are built from that.
Both files are JSON, so UTF-8 is the only correct decoding.

FONT_CANDIDATES listed macOS and Linux paths only. load_font falls back to
ImageFont.load_default(), so the timeline still renders, in a bitmap font at a
fixed size. Consolas and Arial appended.
DarkStyleee added a commit to DarkStyleee/video-use that referenced this pull request Aug 20, 2026
…oted path

Review on browser-use#135 pointed at the write side and at a gap in the escaping.

The concat list is written with the locale codepage, so a segment under a path
with non-ASCII characters is written in a form ffmpeg cannot open: "Impossible to
open ..." and the render stops. The same list written as UTF-8 concatenates. This
is the write half of the read fix already in this branch.

The master SRT had the same problem with a worse ending, since it is the artifact
the viewer sees: written in a codepage, libass reads it as UTF-8 and the subtitles
come out as mojibake. browser-use#131 carries this one line too. Keeping it here so the
branch fixes reading and writing together rather than half of each.

pack_transcripts.py reads the same transcript JSON with the locale codepage. What
transcribe.py writes is ASCII-escaped by json.dumps, so this one needs an
externally produced transcript to bite, but the read should still name UTF-8.

_esc_filter_path escaped the drive colon but not the quote, while the value it
produces is wrapped in single quotes. A temp path under a user named O'Brien
would close the quoted section early. render.py escapes both; now this does too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant