chore(release): v1.8.0 review — do not merge - #218
Closed
EtienneLescot wants to merge 385 commits into
Closed
Conversation
… architecture Moves the performance material out of export-pipeline.md and drops its supersedes preamble. Two stale claims corrected against the code: audio.rs now mixes every audio stream (the av_find_best_stream single-track defect that dropped the macOS mic is fixed), and the concurrent-save corruption is fixed — what survives is the rule that a partial document which parses is worse than one that does not.
recording.md merges the Windows and macOS recorder docs into one description with a per-platform helper-contract table, replacing two roadmaps' worth of goals, phases and ship criteria.
Locates the cursor settings UI where it actually is (CursorPane in RightPanes.tsx, reached through the v4 inspector's cursor facet) instead of the deleted panel the old inventory named.
Widens the STT spec into the whole audio-to-captions path and documents the caption layer for the first time: cues are a rendering of the transcript, not a parallel edit surface. Drops the CTranslate2 history and the changelog.
Replaces 19 sequenced spec files (~4,800 lines) that targeted a deleted editor shell with 198 surface-organised checks derived from the components that exist, merged with the RC smoke checklist.
Everything worth keeping from the merge roadmaps, the collision analysis, the two source-repo inventories, the STT migration and POC plans, the recorder roadmaps, the poc-d3d stage logs and the 26 spec files has been folded into the architecture, engineering and testing docs. The rest was history, and git already has it.
ci.yml was scoped to main only, so every PR landing on the long-lived feat/ai-edition integration branch merged with no lint, no typecheck, no tests and no PR-title check. Adds a Docs job running check-docs.mjs. It skips the shared setup action on purpose: the script imports only node builtins, so npm ci would cost a minute for nothing. Drops this branch's comment edit to tests/e2e/roadmap-coverage.spec.ts — #166 deletes that file, and modify-vs-delete is a conflict for no gain.
…ear lint Turning CI on for feat/ai-edition surfaced failures that had been invisible because ci.yml only ran on main. None were caused by the docs work. compositorViewService: the absolute-pin test hardcoded "C:/vendor/ffmpeg", which path.isAbsolute rejects on POSIX, so the pin got nested under the crate dir. Spell it through path.resolve and assert the behaviour instead of one platform's drive letter. The production code was right. whisperServer: two tests wrote a fake helper binary with the default mode, which is not executable on POSIX, so the manager refused it before reaching the assertion under test. Write it 0o755. Lint: biome format across 11 files, plus five dead variables it would only remove under --unsafe. Each checked for side effects first — run_wcpp's reported_rtf was written and never read (rtf is recomputed from wall_s), and e2e-pipeline-smoke keeps its dynamic import, only dropping the unused binding.
… click FloatingInspector floats its "Collapse inspector" button over the reused pane header (position: absolute, right: 12, 26px wide). That lands squarely on whatever the header puts at its right edge and eats its clicks. The Help button was the casualty: rendered on every facet, visible, and unreachable by mouse. CaptionsPane's status icon sits under it too. Fixed where all three headers share a definition rather than nudging the one button: .paneHead now reserves the floated control its own lane. Every pane wearing that class is rendered inside FloatingInspector, so there is no other caller to regress. Caught by the e2e rewrite in the following commit.
The three editor specs were written against the pre-v4 shell and could not pass: they waited on a `timeline-pane` testid that died with TimelinePane.tsx, read data-clip-count / data-zoom-multiplier / data-px-per-sec attributes the v4 timeline never had, and seeded the browser shim through the localStorage keys it stopped reading when it moved to `browser-shim-projects-v2`. Their fixtures were schemaVersion 3, so they exercised the v3→v4→v5 migration path rather than the shape the app actually stores. roadmap-coverage.spec.ts → v4-shell.spec.ts, renamed off the retired roadmap task ids, three behaviours kept and two dropped: - asset file size, now read off MediaStage's cards; the sized and unsized branches share one fixture instead of two page loads - the facet help popover, still live in RightPanes' Pane but reached from the inspector rail rather than a right-panel tab. Its "Settings for X" fallback assertion is gone with the fallback itself — every pane has real help text - ctrl+wheel zoom, which no longer has a px-per-sec ceiling; v4 zooms a nav window bounded at a 2% minimum span, so that is what the test measures Dropped: the ruler hover-guide, which does not exist in v4 (the only ruler overlay left is the pill-drag snap guide), and the chat "applied:" case, which asserted that the browser shim's canned reply carries no tool calls — the mock, not the app, and the positive half is unreachable without a real provider. seed.spec.ts → editor-smoke.spec.ts. Its timeline half was unsalvageable but its preview half still holds, and it seeds nothing, which makes it the only cover for the cold-boot branch (Preview → EditorEmptyState) and for a crash during mount against an empty shim. Now also listens for pageerror, not just console.error — an uncaught mount exception is the failure a boot smoke test exists to catch. diagnostic.spec.ts deleted. It dumped the DOM and a11y tree to find selectors in the old shell and ended in expect(true).toBe(true); the selectors it existed to discover are written down in v4-shell.spec.ts now. Both specs read E2E_BASE_URL, so tests/e2e no longer hardcodes a port. E2E_BASE_URL=http://localhost:5199 npx playwright test tests/e2e/v4-shell.spec.ts tests/e2e/editor-smoke.spec.ts → 4 passed
ExportDialog has rendered MP4 through exportMultiNative since the D3D compositor landed; the mp4 branch returns before ever reaching exportAxcutDocument, so VideoExporter was reachable only from the bench. GIF is the one format the document adapter still renders. Removes VideoExporter, WgslFrameRenderer, audioEncoder, muxer, nativeFrameSink, planarChunkQueue, perfTimings, videoDecoder, asyncVideoFrameQueue, the RenderPlan/audioConcat layer that only fed VideoExporter, and src/bench (every arm drove the web path). The renderer->ffmpeg IPC encoder goes with them: its own header says REFUTED and its only consumer was gated on a localStorage flag nothing sets. resolveCropAt + CropScheduleEntry move to exporter/cropSchedule.ts, which GIF still needs. Also drops ~29 modules with zero importers across components/ui, video-editor and ai-edition. tsc clean; 94 files / 1108 tests pass (baseline 109/1338 - the delta is test files covering the deleted code).
poc/ (WebGPU) and poc-native/ (Rust+wgpu) were both replaced by poc-d3d, which is the compositor that actually ships. Neither is referenced by any build script; poc/ was reachable only from .claude/launch.json. Also removes poc-d3d/spikes (throwaway C++ probes with .bat builders hardwired to a local Visual Studio path), the S0-S8 spike reports and HANDOFF.md for a decision already shipped, the stt-eval bake-off harness whose verdict shipped as electron/native/whisper-stt (REPORT.md kept), an unreferenced 428 KB sample mp4 and a duplicate rec-button.png.
Removes specs/computer-use (20 files scripting RightPanelStack, Bottombar, TimelinePane and CropModal — none of which have a source file), seven root specs each headed "Roadmap status: done", the two STT plans whose own successor declares itself the replacement, an axcut-UX spec for a surface that does not exist post-merge, and a divergence audit whose conclusion is consumed. Dangling links in the surviving docs are fixed. Also untracks .worktrees/wt-9ce78f24/HANDOFF.md: .worktrees/ is in .gitignore, but this file predates the rule and pins a dead branch plus a G:\ path from another machine. Kept deliberately: collision-analysis, openscreen-inventory and axcut-inventory, which ai-edition-roadmap.md pins as deep reference, and design/openscreen-editor.html, which it calls the canonical UI target. design/_ds_bundle.js (checked-in bundle) and _adherence.oxlintrc.json (config for a linter absent from the repo) do go.
gsap, @pixi/filter-drop-shadow, emoji-picker-react and fix-webm-duration
have zero imports across src, electron, scripts and tests; the last is a
stale twin of @fix-webm-duration/fix, which has the real call sites.
@langchain/langgraph, @langchain/langgraph-checkpoint and langsmith have
no direct imports either — all three arrive transitively through
deepagents/langchain/@langchain/core, and the declared ranges sit below
what deepagents requires, so they pin nothing. website/ drops clsx.
The four "Cache caption assets" blocks in build.yml key on
hashFiles('scripts/fetch-caption-model.mjs'); neither that script nor
before-pack.cjs exists, so the key degenerates to a constant and the path
caches nothing. bench:export goes with src/bench.
fixture_dir() defaulted to an absolute path inside a personal git worktree (C:\Users\camil\...\worktrees\prerelease-version-tag-ee96ae). Two paths reached it: create_view fell back to it when no screen_path was given, and the exportNative fallback rendered the fixture whenever the timeline was empty — reachable from ExportDialog on any release build, where that directory does not exist. create_view now requires screen_path and errors without it. The fixture-export entry point goes entirely: ExportTask, the napi `export` binding, the compositor "export" IPC action and contract variant, the service method and the exportNative client. An empty timeline now raises exportDialog.nothingToExport (added to all 13 locales) instead of silently exporting someone else's fixture. exportMulti — the real timeline path — is untouched. cargo check clean; tsc clean; 94 files / 1108 tests pass.
The docs reorg on release/1.8.0 landed after this branch forked, so it documented modules this branch deletes. `docs:check` caught two dead links; reading around them showed the prose was describing the removed renderer-side pipeline, not the native one that ships. - export-pipeline.md: the `RenderPlan` section documented a type that only ever fed `VideoExporter` — on release/1.8.0 `renderPlan.ts` was imported by `documentExporter`, its own test, and `videoExporter` alone, while `ExportDialog` already called `exportMultiNative(clips, path, sceneJson)` direct. The diagram claimed it fed the addon; it never did. Points at `buildSceneDescription` instead, which preview.md already documents. - Drops the per-segment cursor section: `cursorSamplesForAsset` and `segment.cursorSamples` went with the web pipeline, and the addon's export `ClipInput` carries no cursor telemetry (only `createView` takes a cursorPath), so the section described capability the native path never had. - Output sizing said "largest clip" via `pickReferenceDimensions`; `tierOutputDims` feeds the crop-aware SMALLEST clip to `calculateMp4ExportSettings`. Codec crosses as a plain `ExportVideoCodec` string now, not a WebCodecs encoder string. - preview.md: removes the "Path B" section for `PreviewCompositor.tsx`, which that same section already called dead code today.
clamp(value, min, max) was defined privately in 16 files across the renderer, the electron main process and the native glue — byte-identical every time. src/utils/math.ts now owns it plus clamp01; electron already imports from src/, so one home serves both sides. Left alone deliberately: the domain-specific clamps (clampPlaybackSpeed, clampVirtualTime, clampFocusToStage, clampDeltaMs, clampSpanAgainstNeighbours) are real functions, not copies. useTimeline.ts redeclared the same three NaN-guards inside nine separate actions. They are floors, not clamps, so they are hoisted once as finiteSec/finiteMs/finiteFraction rather than folded into clamp. Also drops easeOutExpo, smoothStep and easeInOutCubic from mathUtils — exported, no caller. tsc clean; 94 files / 1108 tests pass.
uuid had five call sites across two files, all v4. crypto.randomUUID() is already used ten times elsewhere in the same tree — renderer runs in a secure context, main runs Node 22. Drops the dependency. findVcVarsAll() was duplicated byte-for-byte (modulo comments) between build-windows-wgc-helper.mjs and build-windows-compositor-addon.mjs, as was the promisified spawn. Both move to scripts/msvcEnv.mjs. The two runInVsEnv bodies genuinely differ (SDK lib compat shims vs cargo) and stay put. The two scripts drop from 197/187 lines to 99/93. Kept deliberately: `motion`. Its spring() is a real integrator (keyframes, velocity, stiffness, damping, restDelta, generator protocol) driving cursor smoothing; hand-rolling one to shed a dependency is writing code, not deleting it. tsc clean; 94 files / 1108 tests pass.
m:ss.t was hand-rolled in Modals (formatTc), V4Timeline (fmt) and operations (formatSec); the auto-hours variant existed twice, byte- identical, as MediaStage's formatTimecode and virtual-preview's formatSeconds. All five now come from timeline/format.ts as formatSec and formatSeconds, with formatMs delegating to formatSec. Left alone deliberately, because they are different formats rather than copies: ExportDialog's formatHms (hh:mm:ss, padded hours, no tenths) and timeUtils' formatTimePadded (mm:ss). formatSize was duplicated between MediaStage and LeftPanel; it moves to utils/formatBytes.ts. NOT converted to Intl.NumberFormat as the review suggested — `unit: "byte"` formats in decimal units and localises the unit name, so a French locale renders 1 GiB as "1,1 Mdo": wrong magnitude, unreadable label. The binary ladder stays. Adds format.test.ts covering the cases the six copies disagreed on (negatives, NaN, Infinity, the hour boundary). tsc clean; 95 files / 1113 tests pass.
sceneDescription hand-rolled a linear-gradient parse (strip parens, flat
comma split, first-token per stop) next to a comment admitting "presets
never contain nested parens". exporter/gradientParser's parseCssGradient
already does this and is strictly better: it tokenises rgba()/hsl() stops
that the flat split would have shredded, and resolveLinearGradientAngle
handles the "to bottom right" keyword directions the regex ignored. Same
180-degree default. All 73 sceneDescription tests pass unchanged.
editorSettings' nextLegacy listed 16 `if (patch.x !== undefined)` lines.
Note a plain {...base, ...patch} is NOT equivalent — it writes undefined
over values the caller left alone — so the keys are filtered first. Added
a test for exactly that; it fails against the naive spread.
tsc clean; 95 files / 1114 tests pass.
…dients
Three of the copies folded into the shared helpers were not byte-identical,
and the differences were the interesting part.
`clamp` — blurEffects and video-editor/types both guarded non-finite input;
the shared version did not, so `clamp(NaN, …)` returned NaN instead of a
floor. That reached the exporter: clampFocusToDepth feeds clampFocusToScale,
which frameRenderer and wgsl/evaluate use to build the zoom transform. The
guard now lives in the shared `clamp` (non-finite -> min), which is
blurEffects' old contract exactly and hardens the other 14 call sites for
free. Focus wants a midpoint rather than a floor — an unknown focus means
"centre" — so it keeps a two-line local, same convention as useTimeline's
`finiteFraction`.
`parseWallpaper` — parseCssGradient is anchored on a trailing ")", so it
rejects strings the `startsWith("linear-gradient(")` branch accepts; a
trailing space was enough. Returning null fell through to the image arm and
handed the native scene a CSS string as a file path. The branch is total
again: no parse means angle 180 and no stops, and the Rust side already
falls back to the layout background on empty stops.
Tests: the NaN cases at the shared definition and at both callers, plus the
rgba()/keyword-direction wins the swap was made for — untested until now —
and the reject path.
- `nextLegacy`'s cast was `Partial<LegacyShape>`, which accepts any key. Pick it down to the patch's own keys so an EditorSettingsPatch field with no legacy counterpart fails to compile, as the 16 `if`s used to (verified: a bogus key now errors TS2344). - build-windows-compositor-addon's `run()` picked up ROOT as its default cwd in the msvcEnv extraction; it was poc-d3d/, which is where cargo has to run for .cargo/config.toml's FFMPEG_DIR to apply. Only call site passes cwd explicitly so nothing broke, but the next one would have. - chat-service now imports randomUUID from node:crypto like codex-session next to it, rather than reaching for the global in main-process code. - Dead JSDoc for the deleted firstTokenOf, stacked on parseWallpaper's own and still claiming the presets never nest parens. Plus two import placements.
document.agent, .preview, .export and .history were declared, defaulted, seeded in every test fixture and written on every edit — and read by nothing. preview.revision in particular was bumped at eight sites and consumed only by tests asserting the bump happened. Removes those fields, the ten schemas behind them (pendingQuestion, preview, exportState, suggestion, agentState, operation, revision, plus the three input schemas with no callers) and their inferred types. timelineOperationSchema STAYS: the review called it dead, but AxcutTimelineOperation has 19 call sites. operationSchema only extended it with suggestion ops nothing dispatches, so that one does go. Old saved projects are unaffected: documentSchema is not strict, so zod strips the now-unknown keys on parse. schema/index.ts 709 -> 604 lines. tsc clean; 95 files / 1114 tests pass.
llm-call.ts opened with "no LangChain dependency" and hand-rolled a multi-provider fetch/SSE client — OpenAI-compat, Anthropic, Codex and Copilot dialects, 736 lines — while the live agent loop went through LangChain the whole time (chat-service.ts -> deep-agent/service.ts). Its only two remaining callers were one-shot prompt->text calls: chat compaction and caption translation. Both now use createOpenScreenChatModel(...).invoke(), which already knew every provider. Removes llm-call.ts, codex-session.ts (a bespoke chatgpt.com Responses dialect reachable only from it) and agent-provider-capabilities.ts, whose per-provider reasoning-effort matrix was duplicated inside deep-agent/. The Copilot runtime-token swap is preserved, and this fixes it rather than porting it: chat-model.ts previously used the PAT directly with a comment saying the refresh "can land as a follow-up", so the MAIN agent path was the broken one. It now exchanges the PAT for the short-lived runtime token, honours the base URL that exchange returns, and sends the editor-identifying headers the endpoint requires. Every caller benefits, not just the two being rerouted. extractDelta (MessageContent -> text) is exported from deep-agent/service rather than re-implemented. tsc clean; 93 files / 1082 tests pass (the delta is the two deleted test files, which covered the deleted code).
addClipBefore, addClipAfter, editClip, splitAndInsert, addTrimAt, updateTrimRange and clearClipSelection were defined, memoised and returned from useTimeline, and called by nothing. insertClipAt, setTrimEntries and updateClipSourceRange already cover every UI path. editClip had its own test suite, which is why it looked alive. The live Edit Clip modal does not use it: NewEditorShell.tsx:1316 calls tl.updateClipSourceRange + tl.updateClipCrop. The tests went with the action. Also removes RightPanes' TimelinePaneBody and the RightPaneId type, both exported and unreferenced — FloatingInspector's Facet union replaced the latter. useTimeline.ts 1192 -> 970 lines. tsc clean; 93 files / 1079 tests pass.
Review follow-ups on the llm-call.ts removal. The reroute to createOpenScreenChatModel was right, but three things did not survive it. `accountId` was threaded from both new call sites into OpenScreenChatModelConfig and then read by nobody. The deleted llm-call.ts set `chatgpt-account-id` from it and called the header required by chatgpt.com/backend-api, so caption translation and chat compaction were newly broken for Codex accounts — the same bug class the Copilot fix addressed, pointed the other way. tsc cannot see it: an optional field that nothing reads still typechecks. caption-translate built a model per batch. For Copilot that is a PAT -> runtime-token exchange against api.github.com on every iteration; at batch size 40 a 400-segment transcript paid for ten of them. Built once, above the loop, with the construction failure kept in the same result shape. extractDelta is now messageContentToText in chat-model.ts. It was exported from deep-agent/service.ts, whose top level pulls `langchain` and the whole agent-tools graph — a lot to import for ten lines of content flattening. chat-service.ts worked around it with a dynamic import; caption-translate.ts did not. Both now import it statically with nothing dragged along. Also normalizes the provider once at the createOpenScreenChatModel entry point. The deleted agent-provider-capabilities.ts resolved `claude` -> anthropic and `gemini` -> google and had tests for it; the surviving deep-agent copy compares raw strings, so those aliases stopped resolving and normalizeProviderId was left with no callers at all. And the surviving copy was not the equivalent duplicate the deletion assumed: isOpenRouterReasoningModel strips the `anthropic/` slug prefix but not `openai/`, and both vendor matchers are anchored at the start — so openrouter + openai/gpt-5 silently got no reasoning wiring. The deleted file stripped both and its deleted test asserted it. Adds the tests back for the module that survived, plus runtime checks for the account header and the alias routing.
migrate.ts still built `agent: {pendingQuestions, suggestions,
lastAppliedOperations}` on every migrated project. zod strips it, so it
was harmless — and it was in shipped code, not a fixture.
transcribe.test.ts still carried agent/preview/export, including a
`preview.playbackRate` that was never in the schema.
These survived because tsconfig.json excludes **/*.test.ts(x) and its
`include` is ["src", "electron"] — no test file and nothing under tests/
is typechecked, and vitest transpiles without checking. So the gate the
deletion leaned on could not see the fixtures it rewrote by hand.
Also drops the comments still describing `preview.revision`, a field that
no longer exists.
(The roadmap-coverage.spec.ts fixture this originally also fixed is gone:
release/1.8.0 deleted that spec in the v4-shell e2e retarget.)
The boolean was declared, defaulted, persisted, migrated and
history-tracked across five files and read by nothing. Its only UI was
RightPanes' TimelinePaneBody, which was already unreferenced before PR 3
deleted it — so the toggle had no way to be set and no effect if it was.
Removes it from EditorSettingsSnapshot, DEFAULT_EDITOR_SETTINGS,
LegacyShape, getEditorSettings, EditorSettingsPatch,
DEFAULT_EDITOR_APPEARANCE_SETTINGS, ProjectEditorState, the
projectPersistence parse branch, EditorState + INITIAL_EDITOR_STATE, and
the v2 migration's editor envelope.
The i18n keys go with it: settings.timeline.{title,waveform,help} in all
13 locales, not the 10 the note listed — vi, zh-CN and zh-TW were
missed. Those three keys were the whole `settings.timeline` namespace and
TimelinePaneBody was its only consumer; the separate top-level `timeline`
namespace (FloatingInspector, V4Timeline) is untouched.
Old projects keep working: migrate spreads input.editor wholesale into
legacyEditor, and legacyEditorSchema is z.object({}).passthrough(), so a
stored showTrimWaveform rides along inertly instead of failing the parse
— and getEditorSettings reads legacyEditor through a loose shape check
that ignores unknown keys. Pinned with a test rather than assumed, since
tsconfig excludes test files and would not have caught a regression here.
tsc clean; biome clean; 95 files / 1101 tests pass.
… wallpapers load
The native compositor is a Rust addon that opens asset paths with `image::open`.
`resolveSceneAssetPaths` handed it paths rooted at `VITE_PUBLIC`, which is
`<root>/public` in dev but `<resources>/app.asar/dist` in a packaged build: vite
copies `public/` into `dist`, so the files are in there, but inside the archive.
Only Electron's patched `fs` reads through an asar, so every bundled wallpaper
and every themed cursor sprite failed to load in an installed app — silently, on
the native side's colour/default-art fallbacks. Picking a cursor theme looked
like it did nothing, and a bundled image background came out a flat colour.
A CUSTOM uploaded background kept working the whole time, which is what pinned
it down: it travels as a `data:` URL the addon decodes in memory, never a path.
`extraResources` already copies both trees to `<resources>/{wallpapers,cursors}`
as real files — electron-builder.json5's own "Asset layout contract" comment
names exactly this, and `ASSET_BASE_DIR` in electron/windows.ts already resolves
them that way for the renderer. So probe the bases in order and take the one
that actually holds the file, the same candidate-list idiom
`ffmpegSharedBinCandidates` uses right below. Existence-probing rather than
branching on `app.isPackaged` keeps `--dir` staging builds working too.
This resolver also feeds `exportMulti`, so the same assets were missing from
rendered exports in installed builds, not just from the preview.
The gate looked like an oversight now that Linux has real cursor telemetry, and I was about to lift it. Measurement says the gate is right and lifting it would have made the reported "two cursors" bug worse. `cursor: "never"` is not a constraint Chromium implements — it is absent from `getSupportedConstraints()` (verified on Electron 42 / Chromium 148, where the only screen-capture constraints are `displaySurface` and `suppressLocalAudioPlayback`), so WebIDL drops it silently. Below the web API, `DesktopCaptureDevice::Create` wraps every capturer in a `DesktopAndCursorComposer` unconditionally, with no cursor branch; on Linux `prefer_cursor_embedded` is false, so WebRTC asks the portal for METADATA mode and then paints the cursor back in itself. Linux does not even reach the `cursor:` key in useScreenRecorder — that line is inside the win32 branch. Windows and macOS honour the toggle through their NATIVE HELPERS (`captureCursor` to wgc-capture, `hideSystemCursor` to ScreenCaptureKit), not through the constraint, which is a no-op there too. So the gate is precisely "platforms with a native capture helper", and lifting it would ship a control that changes nothing in the pixels while switching the editor's overlay on — producing exactly the two cursors it was meant to fix. `hideSystemCursor` (useScreenRecorder.ts) is inert on Linux for the same reason: it sits inside startNativeMacRecordingIfAvailable, which returns false off darwin. The real fix needs the cursor out of the captured pixels, which needs a native Linux capture helper. Until then the lever that works is `scene.cursor.show`, and defaulting it to false on Linux is a UX decision, not a bug fix — Linux would lose themed, smoothed and click-bounce cursors. Left for a deliberate call.
Stage 2 of the native capture helper starts here: the piece that turns
frames into a file. Nothing calls it yet — the frame source is the next
commit — but the ladder, the muxer and the measurements are in place.
Three backends, tried in order, first one that OPENS wins.
`avcodec_find_encoder_by_name` succeeds for every encoder compiled into
the library including ones this GPU cannot run, so the only honest probe
is to open it with the real parameters.
VAAPI is guarded by a dlsym rather than tried. The vendored ffmpeg 8.1
calls `vaMapBuffer2`, which libva only grew in 2.22; against Ubuntu
24.04's 2.20 the implib trampoline does not return an error, it calls
assert(0) and the process dumps core. A probe that crashes the helper is
worse than no probe.
Vulkan needs RADV_PERFTEST=video_encode before the instance is created,
or ffmpeg reports "Device does not support the VK_KHR_video_encode_queue
extension" on hardware that supports it fine.
Measured here, 1920x1080, Radeon 610M:
convert (BGRA->NV12) 3.62 ms/frame
upload (CPU->GPU) 0.91 ms
encode (h264_vulkan) 0.48 ms
-------
5.01 ms against a 16.7 ms budget at 60 fps
Software (libopenh264) runs the same clip at 40 fps versus Vulkan's 79,
so the fallback still works, just without the headroom.
swscale threading is set to 1 on purpose. 3.80 ms/frame at one thread,
3.84 ms at eight — the request is granted (sws_threads() reads the count
back out) but the conversion is memory-bandwidth bound. An override is
kept because that is one machine at one resolution.
Also fixes the cursor sprite alpha, which Stage 1 left as a documented
guess. It is premultiplied: a captured GNOME sprite had 126
semi-transparent pixels and not one where a colour channel exceeded its
own alpha, which is only true of premultiplied data. PNG wants straight
alpha, so every antialiased edge was being darkened.
One portal session, two outputs: an MP4 with the pointer absent from
every pixel, and the telemetry saying where that pointer was. Chromium's
getDisplayMedia can give neither — it hands back a frame with the cursor
already painted in and no way to know where it went.
Verified on a real 20-second capture at 1920x1080/60: 1213 frames, 0
dropped, 60.05 fps measured, and four crops taken at the positions the
telemetry reported contain no cursor at all.
convert 1.87 ms/frame
upload 1.06 ms
encode 1.00 ms
-------
3.93 ms against 16.7 ms
THE OUTPUT RATE COMES FROM A CLOCK, NOT FROM ARRIVALS. A compositor
sends frames on damage; mutter will go seconds without one while the
user reads a page. One output frame per delivered frame would produce a
file whose playback speed depended on how busy the screen was. So
Capture::advance asks what frame index the wall clock is on and encodes
forward to it, holding the last picture. That is why the encoder splits
conversion from encoding: a held frame costs the upload and the encode
but not the 1.87 ms conversion.
Frames cross the thread boundary through a one-slot mailbox where the
newest wins. A queue would be wrong twice: unbounded at 8 MB per frame,
and every frame it held would add latency to a recording meant to track
the screen. Dropping costs one duplicated frame; queueing costs drift
forever. The count is reported on capture-stopped so a machine that
cannot keep up says so.
Video mode advertises shared-memory buffers only. pw_stream does not map
DmaBuf even with MAP_BUFFERS, so accepting one would leave datas[0].data
NULL and record nothing — not offering it is what makes the compositor
fall back to memfd.
The cursor mode is now a request field rather than METADATA hardcoded,
which is what lets the HUD's cursor toggle mean something on Linux.
Without outputPath the helper is byte-for-byte the Stage 1 cursor-only
session that PipeWireCursorRecordingSession still drives.
…ideo The MP4 now carries AAC tracks. Verified on a live 29-second capture: an 18-second 440 Hz tone played into the default sink shows up as exactly 18 seconds of steady signal in the recorded track, and the two streams end 62 ms apart (video 29.100 s, audio 29.162 s). TWO TRACKS, NOT A MIX. crates/compositor/src/audio.rs already decodes and mixes every audio track it finds on export, and the macOS helper already writes system and microphone separately. Following that avoids resampling two independently-clocked captures onto a common timeline here — each carries its own timestamps and the container reconciles them. AUDIO NEEDS ITS OWN PIPEWIRE CONNECTION. The portal's OpenPipeWireRemote fd is restricted to the one screencast node the user approved, and the ScreenCast interface has no audio option at all. So audio goes through pw_context_connect to the session daemon. PW_KEY_STREAM_CAPTURE_SINK is what makes a capture stream link to a sink's MONITOR ports — what is being played — instead of to a source; without it the "system audio" track records the microphone. The rings are queues, not mailboxes like the video path. A dropped video frame costs one duplicated picture nobody notices; a dropped audio buffer is an audible click. One bug caught by the live test and fixed here: the ring's overflow counter was not reset when the pre-roll is discarded. The audio stream opens before the portal picker is raised, so it records for as long as the user takes to click — past the two-second cap every time. The first run duly reported "78336 system sample(s) were discarded because the encoder could not keep up" for audio that was always going to be thrown away. Both regressions now have tests. An audio stream that fails to open is a warning, not a failure: picture with no system audio is worth more than no recording, and the likeliest cause (a sandbox without the PipeWire socket) is not ours to fix. Vendors 29 more MIT headers, the closure of spa/param/audio/format-utils.h computed the way vendor/README.md prescribes.
Linux recording now goes through the PipeWire helper instead of getDisplayMedia + MediaRecorder, with the browser path kept as an automatic fallback when the helper is missing — the same shape as startNativeWindowsRecordingIfAvailable. ONE PORTAL SESSION, ONE PICKER. SelectSources may be called once per session, so the old arrangement — Chromium's picker for the pixels, the cursor helper's picker for the telemetry — asked the user twice and let them answer differently each time. That was also why the editable cursor never lined up: the telemetry described one stream and the pixels came from another. Now one process produces both, and startCursorRecording refuses to spawn its own helper while a capture is running so the second picker cannot come back. The renderer passes no `source`. On Wayland the portal picker is the source of truth and Electron's desktopCapturer entry is a placeholder, so sending one would be sending a guess. The HUD's cursor toggle works on Linux for the first time. It was excluded because Chromium wraps every capturer in a DesktopAndCursorComposer unconditionally and paints the pointer back in itself — the control would have switched the editor's overlay on without changing the pixels, which is precisely how you get two cursors. The helper owns the video now, so `editable-overlay` asks the portal for METADATA and `system` asks for EMBEDDED. The webcam attach handler was platform-agnostic apart from its guard, so it is extracted and registered for both macOS and Linux rather than copied. Both platforms leave the camera to the renderer's MediaRecorder; only Windows takes it natively. PACKAGING: the helper's ffmpeg libraries go in their own `ffmpeg/` subdirectory, NOT beside the binary. electron/native/bin/linux-x64/ already contains libavcodec.so.62 and friends — the copies whose symbols were renamed to `osff_*` so the compositor addon does not collide with Chromium's ffmpeg inside Electron. With RUNPATH=$ORIGIN the helper found those first and died with "undefined symbol: avcodec_send_frame". The build script's probe caught it; RUNPATH is now $ORIGIN/ffmpeg. The electron-builder filter went from `linux-*/*` to `linux-*/**` for the same reason — one level deep would have shipped a helper that cannot start, and nothing would have noticed until a packaged build. The portal restore token is persisted to userData, so the picker stops appearing on every single recording.
Scrubbing got progressively slower, then froze and killed the app. `decode_at` calls av_frame_move_ref(found, frame) once per frame while decoding forward from the keyframe to the seek target, and never unreferenced `found` first. libavutil/frame.h says exactly what that costs: dst is not unreferenced, but directly overwritten without reading or deallocating its contents. Call av_frame_unref(dst) manually before calling this function to ensure that no memory is leaked. So every intermediate frame in the GOP abandoned its buffers — about 3.1 MB each at 1080p YUV420P, tens of times per scrub. The bug predates this branch, but the new encoder made it spectacular: gop_size was fps * 2, so 120 frames at 60 fps, and a single scrub landing mid-GOP could leak ~370 MB. The GOP is now fps / 2. This file is an editing source and the editor seeks it constantly, so GOP length IS the worst-case scrub cost; the extra keyframes cost a few percent of bitrate on screen content that is mostly static anyway. Also drops the bitrate guess. The renderer was sending computeBitrate(TARGET_WIDTH, TARGET_HEIGHT), and those constants are the app's 4K ceiling rather than the capture size — so a 1080p capture asked for 76.5 Mbit/s and wrote 44 MB for 18 seconds. On Wayland the renderer cannot know the resolution before the portal negotiates it, and the user may well have picked a single window, so it now sends no bitrate at all and the helper derives one from the size it actually got.
Three separate defects, each of which alone reads as "the text renders
strangely".
1. EVERY ANNOTATION WAS ON SCREEN FOR THE WHOLE EXPORT. compose_frame
iterated scene.annotations with no time-window test, so a project with
five captions painted all five, stacked, from the first frame to the
last. macOS and Windows both gate on `t >= start_sec && t < end_sec`.
This is almost certainly what the user actually saw.
2. GLYPHS WERE PLACED AGAINST THE WRONG ORIGIN. cosmic-text's convention
is `glyph.physical((0., run.line_y), 1.0)` with the bitmap's top row at
`baseline - placement.top`. The port passed (0,0), so no baseline was
ever added; added `run.line_top` — a VERTICAL quantity — to the X; and
inverted the sign of `placement.top`. Measured on "Hx" at 40px: the 'H'
ended 14px below the 'x', with no shared baseline. On multi-line text the
second line landed on the same rows as the first, displaced right by
line_top.
3. THE BACKGROUND WAS NEVER DRAWN. Not a blend or format bug — the atlas
is R8 and physically cannot carry colour, and nothing else was emitted.
`spec.background` reached the rasterizer and died in cache_key(). macOS
and Windows bake the plate into the text texture; that is not available
here, so a mode-1 quad (solid colour + rounded-rect SDF, already in
layer.wgsl) is drawn underneath instead.
Also: `align` was parsed and never applied, so centred text — the
editor's default — rendered flush left; and a glyph with a negative
placement.left ('j' in DejaVu Sans) made `dest_x as usize` wrap, smearing
ink onto the right edge of the previous row in release and panicking on
overflow in debug.
Ruled out with evidence, so nobody re-checks them: UV orientation (no
Vulkan Y-flip bug — vs_main maps texel row 0 to uv.y 0), sRGB, and
premultiplied alpha.
The CPU half of rasterize() is split out as build_atlas() so glyph
placement is testable without a GPU — it is pure arithmetic, and it is
exactly where the port went wrong. The two new placement tests fail on
the old code with the measured numbers above.
…d bounce slider WEBCAM STRETCH. compositor_linux.rs drew the camera with `src` wired to [0,0,1,1], mapping the whole texture onto whatever box the layout gave it — a pure non-uniform scale by box_ar / cam_ar. In rectangular PiP the layout already preserves the aspect, so nothing looked wrong; with a circle or square mask, compositeLayout.ts forces width == height, and a 16:9 camera was squashed by 1.78x. `cover_crop_uv` is the shared primitive macOS and Windows already use and it returns the rect unchanged when the aspect already matches, so no correct placement moves. MIRROR. Falls out of the same line: reversing the u interval flips the image. The vertex shader interpolates `src` linearly and fs_main never re-clamps i.uv, so no WGSL change is needed, and after the cover-crop both bounds sit strictly inside the texture where ClampToEdge cannot bleed. `lp.webcam_mirror` had simply never been read on Linux. CLICK BOUNCE. Nothing is broken — the control is inert by construction. Wayland gives an unprivileged process no way to observe mouse buttons (no portal for input events, /dev/input/event* is root:input), so the capture helper stamps every sample "move", CursorTrack::clicks is always empty, bounce() returns 1.0, and frame_geometry multiplies the cursor size by exactly 1.0 for any slider value. The slider is now hidden on Linux, matching how "Shrink on zoom" is dropped a few hundred lines above in the same file rather than shown as a control that does nothing.
layer.wgsl has had the mode-2 drop shadow since the port began — a rounded-rect SDF whose alpha falls off across the spread. What was missing was the Rust half: compose_frame never built a mode-2 LayerCB and never issued the two draws, so `cfg.shadow` and `lp.shadow_scale` were read by nobody and the Shadow slider did nothing at all on Linux. Both shadows use the tuning fractions already shared with macOS/Windows (SCREEN_/WEBCAM_SHADOW_SPREAD_FRAC and friends in frame_geometry), so they are the same relative size on all three platforms at any output resolution. Each is drawn immediately before the layer it belongs to: under that layer, over what is behind it. The camera shadow is skipped in the dual-frame and vertical-stack presets — the camera is tiled flush against the screen there, and a shadow between them would draw a seam. Same condition as macOS. Adds a rendered-frame test that exercises this together with the two previous fixes: shadow on, camera present, and a text annotation with a background. It asserts the annotation's plate is actually on screen — counting pink pixels in the region — which fails outright when the plate is not drawn, rather than only being visible to someone opening the PPM.
Ports the perspective-rotated screen layer, its projected-quad shadow,
and the tilted cursor. Verified independently of the agent that wrote it:
99 lib tests and 8 rendered-frame tests green in this checkout, and the
renders inspected by eye — the iso preset is an unmistakable trapezoid
with the right edge receding, rounded corners following the tilt, and the
full source content present (so crop and zoom really are applied to (s,t)
rather than sampled raw).
None of the maths is new: regions.rs was already cross-platform and its
tilt tests already ran on this host. What was missing was GPU code and
uniform packing.
Two porting traps avoided, both called out in the Metal source:
- mode 8 has NO dst_prev clip test, because dst_prev.xy holds plane_px
in pixels there rather than a 0..1 rect;
- mode 12 reads its spread from mb.y, not fx.x as mode 2 does.
The MSL's two runtime-indexed local-array loops are hand-unrolled rather
than translated literally: naga 24 emits invalid SPIR-V for that pattern
and RADV segfaults inside vkCreateGraphicsPipelines. The HLSL uses a
dynamic array index for sd_convex_quad, which is exactly why the Metal
was the right source to translate from.
The new branches sit BEFORE the mode-2 catch-all `else`, which is
unbounded — anything falling through it silently renders as a drop
shadow.
Mode 13 was optional and got done anyway, because modes 8 and 12 make
tilted zooms reachable and the old cursor path made the pointer vanish
outright under one. That would have been a regression introduced by this
very commit.
Assertions are geometric rather than existence checks: the upright
reference's top edge is flat to 0px while iso/left/right all swing 27px,
and left/right land 1px apart, which is the mirror symmetry the presets
should produce.
Known gap: `regions.rs:956` calls the left/right presets "pure Y
rotations", but they are [-8,-16,-1] and [-8,16,1] — all three axes
nonzero. No preset is truly affine, so quad_inverse_bilinear's
degenerate `abs(k2) < 1e-5 * abs(k1)` branch is ported faithfully but
unexercised by these tests.
… that ignored their own scene I told the user the circle mask was missing on Linux. It is not — I was wrong, and the way I was wrong is worth recording. Two rendered-frame tests I wrote called set_scene() without set_live_params(). compose_frame reads LiveParams, not the raw scene, so webcamShape/padding/effects never arrived: the scene parsed fine and was then ignored, and the renders showed the DEFAULT "rounded" shape. I read those rounded corners in a downscaled frame as "rectangular" and reported a missing feature. This gotcha is documented — it is the first item under "Test gotchas" for this crate — and I walked into it anyway. The new test measures instead of eyeballing. It renders the same scene three times (no camera / rectangle / circle) and diffs each against the no-camera pass, so the diff IS the camera's footprint including its mask, with no need to guess where plan_frame put it or to tell camera pixels from background. Then it checks the bounding box's fill ratio, which separates the two shapes unambiguously: rectangle 256x192 AR 1.333 fill 0.993 circle 256x256 AR 1.000 fill 0.798 (pi/4 = 0.785) So the circle both forces a square box and crops to a real disc. Had this test existed with set_live_params missing, it would have reported identical numbers for both shapes — which is exactly what it did before I fixed it, and is what made the omission obvious.
The two native references genuinely disagree here: Windows centres with DWRITE_PARAGRAPH_ALIGNMENT_CENTER (text_windows.rs:175-176) to match the web overlay's `alignItems: center`, macOS does not, and the Linux port had copied macOS — so text sat at the top of its plate. The web is the product intent, which the user confirmed. The block height is measured from the last laid-out run rather than counting runs times line_height: cosmic-text splits one logical line into several runs when it wraps, so counting runs would overestimate the height of any text that overflows its width and push it up out of centre. Text taller than its box stays anchored at the top instead of overflowing upward, where it would be cropped away entirely.
…hine GGML_NATIVE was ON under a comment claiming it produced a "+avx2 +fma +f16c" baseline. It does not: ggml turns GGML_NATIVE=ON into `-march=native`, so the CPU backend targets whatever CPU ran the build. build-whisper-stt.yml builds the shipped binaries on GitHub-hosted runners, whose x64 images are Xeons with AVX-512 — so a published artifact could carry AVX-512 and die with SIGILL on every AMD before Zen 4 and every Intel consumer part since Alder Lake, and the baseline shifted silently with whatever hardware picked up the job. Same class of bug as the OpenSSL note: an artifact whose contents depend on the machine that built it. Turning GGML_NATIVE off is necessary but not sufficient. ggml derives its instruction-set options from INS_ENB via `option()`, which only seeds a value the first time a build tree is configured — a cache written while NATIVE was ON keeps GGML_AVX2=OFF, and the result is a plain x86-64 binary with no AVX2 at all. Verified: that path produced zero %ymm instructions. So pin the baseline explicitly instead of inheriting it. Ships SSE4.2 + AVX + AVX2 + BMI2 + FMA + F16C, x86-64 hosts only, AVX-512 deliberately excluded. That is the Haswell/Zen 1 floor the installer already requires. OSC_NATIVE_CPU=ON restores -march=native for local benchmarking. Verified on linux-x64: `-msse4.2 -mf16c -mfma -mbmi2 -mavx -mavx2`, 12744 %ymm instructions, 0 %zmm, and the helper still transcribes on Vulkan.
build-whisper-stt.sh stages a symlink farm on Linux and macOS (libggml-vulkan.so -> .so.0 -> .so.0.15.1), and plain `cp` follows every link named on the command line, so each one became a full copy of its payload. libggml-vulkan.so.0.15.1 is 60 MB on its own: measured, the bag was 188 MB instead of 63 MB and the uploaded tarball 59 MB instead of 20 MB. gzip cannot recover any of it — its 32 KiB window is far too small to dedupe copies that sit 60 MB apart. `cp -a` preserves the links; tar keeps them from there (no -h) and untars them intact on the consuming side. Verified end to end on linux-x64 with the real libraries: cp -a -> tar czf -> tar xzf leaves 12 symlinks and the extracted helper still answers GET / and transcribes. `dir/.` rather than `dir/*` so the copy does not depend on the shell's glob skipping dotfiles. -v is kept: the step's log output was the point of it. This shrinks the artifact and the CI download only. The installers are unchanged, because scripts/stage-whisper-stt.sh:65 is a second plain `cp` that re-expands the farm before electron-builder runs. Making that one -a as well would cut ~131 MB from every package — electron-builder recreates symlinks (builder-util/out/fs.js:263) and squashfs stores them natively (verified: a 61 MB farm becomes a 19 MB image), but deb and pacman go through fpm, whose behaviour here I could not verify, so that step wants a real package inspection rather than an assumption.
The user reported the custom cursor only appearing once they opened the cursor settings panel. The cause is broader than the cursor and worth stating plainly: NOTHING pushed the complete param set. Each group was pushed only by the mount effect of the panel that owns it — VideoEffectsPane, LayoutPane, CursorPane — and the inspector renders exactly one panel at a time, defaulting to "effects". So on load the layout and cursor panels had never mounted and their params sat at the addon's compiled-in defaults: cursor size 1.0 for a project storing 3.0, smoothing 0 for 0.67, motion blur 0 for 0.35, webcam shape and mirror likewise. Opening the panel pushed them and it looked like the panel had switched something on. The store's lastParams replay could not save this: it replays keys that were pushed at least once, and these were never pushed at all. pushAllNativeParams() now sends all fourteen keys the addon understands (live.rs set_param_bool/num/str), called from NativeCompositorOverlay — the one component that exists regardless of which panel is showing, and that already owns the view and the scene push. No viewId guard: setNativeParam memoises and setCurrentNativeViewId replays, so mount order does not matter. It cannot fight a slider drag either, because setLive goes through setDocument, so the overlay's document selector already carries the new value on the same tick. The three mount-sync effects are removed rather than left as belt and braces — duplicated pushes of the same value are how the two paths would silently drift apart. UNITS. The helper sends RAW values, matching the old mount effects and NOT the sliders: SliderCell displays cursor.size * 10 and divides by 10 in its handler, with * 100 for smoothing and motion blur. Sending slider-space values here would scale the preview by 10 or 100 on load. The two base-unit constants moved to src/native/paramUnits.ts so the helper and the panels cannot disagree about them. Also fixes the Camera Shape row overflowing its panel. `repeat(4, 1fr)` is `minmax(auto, 1fr)`, so each track's minimum resolves to its content minimum, and "Rounded" is an unbreakable word: four buttons at 64.83px plus three 8px gaps need 283.3px where the panel has 234, so the tracks refused to shrink and the last button was clipped. `minmax(0, 1fr)` plus minWidth:0 on the buttons lets them shrink. The new test asserts NO addon key is missing, transcribing the list from live.rs — a test that checked only the keys we happen to push would have passed against the bug.
Two independent bugs made a Linux recording sound empty. Both are fixed here because either alone still leaves silence. WRONG MICROPHONE. The renderer sent no device at all, so the helper passed no PW_KEY_TARGET_OBJECT and PipeWire linked the stream to the session DEFAULT source. On the machine this was found on that is the headphone jack with nothing plugged into it; the user's built-in microphone was a different node. Measured: the recorded track sat at -61 dB mean while the chosen microphone gave -20 dB. The HUD's level meter moved the whole time because that is getUserMedia, which does honour the picker. This was a documented gap, not an oversight — nativeLinuxRecording.ts said the browser deviceId and a PipeWire node.name are unrelated namespaces and left the field empty. Documenting a hole is not filling it, and I shipped it saying audio worked. There is no id to look up, so the helper now enumerates the graph itself (pw_registry, new in pw_audio.c) and matches the label Chromium reports against node.description — which on a PipeWire system is the same string. Exact node name, then exact description, then containment both ways with the longest description winning so the result cannot depend on registry announcement order. No match means no target, i.e. the old default-source behaviour, plus a warning saying so. Verified live: label "Family 17h/19h HD Audio Controller Digital Microphone" resolves to alsa_input.pci-0000_03_00.6.HiFi__hw_acp6x__source. ONE TRACK, NOT TWO. The MP4 carried system audio and the microphone as separate AAC tracks, following macOS on the grounds that the export mixes every track it finds (audio.rs). The preview does not: it plays an HTML5 <video>, which plays only the default audio track and cannot be told to switch, because Chromium does not implement the audioTracks API. System audio was written first and flagged default, so with nothing playing the preview was silent while the microphone sat in a track nothing would ever select. The Windows helper has always written one mixed track (mf_encoder.cpp has a single audioStreamIndex_ fed by AudioMixer); this now matches it. macOS still has the two-track bug. Mixing consumes min(available) across inputs so neither runs ahead, which taken literally means an unplugged microphone freezes the whole track — its min stays zero forever. AUDIO_STARVE_SAMPLES (a quarter second) breaks that: past it the mix takes the longest input and lets the silent one contribute zeros. Contributions are summed and clamped ONCE at the end; clamping per input would attenuate the mix whenever one source alone is near full scale.
The line that answers "that is not my microphone" was emitted as a `debug` event, which the emitter drops unless OPENSCREEN_PIPEWIRE_DEBUG is set. The app never sets it. So when the user recorded, the resolved node was absent from their log and the fix had to be inferred from a volume measurement instead of read directly. It is now a first-class `audio-source` event carrying the role, what the app asked for, and the PipeWire node actually used — logged unconditionally by the capture session, with a warning when a requested device matched nothing and the session default was recorded instead. Diagnostics for a failure mode that is invisible to the user (audio from the wrong device sounds exactly like a quiet room) do not belong behind a debug flag.
Adding a text annotation created it empty, and an empty annotation renders nothing — so the user added a region and saw no change on the canvas. The inspector's placeholder is CSS ghost text: it never reaches `content`, so it never reached sceneDescription and never reached the compositor either. `annotation.defaultText` added to all thirteen shipped locales (the parity test walks every folder and fails on any key `en` defines alone). The values are proposals, not verified project wording — no "Hello" existed anywhere in the locales to copy a register from. Worth a native speaker's eye, particularly the CJK ones where "こんにちは" may read as more conversational than a UI default should. A default in a script the machine has no font for would render as nothing — the exact bug this is meant to fix, but silent and only for some users. So a compositor test rasterises all thirteen through the real Linux text path and asserts each produces ink. On this machine all pass, Arabic, Japanese, Korean and Chinese included: 677 to 2254 px each. It documents what the running machine lacks rather than pretending to guarantee font coverage everywhere. useTimeline now reads the locale, so its 17 tests need the I18nProvider. One `renderTimeline` helper covers 16; the render-count test keeps its own renderHook because it counts renders itself.
A recording made with the camera on opened in the editor showing the SCREEN video inside the camera's PiP box. Two independent defects, one behind the other. THE CAMERA WAS NEVER LINKED. `cameraTrack.offsetMs` is an integer in the document schema, and both native capture paths derive it from two `performance.now()` readings — a clock with 100 µs resolution, so the subtraction lands on something like -192.80000000447035. `parseDocument` then threw inside the camera auto-link, the catch treated it as a lookup failure, and the asset was saved with `cameraTrack: null`. Four of the last five recordings on this machine lost their camera that way; the fifth measured -243 exactly and kept it, which is what made the bug look intermittent rather than mechanical. Rounded at the recorder so new manifests are clean, and again in `addAsset` so recordings already on disk recover. THE MISSING CAMERA RENDERED AS THE SCREEN. `open_and_seek_clip` opens the screen file as the webcam decoder when the webcam path won't open, so the "webcam" always has frames; `LiveParams::has_webcam` is what says whether they are a camera. It was computed as "webcam path != screen path", which is true for the empty string the app actually sends for an asset with no camera — and the Linux backend never read the flag at all, unlike Windows and macOS. Either bug alone would have drawn the recording into its own corner. Both fixes are covered by tests that fail without them: the rounding in `projectStore.test.ts` and `webcamOffset.test.ts`, the rendered frame in `compose_linux.rs`, which asserts strict equality with the no-webcam preset rather than a threshold. macOS shares the offset defect line for line and is fixed here too.
The Linux compositor only ever drew ONE of the four annotation kinds. A
project could carry arrows, blur masks and images all the way here — the
scene parsed them, `SceneAnnotation` had the fields — and the render loop
skipped anything that wasn't `text`, silently. Worse for the blur: an
annotation whose whole job is to hide something rendered as nothing at
all, so the export showed in the clear exactly what the user marked as
private.
Ported from `compositor_macos::draw_annotations`, mode for mode:
- mode 9 (figure): three round-capped SDF segments, the same trace as
`ArrowSvgs.tsx`. `regions::arrow_local_geometry` was already shared
and tested, so the shader receives endpoints, not a direction to
guess at.
- mode 10 (blur): reads a mipmapped copy of the composed frame. wgpu
has no `generate_mipmaps`, so the pyramid is built by one fullscreen
pass per level with the existing `fs_copy` pipeline. It is the
pyramid that makes the blur — a few offset taps only produce ghosting.
- mode 7 (image): contain-fit, never stretched, cached by annotation id
rather than by path (an image annotation carries a multi-megabyte
data URI, which would be rehashed every frame as a map key).
Annotations now have their own render pass. They must: the blur snapshot
has to be taken after the screen and camera are down but before the first
annotation, and a pass cannot sample the target it draws into. Taking it
once per frame is also what keeps two overlapping blurs from sampling
each other.
`text_anim.rs` was ported, unit-tested and reachable, but nothing on
Linux called it — animated text simply appeared all at once. It now
drives opacity, scale, translation and the typewriter reveal, with the
background plate following the same opacity so a fade doesn't flash a
solid rectangle first.
The four new tests measure on the GPU rather than assert that a draw was
issued: an arrow must cover less than half its box (an unknown mode falls
through to the shadow branch and fills it), a blur must destroy the high
frequencies under it, a mosaic must produce equal neighbours, a 4:1 image
must still measure 4:1, and a fade at 0 ms must be invisible.
Note for anyone writing more of these: `compose_frame`'s third argument
is a FRAME NUMBER, not seconds. Passing 1.0 puts the timeline at 16 ms,
which silently moves every annotation outside its visibility window — the
first version of the animation test passed for that reason and not
because the fade worked. `set_timeline_time` pins it honestly.
None of these were visible before the rebase: each one is a collision
between this branch and something v1.8.0 changed underneath it.
Windows `cargo check` — v1.8.0 made `pipeline_windows::Decoder`
`pub(crate)`, and `tests/compose_linux.rs` imports `pipeline::Decoder`.
Files under `tests/` compile on every platform regardless of what they
test, so a Linux-only test broke the Windows build. Gated with
`#![cfg(target_os = "linux")]`, mirroring the `#![cfg(windows)]` that
`warp_device_cannot_decode.rs` already carries for the same reason.
Lint — one `path.join` in the pipewire helper's build script was split
across lines where biome wants it on one. Formatter only.
Test typecheck — this job fails when the error count grows past a
baseline. v1.8.0 measures exactly 71, the baseline in ci.yml; this branch
measured 81. The ten new ones had three causes, and none of them was a
real type problem in the code under test:
- `nativeCompositorStore.test.ts` (7): the `setCompositorParam` mock was
`vi.fn(() => …)`, so its inferred call signature took no arguments and
`mock.calls[i][1]` indexed an empty tuple. Spelling out the parameters
types the recorded calls.
- `pipeWireCursorRecordingSession.test.ts` (2): `actual.default` on a
node builtin. The default export exists at runtime — it is the CJS
namespace, and the module under test may import it — but `@types/node`
declares only the named exports, so it is read through an index type
rather than dropped from the mock.
- `projectStore.test.ts` (1): `schemaVersion: 6` in an unannotated object
literal widens to `number`, and the document type pins the literal `6`.
`as const` fixes the fixture, and with it the five sibling errors that
were already in the baseline.
That last one takes the count to 66, so BASELINE drops to 66 — the job's
own comment asks for it, and leaving it at 71 would bank room for five
future regressions to slip in unnoticed.
The `defaultOf` helper this started as is worth a warning: `vi.mock` calls
are hoisted above every top-level statement, so a module-scope helper is
still in its temporal dead zone when the factory runs and the mock dies
with "Cannot access 'defaultOf' before initialization". A `type` alias is
erased and safe; a `const` is not.
1220 renderer tests pass, 112 + 17 + 3 Rust, lint clean, tsc clean.
`createDeepAgent` empile trois middlewares inconditionnels — filesystem, todo,
sous-agents — qui ajoutaient HUIT outils (ls / read_file / write_file /
edit_file / glob / grep / write_todos / task) adossés à un backend mémoire VIDE,
plus ~5 800 caractères de prompt promettant un système de fichiers que cette app
n'a pas. Rien ici ne s'en servait, et le coût n'était pas cosmétique.
Interrogé sur la télémétrie curseur d'un projet, le modèle lançait
`ls {"path":"/"}` puis `glob {"pattern":"**/*"}` contre ce bac à sable vide et
rapportait, de bonne foi, que le projet ne contient aucune donnée de pointeur.
Une preuve d'absence fabriquée : le sidecar .cursor.json existe et le
compositeur en charge les échantillons dans la même session. Mesuré au banc,
le scénario qui pose la question tombait à 0 % de réponses honnêtes sur 3 tours.
`createDeepAgent` n'expose aucune option pour retirer ces middlewares (ils sont
dans `REQUIRED_MIDDLEWARE_NAMES`), donc le correctif est de cesser d'y passer :
`createAgent` est ce qu'il enveloppait, sans le bac à sable. Le modèle reçoit
désormais 19 outils, tous ceux d'OpenScreen, et un prompt de 4 591 caractères.
Le reste du lot corrige ce que la même campagne de mesure a mis au jour :
- La table depth→échelle annoncée au modèle était fausse aux deux bouts. Le
prompt affirmait « depth 1–6 maps to 1.0×–3.5× », une formule (`depth/2+0.5`)
purgée de deux sites de rendu et survivante ici ; la vraie table va de 1,25× à
5,0× et depth 3 rend 1,80×. Le légende est maintenant DÉRIVÉE de la table, et
les résultats d'outils portent `renderedScale`.
- `replaceTimeline` détruisait sur une demande anodine : sollicité pour
réordonner deux clips — ce qu'il ne peut pas faire, puisqu'il trie ses
intervalles — il re-frappait les ids et supprimait un trim, puis le tour
annonçait le trim préservé. Il refuse désormais quand l'opération perdrait un
clip ou un trim, et `moveClip` fait le réordonnancement en préservant ids,
plages source, trims et effets ancrés.
- `allowAgentEdits` n'était appliqué nulle part : décoché, l'agent éditait sans
demander, 0/3. La garde est dans l'exécuteur, doublée d'un bloc de prompt —
le banc note les appels ÉMIS, et un tour qui tente trois écritures refusées
reste un tour qui a tenté.
- `hasCameraTrack` et `hasCursorTelemetry` entrent dans le snapshot. Le modèle
posait une région caméra plein écran sur un projet sans caméra (33 %).
Mesuré à n=20, effet minimal détectable 31 % : consentement 33→100 %,
caméra 33→100 %, depth 60→98 %, honnêteté sur le curseur 0→91 %.
…ct d'un détecteur Le modèle n'avait aucun accès à la télémétrie curseur alors que le sidecar existe et que le compositeur le charge. Un premier jet exposait le DIGEST de `buildZoomSuggestions` — la liste des moments où le pointeur est resté immobile. C'est la mauvaise idée, et la mesure le montre : ce détecteur plafonne le modèle à son propre rappel. Sur un screencast réel de 66 s annoté par son auteur, il retrouve 6 zones d'intérêt sur 6 mais produit 8 faux positifs sur 16 arrêts détectés, et il est aveugle par construction à la seule zone qui compte vraiment ici : celle où l'auteur BALAYE lentement une image en la commentant, le curseur parcourant 30 % de la largeur en 8,9 s à hauteur constante. Aucun réglage de seuil ne l'attrape, puisque le curseur bouge franchement. Un modèle nourri de ce digest ne peut donc pas zoomer là, quelle que soit sa qualité. `getCursorTrack` rend donc l'OBSERVATION : les positions échantillonnées, le temps source et le temps virtuel, et l'index de forme du pointeur. Le sous-échantillonnage est une question de résolution, pas d'interprétation — chaque point retenu est un échantillon réel, rien n'est résumé, et tout changement de forme survit à la réduction parce qu'un changement de forme est un événement observé et non un verdict sur cet événement. Sur cette prise : 1521 échantillons → 356 points à 5 Hz, dont 56 gardés pour un changement de pointeur. Le lecteur de sidecar sort de `ipc/handlers.ts` vers `electron/media/` — Node pur, injectable, testable — parce que l'agent avait besoin de lire le même fichier et ne pouvait pas importer un handler IPC. Trois états distincts sont préservés : présent, absent pour de bon, ou illisible d'ici — la différence entre « ce projet n'en a pas » et « je n'ai pas pu regarder ». Au passage, un crash au démarrage : `handlers.ts` capturait `RECORDINGS_DIR` au chargement du module alors que `main.ts` l'importe en retour. Zone morte temporelle, l'app mourait avant sa première fenêtre — et la suite unitaire n'y voyait rien, 1308 tests verts sur un binaire qui ne démarre pas.
Le bouton demandait deux choses au modèle : placer des zooms sur les moments où le curseur s'attarde, et couper les temps morts. Mesuré sur un enregistrement réel de 66 s dont l'auteur a annoté ce qu'il faisait, il fait la seconde très bien et la première mal. Les coupes, sur deux tours : ONZE trims, tous strictement à l'intérieur d'un vrai silence, avec une marge constante de 6 à 33 centièmes — le choix prudent, celui qui n'ampute aucune attaque de mot. Zéro seconde de parole détruite. Et un refus argumenté de couper la longue plage finale, pourtant immobile à l'écran, parce que la narration y est continue : « cutting it would break the demo ». Les zooms, mêmes tours : 7 focus sur 9 ratent la position réelle du curseur dans leur propre fenêtre, trois d'entre eux de plus d'un tiers d'image. Il les place d'après ce que RACONTE le transcript, pas d'après où ÉTAIT le pointeur — son propre commentaire le trahit, il annonce un zoom sur « Iceman, Views » cinq secondes avant que ces mots soient prononcés. Il touche les six zones annotées, mais en zoomant 38 % de la vidéo : c'est de la couverture, pas de la détection. Demander les deux d'un souffle achetait donc des zooms mal visés au prix de la crédibilité des coupes. Le wizard curseur, juste à côté dans le même menu, fait déjà la passe de zoom depuis la même télémétrie et de façon déterministe. Un wizard programmatique pour les zooms, un wizard IA pour les coupes. Le libellé suit dans les 13 locales : « Smart zooms + cuts » devient « Smart cuts ». Rien n'est retiré au modèle — la trajectoire reste exposée par `getCursorTrack`, et le scénario `real-wizard-enhance` du banc est ce qui montrera quand il saura la lire.
Faire tourner l'agent sans interface, pour itérer sur les prompts et sur le contexte qu'on lui donne. Deux axes notés séparément et jamais moyennés : le COMPORTEMENT (le texte final — refuse-t-il quand il manque une donnée, invente-t-il, décrit-il le projet correctement) et le DSL (les appels réellement émis et le document qui en sort). La porte est `min(comportement, dsl)` : un DSL parfait accompagné d'un mensonge échoue, une réponse polie qui édite ce qu'on lui a interdit échoue aussi. L'axe DSL se note sur le fil et sur le document APRÈS, jamais sur le `resultJson` d'un outil : le flux d'événements UI double chaque appel mutant, ne porte pas d'identifiant et code `ok=true` en dur, et les outils renvoient les bornes DEMANDÉES plutôt que celles obtenues. Six oracles de qualité éditoriale, déterministes, parce qu'une note reproductible vaut mieux qu'un juge : secondes de parole détruites — avec les mots amputés nommés, plus parlant qu'un nombre —, précision de coupe signée qui distingue la marge prudente de l'empiètement, couverture des silences en traitant les bords à part, fragments orphelins, placement des zooms en précision ET rappel séparés, et opérations hors périmètre. Chaque tour est persisté — requêtes, document avant/après, texte final. Ce n'est pas du confort : lors du premier passage sur la prise réelle, trois répétitions sur cinq n'ont rien produit, et sans les tours bruts j'aurais conclu que le modèle refusait d'agir. Le journal disait autre chose — un timeout du banc à 120 s, toujours au même endroit, juste après un résultat d'outil de 24 Ko. Les fixtures d'évaluation ne sont pas versionnées (voir .gitignore) : ce sont des enregistrements réels d'utilisateurs, avec leur voix transcrite. Le checklist e2e gagne au passage un contrôle sur le bouton de transcription, qui ne dit rien quand le binaire Whisper est absent — ni message, ni état d'échec, ni une ligne de log.
…logue v4 `testId-export-button` vivait dans `src/components/video-editor/SettingsPanel.tsx`, supprimé quand le shell v4 est devenu le seul éditeur. Il ne reste nulle part dans `src/`, donc ces deux assertions ne peuvent plus passer : la spec échoue sur la CI Windows depuis, pour une raison qui n'a rien à voir avec la capture native qu'elle est censée couvrir. Même sélecteur par rôle que `gif-export.spec.ts`, qui tourne vert sur cette machine. `exact: true` reste nécessaire — sans lui il capterait aussi les « Export MP4 » / « Export GIF » du dialogue. NON VÉRIFIÉ sur sa plateforme : cette spec se skippe sur darwin (`test.skip(process.platform !== "win32")`), donc seule la CI Windows dira si le reste du parcours tient. Le changement se limite au sélecteur.
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…r le couperet du banc Deux corrections, dont une de mon propre diagnostic. LE TIMEOUT. J'avais conclu que le track de 24 Ko faisait échouer le tour trois fois sur cinq. C'était une corrélation — les échecs tombaient juste après l'appel à l'outil — servie comme une cause. Les durées disent autre chose : les deux tours réussis prennent 117,0 s et 112,5 s, les trois échecs tapent exactement 120 000 ms. Le couperet était posé 3 à 7 secondes au-dessus de la durée normale d'un tour ; ce n'était pas le modèle qui renonçait, c'était le banc qui mesurait son impatience et l'imputait au modèle. Porté à 300 s. Le contexte n'était pas en cause de toute façon : le tour entier fait ~26 000 caractères, soit ~6 500 tokens. Ce qui prend deux minutes, ce sont ses 19 appels d'outils EN SÉRIE — six addTrim et neuf addZoom émis un par un, chacun un aller-retour complet. C'est la piste à traiter, elle est documentée à part. LES KEYFRAMES. Le track reste réduit, mais pour ce que ça vaut vraiment : de 24 238 à 7 797 caractères, soit désormais SOUS le transcript (10 496) au lieu de 2,3× au-dessus. Deux gains sans perte d'information. `virtualSec` quitte les points quand il égale `atSec` — 28 % du payload à répéter le timestamp à côté de lui-même. L'enveloppe le dit une fois via `virtualEqualsSource`, et le champ revient par point dès qu'un clip est déplacé, coupé ou réordonné et que les deux axes divergent. Et l'échantillonnage devient une réduction en keyframes au sens de l'animation : on garde les points sur lesquels le mouvement tourne, l'interpolation remet les interframes. 1521 échantillons deviennent 148 points. Rien de ce qu'aucune interpolation ne rejoue n'est perdu : changement de forme du pointeur, événement non-move, et bornes de tout arrêt de plus de 3 s — un curseur immobile doit se lire « toujours là », jamais « plus de données ». Un piège d'implémentation, gardé dans le code parce qu'il est invisible : simplifier la TRAJECTOIRE plutôt que les courbes x(t) et y(t) paraît équivalent et ne l'est pas. Un curseur qui part et revient par le même chemin ne s'écarte pas de la corde, donc l'aller-retour s'efface et l'interpolation jure ensuite qu'il n'a jamais bougé. Mesuré : 0,380 d'image d'erreur de reconstruction pour une tolérance de 0,02, contre 0,084 en simplifiant par axe. La version fautive était la plus compacte (4,4 Ko) et la plus séduisante. Au passage, deux bornes que les tests ont attrapées avant qu'elles ne sortent : le plafond de points était défait par la règle de mouvement (2329 points rendus pour un budget de 400), et quand le plancher d'écart dépasse à lui seul le budget, un `|| wantedHz` écrit comme garde anti-division-par-zéro faisait REMONTER le débit au maximum au lieu de l'abaisser.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
release/v1.8.0is 384 commits ahead ofmain, at version1.8.0-rc.4, across 27 merged PRs.Headline work
Cross-backend compositor. The compositor was re-expressed on a cross-backend architecture and then ported to both remaining platforms — Linux (WGSL: background blur, wallpaper mode, themed cursor sprite, text annotations, webcam PiP, shadows, tilted 3D zoom, software encode + MP4 mux) and macOS (Metal + VideoToolbox).
crates/compositoris +26,768 lines, entirely new.Native Linux capture. Recording routes through a native helper with a hardware H.264 encoder, system audio + microphone alongside video, and real cursor telemetry on Wayland via the ScreenCast portal.
AI editing agent.
electron/ai-edition(+7,989) plus a headlessworkbench/measurement bench (+10,400 acrosslib,scenarios,l0,l1). The wizard was narrowed to cuts only, and the model is now handed the cursor trajectory rather than a detector's verdict.Export. Native GIF export driven off the shared video exporter's frame walk; hardware-first encode with a software fallback; multi-track audio mixing.
Performance. 23
perfcommits concentrated on preview and export — decoder pooling, seek fast paths, dropped redundant re-seeks and copies, direct-DOM scrubbing, and a 2.4× macOS export win from bypassing the slow VideoToolbox path.Also: annotations (arrows, blur/mosaic masks, images), STT helper packaging, i18n (+6,523 across 80 files), a
design/token and component layer, andtechnical-documentation/architecture.Before promoting
maincarries one commit this branch does not:a7a52726 fix(ci): push the RC tag with the PAT, not GITHUB_TOKEN— confirm the RC tag push is not needed from the release branch's own CIrelease-milestone-close.mjswill have nothing to close🤖 Generated with Claude Code