fix(rtmg/web): re-dispatch restored LoRA strengths to the engine on session ready#276
Closed
qianghan wants to merge 3 commits into
Closed
fix(rtmg/web): re-dispatch restored LoRA strengths to the engine on session ready#276qianghan wants to merge 3 commits into
qianghan wants to merge 3 commits into
Conversation
…ing an outage doesn't keep playing stale When the WS drops and the user switches to a different audio source before the auto-reconnect completes, the recovered session kept playing the previously-loaded track even though the UI showed the new selection. Root cause: the reconnect path (useStartSession) rebinds the fixture snapshotted at session start. The mid-outage track change is dropped because useFixtureSwap.run() bails while status !== "ready", and nothing re-applies it once the session recovers — so the recovered backend + AudioPlayer stay bound to the stale track while the perf store's fixture shows the new pick. Fix: track the live session's bound fixture in useSessionStore (boundFixture), set on initial connect and reconnect. useFixtureSwap now reconciles on the "reconnecting" -> "ready" edge: if the selected fixture diverged from the bound one, it re-runs the swap exactly once. No effect on a fresh Play or a clean reconnect where the selection never changed. Adds needsFixtureReconcile() + a focused unit test pinning the decision matrix, and asserts boundFixture clears on session reset. Co-authored-by: Cursor <cursoragent@cursor.com>
… clip A fast knob sweep landed as a raw discontinuity on consecutive generations (the runner reads the knob target once per tick and the backend turns it straight into per-inference-step values), driving the audio into distortion/clipping. There was no hysteresis or rate-limiting on control changes. Add an engine-side per-knob slew limiter at the once-per-tick knob read (ACEStepBackend.read_knobs), so a large jump ramps over a few ticks instead of snapping. Applied behind the GeneratorBackend seam, it protects every transport (web, VST, MCP, headless) with one implementation rather than per-UI tweens. Contract-first: the rate ceiling is registry metadata (KnobSpec.slew_max_per_s, projected into the /api/knobs catalog and bumped to KNOB_SCHEMA_VERSION 2). Default is range-relative (DEFAULT_SLEW_FRACTION_PER_S = 3.0, a full-range sweep ramps over ~0.33s) and applies only to continuous float knobs; int/enum/bool knobs (seed, step count, modes, toggles) and non-registry keys (curve specs, the playback clock) pass through verbatim, so the rebuild signature and discrete controls are unaffected. A slew limiter caps only the rate of change, so normal/slow adjustments incur zero added latency. hint_strength and x0_target now read from the slewed knob dict instead of the backing KnobState directly, so the whole translation path honors the limit. Regenerated packages/demon-client/types/wireContract.gen.ts; added tests/unit/test_knob_slew.py (bounded per-tick deltas on a jump, discrete/unknown passthrough, no-lag slow changes, dt cap, catalog projection). Co-authored-by: Cursor <cursoragent@cursor.com>
…ession ready A saved-session resume restores the persisted LoRA strength into useLoraStore.strengths (which the fader UI reads, so it shows e.g. 1.0) but never into perf.sliderValues — the value useParamSync actually ships to the engine each tick. useParamSync prefers sliderValues and only falls back to lora.strengths when the lora_str_<id> key is ABSENT, so the boot-time catalog seed's stale entry pins the engine to the old strength: the LoRA shows its restored strength but stays inaudible until a fader drag finally writes sliderValues through the dispatcher. Add reconcileEnabledLoraStrengths(), which seeds the engine-facing slider value of every enabled LoRA from the store strength, and call it once per session-ready edge (initial connect + reconnect) in useStartSession. Idempotent on a fresh session (values already agree → no extra server refit); gated on the ready path so it never fires before the engine is connected and never spams per-tick dispatches. No contract registry touched, so no wire-type regeneration is needed. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the tracker bug "Web app bug: LoRAs not loading upon refresh / session reload."
Repro (from the tracker): set a LoRA strength to 1, save the session, reload it. The fader still shows 1, but the audio doesn't sound like the LoRA is applied — it only "kicks in" once you wiggle the fader (drop to 0.8, raise back to 1).
Root cause
LoRA strength has two representations in the web client:
useLoraStore.strengths[id]— the source of truth the fader UI renders.usePerformanceStore.sliderValues["lora_str_<id>"]— the engine-facing value thatuseParamSyncactually ships to the pipeline every 8 ms tick.useParamSyncpreferssliderValuesand only falls back tolora.strengthswhen thelora_str_<id>key is absent:The boot-time catalog seed (
useLoraStore.setCatalog→seedLoraSliderValue) writes asliderValuesentry for the default-on LoRAs. A saved-session resume then restores the persisted strength intolora.strengthsonly — it never updatessliderValues. So at resume the key is present-but-stale:useParamSynckeeps streaming the old seeded value and never reads the restored one. The slider shows 1.0 (readslora.strengths) while the engine stays pinned to the stale strength.A fader drag is the first thing that writes
sliderValues(via the dispatcher's debounced commit), which is exactly why the LoRA only becomes audible after the user moves the fader. Symptom = "shows 1.0 but inaudible"; true cause = the restored strength is never pushed into the engine-facingsliderValues.This is a state-restoration / initial-sync gap, not a number-persistence gap — the number persists fine.
Fix
reconcileEnabledLoraStrengths()(engine/lora/dispatcher.ts): for every enabled LoRA, seedperf.sliderValues["lora_str_<id>"]fromlora.strengths[id].useStartSession— the initial connect path and the reconnectonSuccesspath — right after the LoRA cap is finalized.Why this is regression-safe
sliderValuesandlora.strengthsalready agree (both seeded from the same config/defaults), so every write lands the same value. The server only refits on a> 0.02delta, so re-seeding the matching value triggers no extra refit.remote.sliderValuesduring a fader drag.knobs.py/protocol.pychange), so nogen_wire_types.pyregeneration is required and the drift guards are unaffected.Test plan
New focused unit test
tests/unit/loraStrengthReconcile.test.ts:sliderValuesresults in the engine receiving 1.0 on ready (mirrorsuseParamSync's exact selection logic);Evidence:
Note:
tests/unit/float16.test.tsandtests/unit/sliceEpoch.test.tsfail locally withTypeError: Math.f16round is not a function— a pre-existing Node-version limitation in the float16 wire helpers, unrelated to this change (no LoRA/param code involved).Made with Cursor