Skip to content

qFromCharPerActuator: derive Q[3,3] (sigma_do_freq) from char ADEV#83

Open
bobvan wants to merge 2 commits into
mainfrom
bravo/q-from-char
Open

qFromCharPerActuator: derive Q[3,3] (sigma_do_freq) from char ADEV#83
bobvan wants to merge 2 commits into
mainfrom
bravo/q-from-char

Conversation

@bobvan
Copy link
Copy Markdown
Owner

@bobvan bobvan commented May 29, 2026

Summary

The foundational fix Main re-sequenced to first: make Q[3,3] (sigma_do_freq) honest by deriving it from the freerun characterization, so the 9.2 ns band-aid can be dropped and √(P22) becomes trustworthy for the OCXO gate, the longTauGnssCoupling coast-cap, and the derived-convergence state.

The bug: derive_do_process_noise derived sigma_do_freq only from an adjfine source — which do_freerun_char.py never writes. So for every real freerun char, Q[3,3] silently fell back to the 0.01 default while only Q[2,2] (phase) was derived. Q[3,3], not Q[2,2], is the coast/convergence knob (the freq-state uncertainty integrates into phase: Var_φ ≈ q_f·τ³/3) — confirmed three ways (Main's analysis, my routedQErrArm tick-discrimination coupling, and Charlie's closedLoopServoSim diverging deterministically on it).

The fix (do_state.py): when there's no adjfine, derive from the rising-ADEV (RWFM) tail:

RWFM Allan variance:  σ_y²(τ) = q·τ/3,  q = Q[3,3] (ppb²/s)
  ⇒  sigma_do_freq_ppb = √3·ADEV(τ)·1e9/√τ

evaluated at the largest available τ (RWFM dominates the long-τ tail; where it doesn't, ADEV(τ) sits above the true RWFM contribution → over-estimates Q[3,3], the safe direction). White-FM↔Q[2,2] / RWFM↔Q[3,3] matches the canonical two-state clock model. adjfine stays primary when present.

Engine: removes the universal 0.92 literal for a per-actuator-structured fallback (behavior-preserving — deliberately not inventing a pessimistic uncharacterized-OCXO constant; that's a sim-validated follow-up). Adds a DOFreqEst Q: summary log showing phase/freq values + sources.

Lab impact (Main coordinates)

  • PiFace / clkPoC3 (have freerun chars w/ ADEV): Q[3,3] now derives from ADEV instead of 0.01 on next run — the intended honest-Q change. For a quiet OCXO this is tighter than 0.01, which is correct: an honestly-quiet DO is allowed to coast longer; divergence only happens when Q is dishonestly small vs the true noise.
  • MadHat: no freerun char yet → keeps the fallback. Run do_freerun_char on OCXO-33, then test-then-drop the 9.2 ns band-aid (EFC saturation, the root it masked, is already fixed).

Test plan

  • 7 new tests (RWFM formula, largest-τ selection, adjfine precedence, non-finite/non-positive skipping, no-ADEV fallthrough, ADEV from a non-chosen phase source, realistic freerun-char schema). Full suite 1387 passed.
  • closedLoopServoSim A/B (Charlie): with honest Q[3,3] from ADEV, confirm the >60 s coast no longer diverges — then wire coast_cap_from_p22 (PR longTauGnssCoupling: coast-cap + graded-taper primitives (unit layer) #80) on the now-honest √(P22).
  • MadHat: do_freerun_char on OCXO-33 → honest Q → drop 9.2 ns band-aid (Main, lab).

Sequencing: this lands first → coast-cap wiring (PR #80) → sim validation.

🤖 Generated with Claude Code

bobvan and others added 2 commits May 29, 2026 11:43
Closes the gap that made the 9.2ns band-aid necessary.  Per Main's
analysis (3x independently confirmed, incl. charlie's closedLoopServoSim
which diverges deterministically on it): coast/convergence P-growth is
governed by Q[3,3] (frequency), NOT Q[2,2] (phase).  During a coast the
freq-state uncertainty integrates into phase (Var_phi ~= q_f*tau^3/3),
so an under-set Q[3,3] makes the filter overconfident and diverge.  The
band-aid inflated the WRONG knob (phase) as a crude proxy.

THE BUG: derive_do_process_noise derived sigma_do_freq only from an
"adjfine" source, which freerun characterizations (do_freerun_char.py)
never write.  So for every real freerun char, Q[3,3] silently fell back
to the 0.01 default while only Q[2,2] (phase) was derived.

FIX (do_state.py): when there's no adjfine source, derive
sigma_do_freq_ppb from the char's rising-ADEV (RWFM) tail:

    RWFM Allan variance  sigma_y^2(tau) = q*tau/3,  q = Q[3,3] (ppb^2/s)
    => sigma_do_freq_ppb = sqrt(3)*ADEV(tau)*1e9/sqrt(tau)

evaluated at the largest available tau (RWFM dominates the long-tau
tail; where it doesn't, ADEV is above the true RWFM contribution so
this over-estimates Q[3,3] rather than under — the safe direction, and
the longTauGnssCoupling coast-cap bounds the coast on the same honest
sqrt(P22)).  adjfine stays the primary source when present.

The phase/White-FM <-> Q[2,2] and RWFM <-> Q[3,3] split matches the
canonical two-state clock model.

ENGINE: removes the universal 0.92 ns literal in favor of a
per-actuator-STRUCTURED fallback (behavior-preserving for now — NOT
inventing a pessimistic uncharacterized-OCXO constant; that's a
sim-validated follow-up).  Adds a "DOFreqEst Q:" summary log showing
phase/freq values + sources (char-derived vs fallback).

LAB IMPACT (main coordinates): PiFace/clkPoC3 (have freerun chars w/
ADEV) will now derive Q[3,3] from ADEV instead of 0.01 on next run —
the intended honest-Q change.  MadHat has no freerun char yet, so it
keeps the fallback until OCXO-33 is characterized.  Validate in
closedLoopServoSim before hardware.

7 new tests (RWFM formula, largest-tau selection, adjfine precedence,
non-finite/non-positive skipping, no-adev fallthrough, ADEV from a
non-chosen phase source, realistic freerun-char schema).  1387 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bob's naming-honesty catch (2026-05-29): "adjfine" names a PHC-specific
mechanism (clock_adjtime fine-frequency adjust), but as a
characterization source key it stands for the actuator-agnostic
frequency-steering command (ppb) — already abstracted as
FrequencyActuator.adjust_frequency_ppb (PHC / DAC / ClockMatrix).
derive_do_process_noise now reads "freq_command" as primary with
"adjfine" as a back-compat alias.  (Nothing currently writes either —
the read path is vestigial; freerun chars write "DO PPS (chA vs TICC
Rb)".)

Logged in docs/misnomers.md, plus a second (deferred) entry: the
characterization "sources" dict is really "signals"/measurements — and
the freq_command entry is a recording of a SINK-bound signal sitting
under "sources", which Bob flagged as confusing.  Deferred because
renaming the persisted key touches 1 writer + 5 readers + lab-host
state JSONs; do it with a back-compat alias when next in the writer.

2 new freq tests (freq_command primary, freq_command-over-adjfine
precedence); adjfine test kept as the legacy-alias case.  26 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant