Audit remediation: core numerics, binding contracts, and the tests that missed them - #393
Merged
Conversation
The Unreleased entry still described the first attempt, which resolved objdump and dlltool through the compiler's -print-prog-name. That approach did not work: -print-prog-name returned the same x86_64 mingw objdump, which has no aarch64 PE backend. The shipped fix drops objdump entirely and derives the export list from the cbindgen header, then passes the target machine to dlltool explicitly. Describe what the code actually does.
PlusDm, MinusDm, PlusDi, MinusDi and Dx all returned `self.period` from `warmup_period()`, while every one of their module docs states that the first value is emitted after `period + 1` candles. The docs are right: the first candle only seeds `prev` and returns early, so `seed_count` starts advancing on bar 2 and the seed completes on bar `period + 1`. The existing `seeds_then_smooths_a_constant_*` tests already showed this (period 3, first `Some` at index 3), but the accessor tests pinned the wrong declared value, so the contradiction never surfaced. The trait defines `warmup_period()` as the number of inputs required before the first non-`None` output, so a caller slicing `out[ind.warmup_period()..]` kept a leading `None` in what it treated as the dense region, and any binding mapping the value to an offset surfaced that as a NaN first "valid" sample. Return `period + 1` and add a contract test to each of the five files that derives the first emitted index from a real batch and asserts it equals the declared warmup across several periods. Adx (`2 * period`) and Adxr (`3 * period - 1`) already accounted for the extra bar and are unchanged — with Dx emitting at `period + 1`, Adx seeding over `period` values lands exactly on `2 * period`, which the corrected figure now makes internally consistent.
`Trix::is_ready()` was `prev_tr.is_some() && ema3.is_ready()`. The `None` arm of `update` assigns `prev_tr` and *then* returns `None` — that bar is the rate-of-change baseline, not an emission — so readiness flipped to `true` one input before the first value appeared. The trait defines `is_ready()` as whether the indicator has emitted at least one value since the last reset, so this broke the contract, and it propagated to every binding's `isReady`/`is_ready` and to `Chain::is_ready`. Track emission explicitly with a `has_emitted` flag set only in the two arms that return `Some`, cleared by `reset()`. The `ema3.is_ready()` conjunct is now redundant: a value can only have been emitted after `ema3` produced one. The new test walks a series and asserts `out.is_some() == is_ready()` on every single input, which pins the flip point exactly, and then checks the declared warmup against the index of that first emission — confirming `3 * period - 1` was already right.
`BinanceConfig::max_reconnect_attempts` is a public `u32` whose documentation
says it must be at least 1, but nothing checked it. `connect_with_config`
validated only that the symbol list was non-empty, so
`BinanceConfig { max_reconnect_attempts: 0, ..Default::default() }` was
accepted. `reconnect` then loops `0..0`, never enters the body, leaves
`last_err` as `None`, and the closing
`Err(last_err.expect("max_reconnect_attempts is non-zero"))` panics the task on
the first dropped connection — a reachable panic from an entirely safe public
API, and the only `expect` in the crate whose stated precondition was not
actually enforced anywhere.
Validate the field alongside the symbol list, before any socket is opened, so
the rejection costs nothing and needs no server to test. The `expect` stays, now
genuinely unreachable by construction, with a message naming the guard that
makes it so.
`wickra_core::Error` has grown from roughly four variants to eleven, and every addition was a breaking change for any downstream crate that matched it exhaustively. `wickra_data::Error` is worse: several of its variants are behind `#[cfg(feature = "live-binance")]`, so the set a consumer sees already depends on which features are enabled and an exhaustive match cannot be written portably. Both are now `#[non_exhaustive]`, which is what `CrossSection` and `Member` already use elsewhere in the crate. The only exhaustive match in the workspace was the Python binding's `map_err`, whose eleven arms all produced the same `PyValueError::new_err(e.to_string())`. Rather than bolt a catch-all onto a match that never discriminated, drop the match: every core error is a rejected argument and maps to one Python exception. Behaviour and message are identical, and there is no dead arm to justify later.
The dispersion family maintained `sum` and `sum_sq` over raw price levels and
computed `(sum_sq / n - mean * mean).max(0.0)`. That is the textbook unstable
form: when the values are large relative to their spread the two terms agree to
most of their bits and the difference is dominated by rounding. A price series is
precisely that shape, and the `.max(0.0)` clamp — added to absorb rounding noise
— silently converted the failure into a plausible-looking zero.
Measured against a two-pass reference over a 20-bar window of
`level + sin(i * 0.7) * spread`:
level spread E[x^2]-E[x]^2 shifted
1e2 1 3.9e-12 3.3e-16
1e2 0.01 9.0e-08 5.1e-16
1e5 1 4.3e-06 1.6e-16
1e5 0.01 9.4e-02 1.3e-16
1e8 1 1.0 4.9e-16
The 1e5 / 0.01 row is Bitcoin on one-second bars in a quiet stretch: Bollinger
bands roughly four percent too wide, and every %B, squeeze and breakout signal
shifted with them. The 1e8 row is a total collapse — TtmSqueeze reports a
permanent squeeze and ZScore divides into a clamped zero.
Add `ShiftedMoments`, which accumulates the moments of `x - offset` for an
`offset` taken from inside the window, so the residual cancellation is between
quantities of order `spread^2` rather than `level^2`. It deliberately does not
own the window: every affected indicator already keeps its values, often for
other purposes, so the accumulator attaches to what is there and is driven by
`push`/`evict`. `reseed` re-centres the reference point and recomputes both
moments from the live window once per period — amortised O(1), and it bounds the
add/subtract drift at the same time.
This commit migrates the four indicators whose shape is identical to the
reference case: StdDev, Variance, ZScore and CoefficientOfVariation. The
remaining members of the family follow. `StdDev` carries the regression test
that pins all three price levels against a two-pass reference; before the change
it failed at 1e5.
The helper intentionally ships only the methods that have a consumer today, so
there is no unreachable surface to justify later.
BollingerBands maintained its own `sum`/`sum_sq` pair over raw price levels and computed `(sum_sq / n - mean * mean).max(0.0)` in `current()`, with the same arithmetic inlined a second time in the vectorized `batch_bands` fast path. Both inherited the cancellation described in the previous commit, and the band width *is* that standard deviation: at a price level of 1e5 the bands came out roughly four percent too wide on a tight range, and at 1e8 the deviation collapsed to exactly zero, producing zero-width bands and a permanent squeeze reading in every downstream consumer. Both paths now drive `ShiftedMoments` over the existing ring buffer. The reseed that used to live here (`RECOMPUTE_EVERY * period`) is subsumed by the accumulator's own once-per-window reseed, which is strictly more frequent and re-anchors the reference point as well as bounding drift, so the local constant is gone. Because both paths call the identical accumulator with the identical cadence, `batch_bands_fast_path_is_bit_identical_with_reseed` still holds, as does `long_stream_drift_stays_bounded`. The freshness guard in `batch_bands` dropped its `updates_since_recompute != 0` clause: the reseed counter can only be non-zero after a value has been pushed, so `count != 0` already covered it. Adds a band-width accuracy test across three price levels, mirroring the one on StdDev.
Skewness and Kurtosis reconstructed `m3` and `m4` from raw power sums by
binomial expansion:
m3 = E[x^3] - 3*mean*E[x^2] + 2*mean^3
m4 = E[x^4] - 4*mean*E[x^3] + 6*mean^2*E[x^2] - 3*mean^4
Every term on the right is of order `level^4`, while the result is of order
`spread^4`. That is the variance cancellation from the previous commits, made
worse by two further powers of the price level: at a level of 1e5 a spread of
1e-2 leaves nothing at all of the fourth moment. Both indicators also computed
`m2` in the same unstable form.
Add `ShiftedHigherMoments`, the four-power sibling of `ShiftedMoments`: the same
reference-point trick and the same once-per-window reseed, tracking sums of
`(x - offset)` through the fourth power so the expansions above operate on
spread-scale quantities. `m2`, `m3` and `m4` are exposed directly, so neither
indicator carries the expansion any more.
The accumulator is tested against a two-pass reference for all three moments at
price levels 1e2, 1e5 and 1e8.
… accumulator RviVolatility, SpreadBollingerBands and FundingRateZScore each carried their own `sum`/`sum_sq` pair and the same `(sum_sq / n - mean * mean).max(0.0)` expression. They now drive `ShiftedMoments` over their existing window. RviVolatility is the exposed one: it is literally a rolling standard deviation of price, so it inherited the full cancellation — at a level of 1e5 the deviation that its up/down classification is built on lost most of its significant digits, and at 1e8 it collapsed to zero, which would classify every bar as zero-volatility. SpreadBollingerBands accumulates a spread between two series, so its exposure depends on how large the spread values are relative to their own variation. FundingRateZScore accumulates funding rates, which are tiny and centred near zero, so it was barely affected; it is migrated for consistency, and to pick up the drift bound the accumulator's reseed provides. Six further files matched a `sum_sq` grep but are *not* affected and are deliberately left alone: `hurst_exponent`, `linreg_channel`, `parkinson`, `realized_volatility`, `spread_hurst` and `ulcer_index` all accumulate squares of quantities that are already centred (deviations, residuals, log returns, squared drawdowns) and never subtract a squared mean, so there is no cancellation to remove. Changing them would be churn. RviVolatility keeps its `n` binding: the Wilder smoothing further down the function still needs it.
…ator Ten indicators computed a sample variance as `((sum_sq - n * mean * mean) / (n - 1)).max(0.0)` over their own `sum`/`sum_sq` pair — the same cancellation as the population form, with the same clamp hiding it. Three of them (KaseDevStop, VolatilityCone, VolatilityOfVolatility) had each grown a private `sample_stddev(sum, sum_sq, count)` helper, three copies of one function. `ShiftedMoments` gains `sample_variance`, which applies Bessel's correction to the reference-point-relative moments, and all ten now use it: HistoricalVolatility, InformationRatio, JumpIndicator, KaseDevStop, M2Measure, RegimeLabel, SharpeRatio, VolatilityCone, VolatilityOfVolatility and YangZhang. The three private helpers are gone. Exposure varies and it is worth being precise: most of these accumulate log returns or active returns, which are small and centred near zero, so their level-to-spread ratio is O(1) and the cancellation was mild. KaseDevStop is the outlier — it takes the standard deviation of the two-bar *true range*, a price-scale quantity. All are migrated regardless, because the cost is identical, it removes the duplicated helpers, and the accumulator's reseed bounds the add/subtract drift that none of them had any protection against. One test in VolatilityOfVolatility built its expectation by calling the same `sample_stddev` the implementation used, so it could not have caught an error in it. It now computes a two-pass reference directly. After this commit no indicator in the catalogue computes a variance as `E[x^2] - E[x]^2`.
`batch.wickra_indicator` forwarded its `...` straight into `.Call`, and the generated C took the row count from the first column only, then indexed every other column with it. A shorter column was therefore read past its end. This was reachable from ordinary R code — passing a three-element timestamp alongside full OHLCV vectors segfaulted the interpreter — and it applied to all 353 generated batch routines. Every other binding already validated this: C# throws ArgumentException, Go panics, Java throws IllegalArgumentException, Node and WASM return an error. There was a second, quieter hazard in the same place: the C read every argument with `REAL()` without checking its type. An integer vector such as `1:100` is an INTSXP, and reinterpreting its storage as doubles yields nonsense rather than an error. Guarded at both levels. `batch()` now coerces each column with `as.double()`, rejects a non-numeric column, rejects an empty call, and reports a length mismatch with every column's length in the message. The generated C checks the handle is non-NULL, checks each argument is a REALSXP, and checks each length against the first — so a direct `.Call` that bypasses the R layer is refused too. All guards run before the first PROTECT, since Rf_error long-jumps out and there is nothing on the protection stack at that point. The guards are emitted by the generator, not hand-written into the output. Verified against a freshly installed build: the reported segfault case now raises from R, the same case through a direct `.Call` raises from C, an integer vector produces the same result as its double equivalent, and the package's 119959 assertions pass with no failures.
`testdata/golden/` holds the language-neutral reference outputs that all eight bindings replay through their own FFI. They are produced by the Rust core, so the shifted-moment accumulator moved the trailing digits of every fixture in the dispersion family. 21 files change; the indicators behind them are exactly the ones migrated in the preceding commits, plus the three that derive from Bollinger bands (BollingerBandwidth, DoubleBollinger, PercentB). This is worth spelling out because `cargo test --workspace` does not cover it. The golden replays live in the bindings' own suites, so the Rust runs stayed green while the committed fixtures were quietly stale, and CI would have failed at the first binding job. Every changed value was checked against an independent two-pass reference computed from `input.csv`. For StdDev all 67 values are closer to the reference than the ones they replace, several now matching it exactly; Variance is 67 of 67. ZScore is 64 of 67, with three values one ULP further out — expected for a ratio of two rounded quantities where the reference carries rounding of its own. No value changed by more than its last digits, and no sign or structural change occurred anywhere. `g_LinRegAngle.csv` is deliberately excluded. It also rewrites one line when regenerated, but it does so on an untouched `main` checkout too — verified in a scratch worktree in both dev and release profiles — so it is pre-existing drift in a fixture whose indicator this branch never touches. Recorded separately rather than folded in here. Verified end to end: the C ABI was rebuilt from this branch, the R binding installed against it, and its 119959 assertions — golden replay included — pass against the regenerated fixtures.
Every generated method called the native library as `NativeMethods.wickra_x(_handle.DangerousGetHandle(), ...)` followed by `GC.KeepAlive(_handle)`. `DangerousGetHandle` keeps returning the pointer after `Dispose()` has already run `ReleaseHandle` and freed it, and `GC.KeepAlive` only prevents collector-driven finalisation — it does nothing about an explicit dispose. Any use of an indicator after disposing it therefore read freed memory: a captured lambda, a cached dictionary of indicators, an async continuation that outlives its `using` block. Depending on the heap layout that is a silently wrong number or an access violation, with a Rust panic unwinding across `extern "C"` in between. There were 2914 such call sites and no guard anywhere: `IsClosed`, `DangerousAddRef` and `ObjectDisposedException` appeared exactly zero times in the binding. The handle parameters are now typed `WickraHandle` rather than `nint`, so the `LibraryImport` source generator emits `DangerousAddRef`/`DangerousRelease` around each call. That both pins the handle for the duration of the call and raises `ObjectDisposedException` when it has already been released — the protection `SafeHandle` exists to provide and that `DangerousGetHandle` bypasses. `GC.KeepAlive` becomes redundant and is gone; so is every `DangerousGetHandle` in the generated binding. `_free` keeps its raw `nint` parameter, since it is stored as an `Action<nint>` and invoked from `ReleaseHandle` with the raw field, where ref-counting would deadlock. Fixing this surfaced a second generator bug: `is_handle` matched only a bare `struct ` prefix, so `const struct CandleReader *handle` was not recognised and `wickra_candle_reader_count` kept a raw pointer parameter. It now tolerates the const qualifier. Adds eight tests covering update, batch, the read-only accessors, reset, a candle indicator, a multi-output indicator, idempotent dispose, and that an undisposed indicator still behaves. The full suite is 556 tests, all passing against a C ABI built from this branch.
Two defects in the generated Java binding, both invisible to its test suite. Use-after-close. Every method read the `handle` field directly, so a call made after `close()` had already run the `Cleaner` action dereferenced freed memory. `grep` found no `closed` flag and no `IllegalStateException` anywhere in the binding, and nothing else stood in the way: the JVM died with an EXCEPTION_ACCESS_VIOLATION. That is worse than the equivalent C# defect, because the whole premise of the Panama FFM API is that safe Java cannot segfault the VM. Calls now go through a private `handle()` accessor that refuses a released handle, `close()` is idempotent, and each downcall carries `Reference.reachabilityFence(this)` in a `finally` so the cleaner cannot run while a native call is still in flight — there was no fence anywhere before. Timestamp marshalling. `emit_batch` typed every input array as `double[]` and allocated it with `JAVA_DOUBLE`, regardless of what the header declared. The C ABI declares candle timestamps as `const int64_t *`, of which there are 186, so the native side reinterpreted the IEEE-754 bit pattern as an integer: an epoch of 1700000000000 arrives as roughly 4.8e18. Inert wherever the timestamp is ignored, silently wrong for every session- and calendar-aware indicator. The array type and layout are now derived from the declaration. That derivation also corrected the cross-section flag arrays declared `const bool *`, which were likewise typed `double[]` and squeezed through a `boolSegment(Arena, double[])` helper testing `!= 0.0`. They are now `boolean[]`, which is what the C# binding has always exposed; the helper takes `boolean[]` to match. BinanceFeed is emitted by a separate path and carried the same use-after-close hazard, so it gained the same guard. Verified against a C ABI built from this branch: all 626 sources compile, and nine tests pass covering update, batch, the accessors, reset, a candle indicator, idempotent close, an unclosed indicator, and streaming-versus-batch agreement over 400 bars of SessionVwap and 200 daily bars of TurnOfMonth — the sharpest available check on the timestamp path, since the two routes cannot agree unless both read the timestamp the same way.
Every constructor sizes its buffers from the period, and the only guard was
`period == 0`. `Ema::new(usize::MAX)` therefore aborted the process with a
capacity overflow raised inside `Vec` — in release as well as debug — and
`Ema::new(1_000_000_000)` reserved eight gigabytes before the caller saw
anything go wrong. Neither is exotic to reach through a binding: a mistyped
literal or a period read from a config file gets there.
Add a public `MAX_PERIOD` and reject anything above it as
`Error::InvalidPeriod`. `1 << 24` is 16777216: one `f64` buffer of that length
is 128 MiB, which is far beyond any real window while leaving the period
arithmetic that appears in `warmup_period` bodies — `6 * period - 5`,
`3 * period - 1` — nowhere near overflowing. The accompanying message
deliberately does not repeat the number so the two cannot drift apart.
The bound is applied after each existing guard, once per distinct validated
parameter, across 219 indicator files: 222 bounds in total. Files whose
constructor takes no window length, which is most of the candlestick and
harmonic pattern family, are untouched.
`DynamicMomentumIndex` already had a file-local `MAX_PERIOD` meaning its slowest
RSI lookback (30). Two different constants of the same name in one crate is a
trap, so that one is now `MAX_RSI_LOOKBACK`.
The new constant is re-exported from `lib.rs` outside the counted
`pub use indicators::{...}` block, so the catalogue count is unchanged at 514,
and the golden fixtures are byte-identical — this only adds a rejection path.
The workspace release profile set `panic = "abort"`. The Python, Node, WASM and C ABI cdylibs are workspace members, and Cargo refuses `panic` in a per-package profile override — "`panic` may not be specified in a `package` profile" — so that setting was theirs too, with no way to give them a different one. Both pyo3 and napi convert a panic into a language-level exception by catching it at the FFI boundary with `catch_unwind`. Under `abort` that machinery can never run: the process is gone before it gets a chance. So any panic anywhere in the core took the caller's Python interpreter or Node process with it rather than surfacing as an exception. Letting a panic unwind out of `extern "C"` is itself undefined, which is exactly why those crates catch it first — and why they need unwinding to exist. Measured cost over 500000 bars, three runs each. Streaming throughput is unchanged: SMA 388 -> 391, ATR 329 -> 331, MACD 193 -> 188 Mupd/s, all inside the run-to-run spread. The C ABI cdylib grows from 2277888 to 3062784 bytes, 34.5%. SMA's batch fast path drops from about 500 to about 296 Mupd/s while ATR's is unchanged. That last one is not mysterious. Of the six hand-written fast paths, exactly `sma.rs` and `bollinger.rs` index the ring buffer inside the hot loop; `rsi`, `ema`, `macd` and `atr` do not, and are unaffected. Under `abort` a failed bounds check just terminates, so the loop stays vectorizable; under `unwind` it becomes an unwind edge carrying drop glue for the output vector. Both loops can walk the ring with an iterator instead of indexing — the fast path only runs from a fresh state, so `head` is zero at every lap boundary and the reseed threshold is a whole multiple of the period. That is tracked separately rather than folded into a profile change. The per-tick latency tables in BENCHMARKS.md measure streaming and remain accurate.
Switching the release profile to `panic = "unwind"` cost `Sma::batch_nan` about 40% — roughly 500 to 296 Mupd/s over 500000 bars. The first diagnosis was that the ring-buffer indexing in the loop turned into unwind edges and blocked vectorisation. That was wrong, and measuring said so: restructuring the loop to walk the ring in laps with an iterator, with no indexing at all, left it at about 300. The actual cost was `Vec::push`. Under unwinding, each push in the loop carries drop glue for the partially built vector alongside its capacity branch. Allocating the output up front and writing through a `split_at_mut` slice per lap removes both, and lands at 513-540 Mupd/s — above what the `abort` build managed, because the lap structure also dropped the per-element wraparound branch. The lap structure is what makes the iterator walk exact. The fast path only runs from a fresh state, so `head` is 0 at every lap boundary and a lap is exactly `period` inputs; `RECOMPUTE_EVERY * period` is a whole multiple of `period`, so the drift reseed can only fall on a boundary, where the chronological order it needs is just the buffer in order. Only the final lap can be short, since a shorter chunk means the input ran out. One ordering detail the bit-identical test caught immediately: `update` reseeds *before* emitting the value for the input that tripped it, so the last value of a reseeding lap has to come from the reseeded sum. Getting that wrong is otherwise invisible — the values differ only in their last bits, and only once every sixteen laps. `BollingerBands::batch_bands` was on the same list, but measuring it in isolation showed nothing to fix: it already writes into a pre-sized buffer, and at 500000 bars it runs at 154.4 Mupd/s under unwinding against 130.8 under abort. Left alone. Golden fixtures are unchanged, as the bit-identical and drift tests require.
Both indicators initialised their two output fields to `f64::NAN` and returned `Some` on every bar once past warmup, whether or not either level had been established. On a series where no TD setup ever completes — a flat market is enough — that is `Some` carrying two NaNs, forever. They were the only two indicators in the catalogue encoding "no value" as anything other than `None`, and the trait defines `None` as exactly this case: insufficient input to produce a defined value. In the bindings it arrived as NaNs in a flat output buffer, where it silently poisons whatever arithmetic follows. They now return `None` until at least one level exists, so a returned value always carries at least one real price. A single `NAN` field stays. The two levels are established independently — a completed buy setup sets one, a completed sell setup the other — so "resistance known, support not yet" is a real state worth reporting, and the C ABI mirrors each output as two plain `double`s, which rules out `Option<f64>` for the public struct. What changed is that this is now documented on the fields rather than left to be inferred, and that the all-NaN case can no longer occur. The internal fields became `Option<f64>` so "unset" is a distinct state rather than something inferred from a bit pattern belonging to a value that is supposed to be a price. `warmup_period()` gains a note on both: it is a lower bound, because a completed setup depends on the data and may never arrive. The golden fixtures for both move from `NaN,NaN` to `nan,nan` on the affected rows — the generator's marker for a withheld value rather than an emitted one, which is precisely the distinction being fixed. Verified by rebuilding the C ABI from this branch and running the C# golden suite: 529 tests pass.
`g_LinRegAngle.csv` rewrites one line whenever `gen_golden` runs, on an untouched `main` checkout as much as here — verified earlier in a scratch worktree under both the dev and release profiles. It is pre-existing drift in a fixture whose indicator this branch never touches, and which of the two values is correct is still open. It slipped into the previous commit because the fixtures were regenerated to inspect a diff and not reset before staging. Restore it to the value on `main` so this branch carries only what its own changes cause.
`sum += new; sum -= old` is O(1) but never forgets a rounding error, so an accumulator's deviation from a from-scratch sum grows with the length of the stream. Long streams are the case this library exists for. Measured before touching anything, over three million updates, comparing a long run against a fresh instance fed only the final window — for a pure windowed indicator the two must agree. `Sma`, which already rebuilt its sum every so often, came out exactly equal. `MedianMa`, which keeps no running sum, likewise. `Vwma` was off by 6e-14 relative, `Cci` by 5e-09, `Dpo` by 2e-07. Small enough to be invisible in any trading context, but unbounded, and inconsistent: two indicators were protected and sixty-one were not. Adds `RollingSum` alongside the moment accumulators — a total plus a rebuild counter, rebuilt from the caller's window once per period. The window stays with the caller because these sums sit next to deques the indicator already keeps, and duplicating that storage would cost more cache than the rebuild saves. Twelve indicators move onto it: the eleven whose evicted value is literally what was popped from the deque the sum tracks, so the rebuild window is unambiguous, plus `Dpo`, whose sum covers a suffix of a longer window and which was the worst drifter measured. `Cci` and `Dpo` now measure exactly zero deviation on the same three-million-update harness. Fifty files are deliberately left alone. They carry two to six accumulators each, and which window backs which sum has to be read case by case; pairing one wrongly would be silent and would survive the test suite, since the values only differ in their last digits. They are tracked for individual passes rather than swept up in a blanket rewrite. The worst known residual is `Vwma` at 6e-14. Eleven golden fixtures move, all belonging to migrated indicators (plus `StochasticCci`, which is built on `Cci`). Every change is last-digit: the largest relative movement anywhere is 4e-13, with no sign or structural change. Verified by rebuilding the C ABI from this branch and running the C# golden suite: 529 tests pass.
Fourteen indicators sort a copy of their window on every update. Seven of them
kept a reusable scratch buffer for it; the other seven allocated a fresh `Vec`
inside `update` and threw it away again. Same family, same work, half of it
already solved.
Measured over a million updates, before and after, with `RollingQuantile` — a
sibling that already had the buffer and is untouched here — as a control:
MedianMa(20) 16.2 -> 21.3 Mupd/s
ValueAtRisk(20) 15.3 -> 19.6
TailRatio(20) 14.3 -> 19.7
RollingQuantile(20) 20.4 -> 20.2 (control)
The three now sit level with the control, which is what you would expect once
the only difference between them is gone.
`CommonSenseRatio`, `ConditionalValueAtRisk`, `VolatilityCone` and
`AdaptiveLaguerreFilter` are migrated on the same reasoning. Two of them sorted
inside a private `compute(&self)` called only from `update`, so that takes
`&mut self` now; `AdaptiveLaguerreFilter` builds a transformed copy rather than
a plain one, which the buffer holds just as well.
`MedianMa::value()` was doing the sort itself, so every read of the public
accessor paid for it. The median is now computed once per `update` into a cached
field and the accessor returns that, matching how the rest of the catalogue
exposes its current value.
Comparators are unified on `f64::total_cmp`. Three of these sorted with
`partial_cmp` and either swallowed the `None` through `unwrap_or(Equal)` or
unwrapped it outright. No `partial_cmp` remains in indicator production code.
No output moves: sorting the same finite values by a total order in place of a
partial one, and unstably rather than stably, cannot reorder them observably.
The golden fixtures confirm it — nothing regenerates differently.
Not one of the 514 indicator files carried an `#[inline]`. A trait-impl body is
not generic, so its MIR is not exported and it cannot be inlined across a crate
boundary without link-time optimisation. Wickra's own release profile sets
`lto = "fat"`, which hides this from every measurement taken inside the
workspace — but a downstream Rust crate on Cargo's default release profile has
LTO off, and pays a call per tick into a body that is often a handful of
arithmetic operations. That is the whole per-tick claim, going out through a
function call.
Measured from a crate configured exactly like a default downstream consumer
(`lto = false`, `codegen-units = 16`), two million updates, best of five:
Sma(20) 369.0 -> 1322.5 Mupd/s
Atr(14) 312.5 -> 341.6
Rsi(14) 292.1 -> 313.6
Ema(20) 338.2 -> 340.7 (unchanged)
The spread is what you would expect: `Sma::update` is small enough to fold into
the caller's loop entirely, `Ema::update` was already cheap relative to the call.
The same crate rebuilt with `lto = "fat"` now measures no better than the
default profile does, which is the point — the LTO build was never the problem.
Applied to `update` where the body is 40 lines or fewer, which is 438 of 513.
The remainder are large enough that LLVM will decline the hint regardless, so
marking them would export metadata for nothing and slow downstream compiles.
`is_ready`, `warmup_period` and `name` are one-liners and are marked
unconditionally.
Cost: the C ABI cdylib grows from 3063296 to 3118080 bytes, 1.8%.
`#[inline]` changes no semantics and the golden fixtures confirm it — nothing
regenerates differently.
`BollingerBandwidth` and `PercentB` were the only two of the 514 catalogue entries that no fuzz target reached. Both are plain scalar indicators derived from the same bands, so they take the generic drive helper alongside everything else in `indicator_update`. Adds `indicator_chain`, covering `Chain`. That is worth its own target because it exercises something no single-indicator target can: the second stage consumes a stream produced by the first, not the raw input. That stream has a different shape — it begins later, because the first stage withholds through its own warmup; it can sit constant for long stretches; and where the first stage is a rate of change it manufactures infinities and NaNs out of perfectly finite input. A stage that copes with arbitrary prices can still be surprised by it. The target also reads `warmup_period` and `is_ready` at every point of the stream, since composing those across stages is arithmetic the fuzzer drives indirectly. Five compositions are covered, including the reverse ordering (oscillator feeding a smoother) and a three-stage chain built through `then`. Doing this surfaced a wider gap: `cargo fuzz run <target>` builds only the target it names, and ci.yml named five of the thirteen. The other eight were neither executed nor compiled, so a signature change in the core would leave them silently broken until someone ran them by hand. CI now runs `cargo fuzz build` across all of them first, and the new chain target joins the five that are executed. The fuzz targets need nightly plus cargo-fuzz, neither of which is available here, so the harnesses could not be built locally. Both additions were instead compiled and executed against `wickra-core` with the libfuzzer harness swapped for a `main` — verifying the API usage and driving each through empty input, NaN, both infinities, negative zero, subnormals, values at the edge of the f64 range, a smooth series and an all-zero series. Nothing panics. The libfuzzer build itself is covered by the new CI step.
The trait defines `is_ready` as whether the indicator has emitted at least one
value since the last reset. Nothing checked it, and four indicators keyed it off
something that changes at a different moment:
Ichimoku required all five output components to be present, long after it
started emitting outputs with some of them still None
ZigZag keyed off a trend state seeded by the very first bar, which emits
nothing at all
LongLine required a full averaging window while emitting from bar one
ShortLine the same
All four now track emission directly. Two ZigZag tests asserted the old
behaviour — feeding one bar, observing `None`, and then asserting ready — so
they encoded the defect; they now drive to an actual confirmed swing and check
the transition there, plus the reset.
`check_ready` is wired into every family macro, so the contract holds for all
513 indicators across scalar, candle, pair, cross-section, trade, derivatives,
order-book and trade-quote inputs. It also pins the two endpoints nothing
covered: a fresh instance is not ready, and `reset()` returns it to not ready.
The check was verified to bite before being trusted: reintroducing the `Trix`
defect fixed earlier in this release makes it fail at input 6 with the right
message.
The warmup bound is deliberately not asserted yet. Measuring all 513 against a
deterministic series turned up 89 declared-versus-observed mismatches, but they
split two ways and only one side is a defect. Twenty declare *less* than
observed and are correct as written — `AverageDailyRange` declares 2, which is
right for daily bars, and only reads as 1441 because the probe feeds minute
bars; their declaration is a valid lower bound. The other sixty-nine are the
harmonic and chart-pattern family, which declares five or six bars and then
emits `Some(0.0)` from bar one. Deciding whether those should withhold until
armed, as `TdLines` now does, or admit that their honest warmup is one, changes
output for sixty-nine indicators and moves their fixtures. That belongs in its
own commit with its own fixture review, and is tracked as such.
No output changes here: `is_ready` does not feed `update`, and the golden
fixtures regenerate identically.
The earlier variance work claimed the catalogue no longer computed a variance as
`E[x^2] - E[x]^2`. That was wrong. The sweep behind it matched a bare
`mean * mean` and so missed every suffixed identifier — `mean_b * mean_b`,
`mean_x * mean_x`, and so on. Eighteen files still use the form, most of them
for a covariance rather than a variance.
Covariance fails the same way: `E[xy] - E[x]E[y]` on raw levels is a difference
of two nearly equal large quantities, and correlation inherits it three times
over, through the covariance and both variances. Measured on
`PearsonCorrelation` over a 20-bar window of two sine series, against a two-pass
reference:
level spread relative error
1e2 1 8.6e-12
1e5 1 1.1e-05
1e5 0.01 9.6e-02
1e8 1 1.0 (collapses)
Those are the magnitudes the original variance defect had. A correlation of two
instruments priced near 1e5 that move by a hundredth was wrong in the second
decimal place.
Adds `ShiftedPairMoments`, the two-channel counterpart to the existing
accumulator: each channel is centred on its own reference point, so the residual
cancellation is on the order of the spreads. `PearsonCorrelation` and
`RollingCorrelation` are migrated and now measure at the 1e-16 floor at every
level tested.
The remaining sixteen are recorded rather than swept up here, because their
exposure differs and each needs reading: those accumulating price levels are
fully exposed, while those accumulating returns or first differences are not.
`RollingCorrelation` is in the second group and was migrated anyway, for
consistency and for the drift bound the rebuild brings.
The overclaim in the changelog is corrected rather than quietly dropped.
Two golden fixtures move, both migrated indicators, last digits only — worst
relative movement 3.9e-13, no sign or structural change.
Six more indicators computed their covariance and benchmark variance as `E[xy] - E[x]E[y]` and `E[y^2] - E[y]^2` over raw inputs, the form the previous commit showed collapses at realistic price levels. They now share `ShiftedPairMoments`. Exposure was checked per indicator rather than assumed, having already misread `RollingCovariance` once by feeding it levels when it consumes returns. Reading each one's documented input: Cointegration price levels fully exposed BetaNeutralSpread a price pair fully exposed Beta a generic (asset, benchmark) exposed if fed prices Alpha (asset_return, bench_return) barely TreynorRatio (asset_return, bench_return) barely PairwiseBeta prices, differenced internally barely The last three are migrated anyway: the cost is identical, it removes the sixth copy of the same expression, and the accumulator's periodic rebuild bounds the drift none of them had protection against. `Beta` over two sine series measured against a two-pass reference now reports exactly zero deviation at price levels 1e2 and 1e5, and 3.7e-16 at 1e8; before, the same measurement on `PearsonCorrelation` showed nine percent at 1e5 with a tight spread. `mean_a` and `mean_b` return to the accumulator, dropped last time for want of a consumer: `Alpha` and `TreynorRatio` need the channel means for their own formulas. Seven golden fixtures move, all belonging to migrated indicators, last digits only — worst relative movement 1.3e-09 on `Alpha`, which divides by a variance and so magnifies it. Verified by rebuilding the C ABI from this branch and running the C# golden suite: 529 tests pass.
This one was mis-triaged as moderate exposure and turned out to be the worst case in the sweep, by twelve orders of magnitude. The indicator regresses `ln(a)` on `ln(b)` and z-scores the residual, so the quantity it reports is by construction tiny next to the log-levels it is derived from — that ratio *is* the signal. Both accumulators held raw power sums, so both cancelled precisely where the indicator is meant to be sharpest. Measured against a two-pass reference over 3000 bars, 20-bar hedge window, 30-bar z window: level 1e2, 5% spread 1.5e-06 -> 1.1e-11 level 1e5, 5% spread 1.2e-05 -> 3.4e-12 level 1e5, 0.01% spread 6.6e+01 -> 3.7e-11 level 1e8, 0.01% spread 1.3e+02 -> 6.0e-11 A relative error of 66 is not a precision problem: at two legs priced at 1e5 with a 0.01% spread — a statistical-arbitrage pair, which is what this indicator is for — neither the magnitude nor the sign of the signal could be relied on. The rolling OLS moves to `ShiftedPairMoments` with the regressor as channel `a`, and the spread window to `ShiftedMoments`. The remaining 1e-11 is the z-score dividing by a small dispersion, which is intrinsic to the statistic. The golden fixture moves in 42 of its 80 cells. Every one moves toward an independent two-pass computation over the same candles: worst deviation from that reference falls from 4.7e-09 to 1.5e-13, with emission positions unchanged. Verified by rebuilding the C ABI and running the C# golden suite: 556 pass. Also drops a stale comment claiming the variance is computed as sum of squares minus mean squared, which is no longer how it is computed.
Five of the six indicators left in this group are migrated. Measuring each one before touching it corrected two things I had assumed. First, three of them do not accumulate anything. `OuHalfLife`, `SpreadAr1Coefficient` and `LeadLagCrossCorrelation` rebuild their regression from the live window on every update, and I had been treating "rebuilt each time" as equivalent to "safe". It is not: recomputing bounds the drift and does nothing whatsoever about the cancellation in `E[xy] - E[x]E[y]`, which is just as bad whether the power sums were carried for a million updates or built a microsecond ago. These needed a second pass, not an accumulator, so this adds `centred_moments` alongside the accumulators in `rolling_moments.rs`. The two spread regressions are the exposed case, because a cointegrated spread sits at a large constant offset with a small wobble on top -- the worst possible ratio for that expression, and precisely the regime they exist for: OuHalfLife, spread 5000 wobbling by 0.1 2.8e-06 -> exact OuHalfLife, same spread wobbling by 0.01 2.6e-04 -> exact SpreadAr1Coefficient, offset spread at 1e5 4.9e-08 -> exact LeadLagCrossCorrelation at level 1e8 2.2e-05 -> exact "Exact" is against a two-pass reference, which they now match bit for bit because they perform the same arithmetic. The lead-lag error is on a quantity bounded to [-1, 1]. Second, `SpearmanCorrelation` is not affected and is deliberately left alone. It correlates ranks, which are bounded by the window length, so the level-to-spread ratio that drives the cancellation is O(1) by construction. Measured at 1.4e-14 and flat from a price level of 1e2 to 1e8. Migrating it would have moved a golden fixture for no gain. `RollingCovariance` and `KylesLambda` are genuinely incremental and move to `ShiftedPairMoments`: 1.5e-11 -> 3.3e-12 and 2.9e-09 -> 2.1e-12. Kyle's lambda is the less obvious of the two, since signed volume is only centred on zero while order flow is balanced; the measurement above is at a trade size around 1e8 with a 99% buy imbalance. The two spread regressions also stop allocating a `Vec` per update. They collected the deque only to call `windows(2)` on it; the pairs are now produced lazily and traversed twice. Eight further indicators do the same thing for the same reason and are recorded as A10b rather than swept in here. Fixture review, against an independent two-pass computation over the same golden candles. Those candles sit around a price of 100, where there is almost nothing to cancel, so this is a check that nothing regressed rather than a demonstration of the fix: OuHalfLife 51/67 cells moved, <=5 ulp, 6.8e-16 -> exact SpreadAr1Coefficient 50/67 cells moved, <=4 ulp, 5.1e-16 -> exact LeadLagCrossCorrelation 41/41 cells moved, 1.7e-13 -> exact RollingCovariance 58/66 cells moved, <=12 ulp, 1.1e-15 -> 1.5e-15 KylesLambda 55/60 cells moved, <=117 ulp, 3.7e-14 -> 4.9e-14 The three that rebuild now match the reference exactly; the two incremental ones stay at the double floor and are not expected to equal a from-scratch pass bit for bit. Emission positions are unchanged everywhere and the lead-lag `lag` column is byte-identical, so no tie was broken differently. One regression test per migrated indicator. Verified the lead-lag one bites by reinstating the one-pass form and watching it fail. C ABI rebuilt and the C# golden suite run: 556 pass.
Twelve indicators built a least-squares fit of a window against its own index from raw power sums of the price, and four more wrap them. The slope is mathematically invariant when a constant is subtracted from y, so none of this was necessary. Two things had to be corrected before the work could even be measured properly. The earlier measurement recorded in the audit scaled the wobble WITH the price level, which holds the level-to-deviation ratio constant and hides the defect completely: it reported the family as seven orders milder than the variance collapse. A fixed absolute wobble is the realistic case -- a high-priced instrument still moves in units -- and it tells a different story. The reference was worse. My two-pass f64 reference formed residuals as `y - (intercept + slope*i)` with both sides the size of the price, which is the same defect being fixed. It made the corrected indicators look like they still carried a 2e-08 error. Dumping the exact f64 windows and computing the fit in exact rational arithmetic settled it the other way: at the worst bar the indicator was right to 2.9e-18 and the reference was 2.0e-08 out. Scored against exact rational arithmetic, 301 windows, price level 1e8: LinRegSlope, LinRegAngle 5.1e-04 -> 1.0e-16 RSquared 5.5e+04 -> 1.1e-14 StandardError, DetrendedStdDev 1.0 -> 8.7e-15 LinearRegression, Intercept, Tsf 2.2e-14 -> 1.4e-16 RSquared was out by four orders of magnitude on a value defined to lie in [0, 1]; only the clamp kept it in range. StandardError and DetrendedStdDev are worse than a relative error of 1 suggests: they rebuild the residual sum of squares by subtracting the explained variation from a total that had collapsed, the subtraction clamped at zero, and both then reported a perfect fit for a series they had not fitted at all. The seven incremental indicators share a new `ShiftedTrend` accumulator holding its sums relative to a reference point inside the window, rebuilt once per window. The sliding identity `Σ((i−1)yᵢ) = Σ(i·yᵢ) − Σyᵢ + y₀` is linear in y and survives the shift unchanged. The reference point is added back only where an absolute price level is returned -- the intercept, the endpoint, the forecast. `ProjectionBands` recomputes per bar and centres its slope directly. `StandardErrorBands` and `LinRegChannel` carried a second defect the audit had not recorded: they formed residuals at the price level, discarding eight digits of a residual at 1e8 before squaring it. Both now fit and take residuals on deviations. Their middle line improves 1.2e-15 -> 1.4e-16; the band levels cannot improve beyond that, because they are returned as absolute prices and a half-width of 0.7 on a price of 1e8 is quantised at 1.5e-08 by the output representation alone. Worth stating plainly rather than implying the bands got sharper than they can be. `TtmSqueeze` was checked and left alone: it already regresses a detrended series. Fixture review, every moved value scored against the same exact rational fit over the golden candles. All eight directly-scored fixtures improved, on data priced around 100 where there is little to cancel: LinearRegression 3.5e-15 -> 2.4e-16 RSquared 8.1e-13 -> 1.3e-14 LinRegSlope 2.2e-13 -> 6.1e-15 StandardError 1.6e-12 -> 1.9e-14 LinRegIntercept 3.7e-15 -> 1.5e-16 DetrendedStdDev 1.6e-12 -> 2.0e-14 LinRegAngle 2.2e-13 -> 6.0e-15 Tsf 4.1e-15 -> 2.1e-16 The angle is scored through its own tangent, since exact rational arithmetic has no atan. Cfo, Inertia, ProjectionOscillator, TsfOscillator, LinRegChannel, ProjectionBands and StandardErrorBands have no independent closed form here and were checked for emission stability and for moving only at rounding scale; the largest is 9.6e-13. Emission positions are unchanged everywhere. g_LinRegAngle moves legitimately this time rather than through the non-reproducibility recorded as A5c, and regenerating twice now yields an identical file. A5c stays open. One regression test per affected indicator, each against a centred fit rather than the price-level form. Verified the StandardError one bites by reinstating the raw total: it fails with "collapsed to zero at bar 19". C ABI rebuilt and the C# golden suite run: 556 pass.
The earlier sweep for this pattern matched only a bare `mean * mean`, which is how it managed to report the catalogue clean while eighteen files still carried the defect. Redone against the spellings that actually occur: suffixed identifiers, `n * mean_y * mean_y`, cross-sum products `sum_a * sum_b`, `powi(2)`, and `mul_add` -- the last of which is what had hidden `TrendStrengthIndex` from every previous pass. Three were left. Measured at a price level of 1e8 with a one-unit wobble, against a centred reference: TrendStrengthIndex 1.0 -> 2.4e-14 CorrelationTrendIndicator 1.0 -> 1.2e-14 VwapStdDevBands (deviation) 32.2 -> 7.6e-14 A relative error of exactly 1 is not a precision problem. Both correlations built `n·Σy² − (Σy)²` from raw prices, which at that level collapses past their own zero guard, so both returned 0 for a clean sine wave -- reporting no trend and no correlation whatever the input did. They now run on deviations from the window mean, under which a correlation is invariant. `VwapStdDevBands` reported a band width 32 times too wide, and unlike the rest it accumulates over a whole session rather than a sliding window, so there was no eviction and nothing to bound it. It now holds its weighted moments relative to a reference price seeded from the first bar of the session, restored only for the absolute band levels. The sweep also settled what does *not* need changing, recorded so it does not get revisited: `SpearmanCorrelation` correlates ranks, which are bounded by the window length; `AutocorrelationPeriodogram` correlates a roofing-filtered series that oscillates about zero; `HurstExponent` fits a log-log relationship; and `DepthSlope` regresses on distances from the mid. Every `n·Σx² − (Σx)²` over the index `0..period` is exact arithmetic on small integers. `wickra-data` is clean. Two sites remain by design: `StandardError` and `DetrendedStdDev` still rebuild the residual sum of squares by subtracting the explained variation. Both terms now come from shifted sums and the exact-rational score is 8.7e-15, but the subtraction still cancels when the fit is nearly perfect, and removing it would cost O(period) per update on two indicators that are O(1) today. Recorded as A5.16 rather than left implicit. Fixture review, against an exact rational computation for the two correlations and a two-pass weighted one for the bands. All three improved: TrendStrengthIndex 6.6e-13 -> 1.5e-15 CorrelationTrendIndicator 3.3e-13 -> 7.7e-16 VwapStdDevBands.stddev 1.1e-13 -> 8.8e-16 Emission positions unchanged. One regression test each, and each asserts the signal is actually present rather than only that it matches -- an indicator stuck at zero would otherwise satisfy a comparison against itself. Verified the correlation one bites by reinstating the raw form. C ABI rebuilt and the C# golden suite run: 556 pass.
Every scalar indicator in the C ABI had a vectorized path and no multi-output one did, so C, C++, Go, C# and Java crossed the FFI boundary once per bar for a fifth of the catalogue. The audit counted 161 handles without a batch; measuring splits them into 92 multi-output indicators and 69 others -- derivatives, cross-section, trade, order book, bar builders, profiles and the data-layer types -- which have different reasons and are recorded separately. `out` is an array of the `#[repr(C)]` output struct the ABI already defines, one entry per input, not the flat `n * k` double buffer the todo assumed. One of the ninety output structs carries an `int64_t` lag, which a flat double buffer would silently truncate. A row the indicator did not produce -- warmup, or an input it rejected -- has every floating-point field set to `NaN`; every output struct has at least one such field, so the warmup signal is universal. `bindings/c/src/lib.rs` is generated, so the work is in the generator (ScriptHelpers 45180fe), as are the Go, C# and Java additions: all three only knew how to emit a batch whose `out` is `double*`. Regenerating turned up that the gap-filling resampler had been edited straight into that generated file. The first run deleted `wickra_resampler_push` and `wickra_resampler_drain` and reinstated the old one-candle-per-push API. Nothing about that fails to compile -- it showed up by diffing the exported function set before and after -- so the generator now carries the streaming resampler and the regenerated file loses no function at all. R is deliberately unchanged: its generator emits a batch only for the `double*`-out shape, and the R surface returns a list per update, so a batch would be a matrix or a data.frame -- a design decision rather than a mechanical port. Regenerating R against the new header produces a byte-identical file, which also re-confirms that the roxygen override table reproduces the hand-written documentation exactly. Two generated tests cover the new path at the ABI: one replays a multi-output batch against the streaming calls row by row, including the NaN warmup rows, and one checks that a null handle, input or output is a defined no-op. The Rust chain and clippy are clean, the C ABI suite is at 6 tests, and Go, C# (560) and Java (553) all pass against the regenerated bindings.
`napi.triples` and `napi.name` are both deprecated, and every `napi build` printed a warning about the first. The second matters more than a warning suggests: `name` sets the binary name, and the CLI default it falls back to is `index`, so losing the compatibility shim would rename every published `.node` artifact. `triples` becomes a flat `targets` list -- `defaults: false` meant only the `additional` entries ever counted -- and `name` becomes `binaryName`. Verified as a no-op rather than assumed: the deprecation warning is gone, the built artifact is still `wickra.win32-x64-msvc.node`, and `index.js` and `index.d.ts` are byte-identical across the change. The Node suite passes at 1119 tests.
Every multi-output `update` built a `js_sys::Object` and handed it back as a
`JsValue`, which wasm-bindgen writes into the type definitions as `any`. A
TypeScript caller lost the field names and their types for 94 of the 517
indicators -- the exact place types are worth most, since the object shape is
the only thing that says what the numbers mean.
Each of those classes gets an extern type carrying a `typescript_type`, the same
mechanism `BoolArray` introduced for `boolean[]`. It renders the object literal
inline in the signature, so `ADX.update` now reads
{ plusDi: number; minusDi: number; adx: number } | undefined
and nothing new appears in the public surface. Every field is `number` because
all 339 `Reflect::set` calls write an `f64` expression -- checked rather than
assumed. The bucket profiles became `Option<Float64Array>` and the resampler
flush an optional candle, neither needing a type of its own. `any` no longer
appears in any method signature the definitions declare.
Behaviour change, and the reason for the `!`: a warming-up `update` returns
`undefined` where it used to return `null`, because `Option<T>` is what
wasm-bindgen maps that way. Every scalar `update` has always returned
`undefined`, so the two halves of the same API disagreed; this settles it in
favour of the majority. `== null`, `?.` and `??` already cover both, but a
strict `=== null` does not. 37 doc comments saying "else `null`" are corrected.
The WASM suite passes at 1038 tests, including the 514-indicator golden replay
and the batch parity added earlier, and the Rust chain and clippy are clean.
The bar builders, the data-layer candle streams and the Footprint and
bucket-profile batches all handed back a `js_sys::Array`, which the generated
type definitions write as `Array<any>`. Twenty-six signatures said nothing about
their elements -- and for a bar builder the element shape is the entire result,
since the numbers differ per builder.
Thirteen extern types cover them: one per bar builder, one shared candle array
for `TickAggregator.push`, `Resampler.update` and `CandleReader.read`, one for
the Footprint batch and one for the profile batch. `RenkoBars.update` now reads
`{ open: number; close: number; direction: number }[]`.
With the multi-output returns typed earlier, `any` no longer appears as a type
anywhere in the definitions; the remaining textual hits are the word "many" and
one prose sentence.
The five batches that reuse `update` cast back with `unchecked_into::<Array>()`
to iterate it. That keeps the object construction in one place, which is the
reason those batches call `update` at all rather than duplicating the field
writes the way the older bar builders do.
The profile batch pushes `undefined` for a warmup bar now instead of `null`, so
it agrees with what `update` returns.
Checked at runtime as well as in the declarations: `RenkoBars.update` and
`TickAggregator.push` hand back exactly the shapes they now advertise. The WASM
suite passes at 1038 tests and the Rust chain and clippy are clean.
These are the handles the multi-output pass left that still emit one value per input, so they take the loop the candle indicators always had: 16 derivatives indicators, 8 trade, 3 trade-with-quote, and MACDEXT. MACDEXT is worth naming because it was missed for a reason that has nothing to do with its shape. It is an ordinary multi-output `f64` indicator, but its constructor takes moving-average type codes, which the generator cannot express, so it lives in the hand-written section and the multi-output pass never saw it. Of the 517 handles, 475 have a batch now. The 42 that do not are not more of the same: the cross-section and order-book indicators take per-bar arrays, so a batch would need an array-of-arrays and a C shape worth designing rather than pattern-matching; the bar builders and profiles have data-dependent output length, so the caller cannot size the buffer up front; and three are data-layer types rather than indicators. The trade shape exposed a defect in the Java generator: `emit_batch` wrote `a.allocateFrom(JAVA_BYTE, boolean[])`, an overload that does not exist, so the whole Java module stopped compiling the moment a batch took a `const bool*` side column. The cross-section `update` already used `WickraNative.boolSegment` for exactly that; the batch path does now too. The new `trade_batch_matches_streaming` test targets that column: it checks the batch against the streaming calls, then flips the side flags and requires the result to move, so a side column that never reached the indicator fails instead of passing by coincidence. Generators updated in ScriptHelpers 8e7baf3. Rust chain green, clippy clean, C ABI at 7 tests, Go green, C# 560, Java 553.
These 22 were left out of the earlier passes because they take an array per bar rather than a scalar, so a batch needs two dimensions. The shape that works in C is the flat one: the member and level arrays cover `n * members` elements with the stride passed explicitly, which also states the constraint these indicators already have -- every snapshot carries the same universe size. The batch walks that array a snapshot at a time by calling the existing `update` with an offset pointer, so the `Member` and `Level` construction stays in one place instead of being copied into a second loop. 497 of the 517 C ABI handles have a vectorized path now. The 20 remaining are the bar builders, the profiles and the data-layer types, whose output length depends on the data rather than on the input length; the resampler's push-and-drain pair is the pattern to follow there, not another out-buffer. Two tests, one per shape, each comparing the batch against the streaming calls. Both also assert that the first and last bar differ, because a wrong stride would produce entirely plausible numbers and pass a same-value check. The Go, C# and Java generators needed no guard: they only wrap a batch for the shapes they know how to express, so the new functions appear in their low-level declarations and nothing broke. Confirmed by regenerating all three -- only the `NativeMethods` files changed, and Go, C# (560) and Java (553) all pass. Generators updated in ScriptHelpers.
…ing it
I went looking for a way to give the bar builders a batch and found that
`update` was already broken.
One candle can complete any number of bars. A Renko builder with a box size of 1
turns a 500-point move into 500 bricks. `wickra_<builder>_update` writes up to
`cap` bars into a caller buffer and returns how many the candle actually
completed -- and every generated binding passes a hard-coded 64, then indexes
that buffer with the returned count. Reproduced before changing anything:
Go panic: index out of range [64] with length 64
C# indexes a 64-element stackalloc Span with the same count
Java reads past a 24*64 MemorySegment
This is ordinary input: any gap, or any candle large relative to the box size.
The bars beyond the buffer were gone in any case, since the builder had already
advanced.
The handle wraps the core builder alongside a pending buffer now, and a new
`wickra_<builder>_drain` yields whatever did not fit -- the shape the resampler
already uses for exactly this. `update` keeps its signature and its meaning, so
no existing call changes; the Go, C# and Java wrappers drain after filling their
buffer.
Each language gets the same 500-brick test, and each asserts the bricks form one
consecutive ladder rather than only checking the count -- a brick dropped or
duplicated at the boundary between the buffer and the drained remainder fails
that, a length check would not.
Generators updated in ScriptHelpers f9d31c5. C ABI at 10 tests, Go green, C# 561,
Java 554, Rust chain and clippy clean, and gofmt clean on the regenerated Go.
…cating The two families left were left for different reasons, and needed different answers. A bar builder completes an unpredictable number of bars, so its batch feeds the whole series and buffers everything into the pending list the drain fix added, returning the count for the caller to drain. A profile emits a fixed number of bins, set by its constructor, so its batch is a flat `n * width` block with `NaN` rows while warming up; the two that also emit scalar fields get an `n`-length array of those alongside. That closes it: all 514 indicators have a vectorized path, and the only three C ABI handles without one are the data-layer types, which are not indicators. Footprint turned out to have the same overflow the bar builders did, and no drain to fix it -- which means the clamp I added with them made it worse than the crash it replaced. The returned list was sized to the reported level count while only the first 64 entries were ever written, so the tail was zero-valued levels: silently wrong numbers instead of a loud failure. It reports one level per distinct price seen, so any session spanning more than 64 ticks of range reaches it. It is wrapped like the bar builders now, with a drain and a batch. Four tests, each checking more than a count. The bar-builder batch is replayed against feeding the same candles one at a time and every brick compared field by field. The profile batch is compared against the streaming rows including the NaN warmup rows. The Go Footprint test walks a 200-tick range and fails on any zero-valued level. The Renko test from the previous commit still requires one consecutive ladder. Generators updated in ScriptHelpers 966bba3. C ABI at 12 tests, Go green, C# 561, Java 554, Rust chain and clippy clean, gofmt clean.
39 of the 514 indicators had a vectorized path in the C ABI that the Go wrapper could not call. The generator's `emit_batch` assumed every pointer parameter is a per-bar series and that `n` is the first one's length, which covers the scalar and multi-output shapes and nothing else. The families it missed are the ones whose per-bar input is itself an array, or whose output length depends on the data rather than on the input length. Four emitters cover them. Cross-section and order book take flat `n * stride` groups: the `uintptr_t` that closes a group carries its stride, and a group with no closing count is one element per bar, which is what separates the cross-section timestamps from the member arrays. Bar builders feed the series, size a buffer from the returned count and drain it. Profiles get one flat allocation whose rows are slices into it, the width coming from the constructor. `n` is taken from the per-bar array where there is one and from `len(first) / stride` otherwise, and every slice length is checked against it rather than trusted. Four tests, one per shape, each comparing against feeding the same data one bar at a time -- including that a profile row is NaN exactly when the streaming call reports warmup, so a batch that quietly emitted during warmup would fail. C# and Java still have the original assumption and remain 39 short; that is the next step, not an oversight. Generator updated in ScriptHelpers 2bc97e3. Go suite green, go vet clean, gofmt clean, Rust chain and clippy clean.
The same 39 indicators Go could not reach were unreachable from C# and Java for the same reason: their generators assumed every pointer parameter is a per-bar series and that `n` is the first one's length. That holds for the scalar and multi-output shapes and for nothing whose per-bar input is itself an array, or whose output length depends on the data. The four emitters mirror the Go ones, adapted to each language. C# takes `ReadOnlySpan` and returns arrays; Java takes arrays throughout and allocates from a confined `Arena`. The Java bar-builder and cross-section paths reuse `WickraNative.boolSegment` for the `bool*` columns, which is the helper the trade batch needed a few commits ago -- the second time that gap would have broken the build had it not already been fixed. All three generated bindings now expose a batch for every one of the 514 indicators. The only classes without one are `BinanceFeed` and the three data-layer types, none of which are indicators. Four tests per language, one per shape, each comparing against feeding the same data one bar at a time, and each requiring a profile row to be NaN exactly when the streaming call reports warmup. One slip, caught by the C# compiler rather than by review: a jagged array is `new double[n][]`, not `new double[][n]`. Generators updated in ScriptHelpers effb6ee. C# 565, Java 558, Go green, Rust chain and clippy clean.
The bindings that need a caller-sized buffer -- C, Go, C#, Java -- guessed how wide a profile is from its constructor parameter names, looking for "bin", "level", "bucket", "slot" or "count" and falling back to 4096 elements when none matched. A heuristic over parameter names is wrong the moment an indicator does not happen to name its parameter that way, and `DayOfWeekProfile` takes only a UTC offset, so it got the fallback for a profile that is seven wide. `update` has always over-allocated there; giving it a batch multiplied that by the series length. The six profile indicators expose `width()` now -- the number of values every emitted profile carries, fixed for the lifetime of the indicator, which is exactly what lets a caller size a buffer before the first value arrives. The C ABI re-exports it as `wickra_<p>_width` and the three generators ask for it after constructing the handle, so the heuristic and its fallback are gone entirely. Six core tests, one per profile, and none of them checks `width()` against the constructor argument -- that would only restate the implementation. Each drives the indicator until it emits and asserts the payload length equals `width()`, which is the property that makes the accessor worth having. A Go assertion pins the visible consequence: a weekday profile row is 7 long, and fails at 4096. Generators updated in ScriptHelpers a2ca3c4. Rust chain green, clippy clean, Go green, C# 565, Java 558.
…heir buffer `batch()` was wired only for the scalar indicators, so 158 of the 514 had a native batch R could not call. The audit named the multi-output ones, and they turned out to need no change to the R surface at all: `batch()` is dynamic, it builds the C symbol from the object prefix, and a multi-output indicator takes the same equal-length double columns a scalar one does. The design question the todo left open answers itself -- an `n x k` matrix with the field names as columns, which is what R wants and what the bar-builder shim already returned. 475 of the 514 have a batch now. The 39 left are the four array-shaped and variable-length families, and those really do need a design decision: the `batch()` generic coerces every argument to a double column and requires equal lengths, which a stride scalar and per-bar arrays are not. Regenerating also turned up that the R bar builders had the overflow the other three bindings had -- and here it was undefined behaviour rather than a checked failure. The `update` shim declares a 64-element stack array, passes 64 as the capacity, and then reads it up to the count the builder returns, which is how many bars the candle completed rather than how many fit. Go panicked on this, C# and Java raised; C simply read past the array. It drains the surplus now. `R/indicators.R` regenerates byte-identically, so the standing caution about it is satisfied by the roxygen override table rather than bypassed; only `src/wickra.c` changed. Verified by building and running the package rather than by reading the generated C: R 4.6.0 and rtools45 are installed here, and `WICKRA_INCLUDE_DIR` / `WICKRA_LIB_DIR` point `configure` at the local C ABI. The suite passes at 120010 assertions, including a new file whose bar-builder test requires a 500-brick move to come back as one consecutive ladder rather than merely counting the rows. Generator updated in ScriptHelpers eb73ecd.
The generated golden suite replayed the shared input through `update` and checked it against the Rust fixtures. `batch` was never exercised, so a batch that disagreed with streaming -- or that did not exist -- passed unnoticed for all 514. It now runs both passes against the same fixtures. Three archetypes need a different comparison, and saying so is the point rather than a wrinkle: a bar builder completes an unpredictable number of bars per candle, so its batch is the concatenation and cannot be split back into rows; and Footprint reports the whole book after each trade, so its batch holds the final snapshot, which is the fixture's last row. 511 of the 514 passed immediately. The three that did not were all on the warmup row, and all the same mistake: a row the indicator did not produce came back as zeros, which is indistinguishable from a value it did produce. `LeadLagCrossCorrelation` is the one multi-output struct with an integer field, and an integer has no NaN, so its lag read back as 0 while streaming reports the whole row as absent. I had written that down as a caveat when adding the multi-output batch; the test showed the caveat was the defect. Integers surface as `f64` in the C output structs now, so every field of every one can carry NaN -- the lag is a bar offset and is exact in an `f64`. `TpoProfile` and `VolumeProfile` NaN-filled their values block but left the scalar array untouched, so those fields read back as zeros. A spot-check would likely have skipped all three: they are on row 0 of three indicators out of 514. With the WASM batch suite added earlier, two independent language paths now cover every indicator's batch. C#, Java and R keep their per-shape tests, which is where their own marshalling lives. Generator updated in ScriptHelpers f3585ed. Rust chain green, clippy clean, Go green with 514 batch subtests, C# 565, Java 558, R 120010 assertions.
The documentation said every indicator updates in O(1) per tick. That reads as constant work regardless of the configured period, and it is false: CCI averages absolute deviations over its whole window on every tick, RollingQuantile sorts its window, and a static pass over the 479 `Indicator::update` bodies finds 106 that iterate or sort a window-sized field. Some of those loops are warmup-only seeding, so the exact figure is softer than 106 -- which is why the new wording carries no number. `traits.rs` has always stated the real contract, and it is a strong one: O(1) *in the input length*. A tick never triggers a pass over the history behind it, and the cost is bounded by the window you configure rather than by how much data has gone before. That is what makes streaming and backtesting share an implementation, and it survives the correction intact. The claim appeared in twelve tracked files, all of which ship: the README, the benchmarks, the architecture notes, the roadmap, and all eight binding READMEs that go to crates.io, PyPI, npm, NuGet, Maven Central and CRAN. The Benchmarks section now names the two shapes that scale with the period instead of implying none do. `CHANGELOG.md` keeps its wording. Those entries record what was claimed at the time, and editing them would be a different kind of inaccuracy. The `sync-about` README check still matches, and the `all 514` that replaces `O(1)` in the comparison table cannot be caught by its `[0-9]+ indicators` pattern. The 19 occurrences in `wickra-docs` are recorded with the other docs corrections, which cannot be pushed until the release.
`Resampler::update` became `push`, returning the candles a bar closed and
taking a `gap_fill` flag, and no example followed. The C, C#, Go and Java
`multi_timeframe` examples no longer compiled, nor did
`examples/c/data_layer_test.c`; the Node, WASM and Python ones treated the
returned list as a single candle, which crashed at runtime or aggregated
nothing. Four WASM demos also guarded a warmup value with `!== null`, but a
WASM `update` returns `undefined` before warmup completes, so the guard passed
and the code read fields off nothing.
Separately, `backtest.py`, `multi_timeframe.py` and `parallel_assets.py` have
raised `TypeError` since `batch` stopped returning NumPy arrays: a scalar batch
returns `array.array('d')` and a multi-output batch a `Matrix`. `Matrix`'s
docstring promised that `numpy.asarray(result)` rebuilds an `(nrows, ncols)`
array; it yields a 0-d object array, which is why the test suite detours
through `tolist()`. The buffer protocol would make the promise true but needs
`Py_buffer`, outside the limited API the abi3 wheels build against, so the
docstring now gives the incantation that works and says why.
All eight `multi_timeframe` examples were run and agree to the digit with
`cargo run -p wickra-examples --bin multi_timeframe` — the browser one by
lifting its own resample and summarize out of the page and driving them from
the node-target build.
The root cause is that CI only parsed these examples. The `node` and `python`
jobs now run the offline ones after building the module and installing the
wheel, the way the C, C#, Go and Java jobs already did, which is why those had
not drifted.
Go, WASM, C#, Java and R each replay all 514 indicators through `update` and compare against the Rust fixtures. That checks what an indicator computes and nothing else: a `reset` that forgot a field, or an `isReady` keyed off a value that happens to move at the right moment, replays perfectly clean through every one of them. Each of the five now also asserts that a fresh indicator is not ready and declares a warmup of at least one, that a fully driven one is ready whenever its fixture holds a finite value, and that a second pass after `reset` reproduces the first bit for bit — equality, not tolerance, since a tolerance would hide exactly the leftover state this is looking for. Whether a fixture ever emits is decided from the fixture rather than inferred at runtime, because a row may legitimately be NaN. The ten bar builders implement `BarBuilder`, not `Indicator`, so they are held only to the `reset` half. Each pass was verified by removing one indicator's `reset` and confirming the suite fails. The Go and C# generators grew a per-indicator driver that both the value pass and the lifecycle pass call, so the two cannot feed different streams. Java and R already had one reflective runner; R's was private to its test body and is now at file scope. The WASM suites had carried two copies of the same harness, which is two chances to drift, so that half moved to tests/harness.js and all three suites drive from it.
Each had a BatchShapes-style test covering one indicator per awkward input shape, which is what it was for, and nothing that drove the whole catalogue through the batch path. A batch that disagreed with streaming, or that mis-shaped a warmup row, only had to avoid those few to pass. C# and Java now replay all 514 through `Batch` against the same fixtures the streaming pass uses, with the archetype's whole-series columns built from the same per-bar values `update` gets. Both were verified by feeding a scalar batch the open instead of the close: Java fails 147 subtests, C# fails on the first post-warmup row. R covers the 475 indicators that have a batch shim. The other 39 — the cross-section, order-book, profile, bar-builder and footprint families — have none at all, because the generic column forwarding in `batch()` cannot express a per-bar snapshot. That is a real gap in the R binding, so the test names the families and pins the 475/39 split rather than quietly skipping them. Node was not in the audit's list for the lifecycle pass and had the same gap its siblings did — one `reset` call across 514 indicators. It gets the same pass, and its golden harness moved to `__tests__/harness.js` first so the two suites drive from one copy.
Each binding is generated or written separately and tested separately, so a method that goes missing in one of them fails nowhere. That is how the WASM binding shipped 73 classes without isReady/warmupPeriod and 63 without batch, and how the Node loader shipped 518 exports that resolved to undefined. Nothing compared the bindings to each other. scripts/check_binding_surface.py reads the wickra_<snake>_<method> symbols out of the C ABI header — the one artefact every binding consumes — derives what each of the 514 indicators must expose, and checks that against the public surface of C, Go, C#, Java, Node, Python and R, each in its own spelling. The check is two-sided: a bar builder that grew an isReady fails as loudly as an indicator that lost one. Verified both ways, by renaming Rsi.IsReady in Go and by adding an IsReady to C#'s RenkoBars. Extra methods are allowed. Node and Python publish parameter accessors that Go and C# do not, which is language idiom rather than drift, so the check asserts the contract is present, not that nothing else is. R's 39 missing batch shims are the one waived gap and are pinned by count, so it cannot widen unnoticed. WASM is the one binding whose surface is a build artefact rather than a file in the tree, so it cannot be read statically. Its completeness test now asserts the same contract from the same manifest at runtime, replacing a hand-maintained class count that said 504 and knew nothing about which methods belong on which shape. This adds a `binding-surface` job, so ci.yml reports 29 checks rather than 28.
475 of the 514 indicators had a batch R could call. The cross-section, order-book, profile and bar-builder families had none, because their C batch takes a per-bar width or hands its output back through a drain, and neither fits the scalar or multi-output emitter. Every other binding covered all 514. A cross-section or order-book snapshot arrives as one flat column per field — bar i at [i*width, (i+1)*width) — with the width passed alongside it, the same shape the Java and C# batches take. A profile comes back as an n x width matrix, the width read from the handle rather than asked of the caller. A bar builder's batch reports what the whole series completed and drains it, so the result is as long as the data makes it rather than one row per input. `batch()` gained the one rule that makes this expressible: a named argument is a per-bar width, an unnamed one is a column. With no width given the guard against a short column being read past its end is unchanged, which is every call that existed before. The R golden suite now drives all 514 through batch instead of the 475 that had a shim. Its first run failed on the cross-section family because the test built its snapshot columns interleaved rather than bar-major — the layout the Java and C# passes already agree with the fixtures on, so the test was wrong and the shims were right. The comparison is vectorised, one expectation per indicator rather than one per value: testthat records every expectation, and at 514 indicators by 80 rows by up to 52 columns that bookkeeping dominated the run against the R job's 30-minute budget on three runners. Two manual pages were stale while regenerating the docs for the new signature. `Resampler.Rd` still showed `Resampler(timeframe)` after the resampler gained `gap_fill`, and `push.Rd` was equally behind; the R job runs `R CMD check` precisely to catch that, so it was failing there. The package description also called every indicator an "O(1) streaming state machine" — the same unqualified claim corrected elsewhere, and the copy that shows on CRAN and r-universe. The binding-surface check no longer waives R's 39 missing batches.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
The branch had never met the matrix — ci.yml fires on push-to-main and pull_request only — so 72 commits were verified on one Windows machine and nothing else. Draft PR #393 ran it: 49 checks passed, 8 failed, and every failure was real and unreachable from here. `manual_midpoint` on eight core indicators, from CI's stable toolchain being 1.98 against 1.92 locally. Averaging two values as `0.5 * (a + b)` is the same arithmetic as `f64::midpoint` for any pair under `f64::MAX / 2` — both scale by a power of two exactly — so nothing moves; regenerating every golden fixture confirms it, not one changed, and the Node suite agrees against a rebuilt binding. Ehlers' high-pass gain keeps its formula by naming the coefficient rather than inlining a midpoint of 1 and alpha. `node --test bindings/wasm/tests/` passed a bare directory, which Node 22 on the runner resolves as a module path and fails to load. This branch introduced it in 120b6ac and nothing ran it until now. The glob form works on both versions and also keeps the shared `harness.js` out of the run, which the directory form only tolerated by accident. `R CMD check` runs the roxygen examples, and the one under `flush()` still drove the resampler with `update()` after it moved to `push`/`drain` — so the symbol it calls no longer exists, and anyone copying the example hit the same wall. All 532 manual pages now run their examples clean.
The vignette introduces the package by saying every indicator is fed "one observation at a time with `update()` (an O(1) streaming step)". That is the blanket claim corrected everywhere else: it holds for most of the catalogue and not for the ones that need an order statistic or a full-window pass, whose own documentation already says `O(period log period)`. This copy ships to CRAN and r-universe, so it was the last user-facing instance. Checked for siblings while here. Of the 97 indicators whose rustdoc mentions a per-update cost, none claims O(1) while sorting or scanning its window, and 27 state a non-constant cost honestly. The five benchmark-harness comments that mention an "O(1) streaming engine" describe the SMA/EMA/RSI-class indicators those harnesses actually measure rather than the catalogue, so they are left as they are.
…list D1 and D2 were closed on the invariants suite covering "all 513 indicators". The catalogue is 514. Diffing the types the suite constructs against FAMILIES named the odd one out: vwap.rs defines both Vwap and RollingVwap, only Vwap was ever listed, and so RollingVwap alone was never held to the warmup, readiness, reset and non-finite contract. A missing macro call is a test that does not exist, which is exactly why nothing reported it in three years of the file growing. It passes as written, so this was a coverage gap rather than a latent defect. The guard is the more useful half: FAMILIES is the catalogue, so it can say what the list should contain, and reading this test file back at compile time is the cheapest way to compare the two. One catalogued name can be the suffix of another, so the match requires that the character before it cannot continue an identifier, or Vwap would ride on RollingVwap's entry. Verified by deleting the new entry and watching the guard name it. The first version did not: the haystack is its own file, and the comment explaining the suffix rule spelled a constructor out in prose, so the name answered for itself. The comment now says why it must not. The [Unreleased] notes claimed "all 513 indicators" in three places; they say 514 now.
The R job used to fail at `R CMD check`, so CI never reached the step that runs the offline R examples. With the check green it ran for the first time and failed: `multi_timeframe.R` still called `update()` on a Resampler, and that symbol went away when the resampler moved to push/drain. The seven other languages were repaired for this in C3b; R's example was missed because nothing local runs it. It now prints what the other seven print — 1200/240/80 bars with EMAs of 114.4200, 117.1417 and 113.7448.
The macOS run failed the batch pass on four accumulating indicators with the message "Adl: 2 of 80 values differ; first at 50 (got 24.80896 want 24.80896)". Both numbers are R's default seven digits, so the message says a difference exists and nothing about its size — which is the one thing needed to tell a last-bit rounding difference from a real one. It now prints seventeen digits and the relative delta. The failure does not reproduce here: on x86 the R batch is bit-exact against the fixture, and Go, C#, Java and the C ABI all pass the same comparison against the same fixtures on the same arm64 runner. The fixtures are not the cause either — gen_golden writes shortest round-trip, so parsing them is exact. Loosening the bound before knowing the magnitude would hide whichever it turns out to be.
The macOS R job failed four indicators by more than 1e-12, and the improved diagnostic gave the sizes: 1.015e-12, 2.491e-12, 3.217e-12 and 1.000e-11. It also corrected an earlier misreading of the log — the streaming pass fails on the same four, at the same rows, with byte-identical values. The library's two paths therefore agree exactly with each other and both differ from the fixture, which rules out a batch defect and points at the inputs. Every other binding parses the golden input with a correctly rounded decimal parser and reproduces the fixture bit for bit on that same arm64 runner. R parses it with its own, which differs by a last bit there. Four indicators amplify that bit because each subtracts nearly equal quantities: `2*close - high - low` is near zero when the close sits mid-bar, `price - mid` when a trade prints at the mid, and Adl and ChaikinOscillator are cancelling differences too. Only 4 of 514 cancel hard enough on this input, which is the pattern observed. So this is a harness precision limit rather than a library defect, and the bound should say what is achievable. 1e-9 keeps a hundredfold margin over the worst measurement and stays a thousand times tighter than the libm bound, so a real regression could not hide behind it. The measurements and the mechanism sit next to the list. libm_dependent.txt was left alone: its stated premise is indicators that reach a transcendental, and none of these four does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remediation of the full audit: 78 commits across the Rust core, the C ABI and all
eight bindings, the examples, the docs and CI.
CI is green: 57 checks, 0 failures. Three OSes across Rust, the C ABI and
every binding, MSRV 1.86 and 1.88, Coverage, Fuzz, Clippy, supply-chain, the
manylinux/musllinux wheel smoke, CodeQL, the workflow audit, and a new
binding-surface job.
Phase A — the core
Rolling variance was computed as
E[x²] − E[x]²over raw price levels andclamped at zero, which cancels catastrophically at realistic price magnitudes and
hid the failure instead of exposing it. The dispersion family now accumulates
around a window reference point; the third and fourth central moments followed.
Also:
MAX_PERIODbounds the accepted window, the panic strategy isunwindsobindings can raise rather than abort,
Trix::is_readyno longer reports ready aninput early, the DI/DM family reports
period + 1, and a zeromax_reconnect_attemptsis rejected at the entry point rather than panickinglater.
Phase B — the bindings
Every binding had drifted from the C ABI in its own direction: Java could
use-after-free and marshalled candle timestamps as
double; C# passed a rawpointer where a
SafeHandlebelonged; R read past the end of a short batchcolumn; the WASM binding shipped 73 classes without
isReady/warmupPeriodand63 without
batch; the Node loader shipped 518 exports resolving toundefined;the bar builders were losing bars in three bindings; and
batchwas missing fora large part of the multi-output catalogue across the C ABI.
Phase C — docs and examples
The unqualified "O(1) per update" claim is now qualified to what the code
promises. More seriously, the resampler change had broken
multi_timeframeinall eight languages — C, C#, Go and Java no longer compiled — and three Python
examples had raised
TypeErrorsince the NumPy dependency was dropped. CI couldnot see any of it, because it only parsed those files; the
nodeandpythonjobs now run the offline examples the way the Go, C# and Java jobs already did.
Phase D — contract tests
The systemic gap behind most of the above: every suite pinned
updateandnothing else. All 514 indicators are now driven through
batchand through thelifecycle contract — fresh is not ready with a warmup of at least one, driven is
ready, and a second pass after
resetreproduces the first bit for bit — in Go,WASM, C#, Java, R and Node. R gained the
batchshims for the 39 indicators thathad none.
scripts/check_binding_surface.pycompares all eight bindings againstthe C ABI header and runs as a new job, so ci.yml reports 29 checks rather than
28.
Phase E — what opening this PR found
The branch had never met the matrix, and the first run returned 11 failures that
were unreachable from a single Windows machine:
clippy::manual_midpointon eight indicators, because CI's stable toolchain isahead of the one used to develop this.
f64::midpointis the same arithmeticunder
f64::MAX / 2; regenerating every golden fixture confirmed not one valuemoved.
node --test bindings/wasm/tests/passed a bare directory, which Node 22 onthe runner resolves as a module path. This branch introduced it and nothing had
run it.
fixed:
R CMD checkon a stale manual page, then — with the check green, so CIreached the step for the first time — an R example still calling
update()ona resampler.
The last R failure was a divergence of 1e-12 to 1e-11 on four indicators, on
macOS only. Rather than loosen the bound to go green, the diagnostic was improved
first; it showed the streaming and batch paths agreeing byte-for-byte with each
other and both differing from the fixture, which points at R's own decimal parser
rather than the library. Four indicators amplify a last-bit input difference
because each subtracts nearly equal quantities. They are bounded at 1e-9 with the
measurements recorded beside the list;
libm_dependent.txtwas left alone, sinceits stated premise is transcendentals and none of these four reaches one.
Docs
wickra-docsandwebpagecarry matching commits, held unpushed until releaseso the docs do not run ahead of the registries. 299 deep dives promised a
numpy.ndarraythatbatchhas not returned since the NumPy dependency wasdropped, and 30 snippets raised the moment a reader ran them. Both repos'
snippets are now executed in their own CI rather than only having their names
resolved.