fix(log): reconcile FinancialDataLog snapshot validity against operating result#3840
Draft
TaprootFreak wants to merge 1 commit into
Draft
fix(log): reconcile FinancialDataLog snapshot validity against operating result#3840TaprootFreak wants to merge 1 commit into
TaprootFreak wants to merge 1 commit into
Conversation
…ing result The total-balance metric is built by live-polling many balance sources (exchanges, chain nodes, OTC custody). A failed or partial read can be persisted as 0 and a glitchy source can report an implausibly large value, so the logged total drifts far from reality in both directions (observed swings from ~0 to >1,000,000 CHF) and the legacy `valid` guard let these through via its unconditional 15-minute escape. A snapshot's total only legitimately moves by operating result + FX. Capture the month-to-date operating result (changeLog.total) in each snapshot and mark a snapshot valid only when its total-balance delta reconciles with the operating delta within financeLogTotalBalanceChangeLimit (FX/rounding absorbed). Glitches that break this reconciliation are flagged invalid and excluded from the valid-filtered series. The month-to-date reset is handled by falling back to the legacy escape when no same-month delta is available. This only sets the `valid` flag; no balance value is altered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
FinancialDataLogtotal-balance metric is assembled by live-polling many balance sources (exchange APIs, chain nodes, OTC custody, bank/in-transit matching). A failed/partial read can be persisted as0and a glitchy source can report an implausibly large value, so the logged total can diverge substantially from reality in both directions for the same underlying book.The legacy
validguard compared the total only to a flat limit and accepted any value after 15 minutes (|| minutesDiff > 15), so during sustained read-flakiness those glitches passed through into the valid-filtered chart series.Why this is a metric bug (not a real balance change)
The total only legitimately moves by operating result + FX.
getChangeLog()computes the operating result purely from transaction fees (buyCrypto/buyFiat/tradingminusbank/kraken/binance/ref/blockchainfees) — independent of balance reads. When a snapshot's total-balance delta does not matchoperating delta + FX, the difference is a read/aggregation error rather than a real movement (customer flow is balance-neutral: plus and minus move together).What changed
changeLog.total) in each snapshot aschangesTotal(additive JSON field on theFinanceLogmessage; no schema/migration —messageis a text column).isReconciledSnapshot(...): a snapshot isvalidonly if|ΔtotalBalance − Δoperating| <= financeLogTotalBalanceChangeLimit(FX/rounding absorbed by the existing limit). Glitches that break reconciliation are flagged invalid and excluded from the valid-filtered series.changeLog.totaldrops to ~0 at month start, so whenchangesTotal < lastFinanceLog.changesTotalthere is no same-month delta and it falls back to the legacy escape (which accepts the near-unchanged month-start total and re-establishes the baseline).This change only sets the
validflag — no balance value is recomputed or altered.Test plan
npm run type-checknpm run lint(changed files, 0 warnings/errors)npm run format:checknpm testforlog-job.service.spec.ts— 22 passed (5 new: reconciled-large-change, FX-noise-in-limit, unreconciled-jump-no-escape, legacy fallback both directions, month-to-date reset accept+reject)Known limitation (pre-existing)
Reconciliation is always measured against the last valid snapshot, so a genuine multi-day invalid gap accumulates FX into a single delta that can exceed the limit and self-perpetuate. This is inherent to the valid-filtered baseline and not introduced here.
Out of scope (separate follow-ups)
This PR makes the valid-filtered series trustworthy. It does not change the unfiltered numeric latest balance (
getLatestFinancialLog, a deliberate product choice) and does not fix the underlying read sources (distinguishing "read failed" from "genuinely zero" in the exchange/blockchain adapters, node sync checks, exchange-stream sanity bounds). Those require the dev environment with the real integrations to verify safely.