Skip to content

fix: full-codebase audit — 25 defects (1 critical: stops expired at the close, positions naked overnight) - #102

Merged
yebof merged 4 commits into
mainfrom
fix/audit-2026-07-16
Jul 16, 2026
Merged

fix: full-codebase audit — 25 defects (1 critical: stops expired at the close, positions naked overnight)#102
yebof merged 4 commits into
mainfrom
fix/audit-2026-07-16

Conversation

@yebof

@yebof yebof commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Full-codebase audit: 25 defects found and fixed (1 critical, 6 major, 18 minor)

A 20-unit audit workflow over the whole repo (src 24k lines / tests 31k / prompts 2.7k) — 15 subsystem finders + 5 cross-cutting lenses (money-arithmetic consistency, concurrency/crash-safety, time/timezone, tests-as-spec, prompt↔code drift, dead guards). 102 raw findings → adversarially verified one-by-one → 25 confirmed and fixed, 16 refuted.

🔴 CRITICAL — every BUY-attached stop expired at the close; positions were naked overnight

alpaca-py's StopLossRequest has no time_in_force field, so an OTO child leg inherits the parent's tif. The entry parent is (correctly) DAY — an unfilled entry limit must die at the close rather than fill into a stale thesis — which silently made every BUY-attached protective stop a DAY order too. Alpaca expired it at 16:00 ET the same session. Any position bought in the morning and not later handed a midday/close TRAIL_STOP (the only path that used the GTC _submit_stop_limit_order) sat completely unprotected overnight — exactly when the gap risk the stop exists for happens.

Production proof: VST bought 2026-06-26 09:47 ET with SL=$158.75; the same evening's coverage reconcile logged VST held=31.0000 but only 0.0000 covered; the gap persisted until an LLM SELL exited at $152.77 on 07-01 — ~$185 worse than the stop would have capped. Control: GTC trail-stops on GE survived day boundaries in the same account. Live check: all 3 open stops today are GTC stop-limits from the trail path; not one BUY-attached stop exists.

It also contradicted the close-session prompt, which tells the reviewer to hold overnight because a broker stop is standing watch.

Fix (the leg's tif can't be set independently, so decouple): the entry stays a plain DAY limit; broker.place_entry_protection() waits for terminal, reads the actual fill, and places a GTC stop-limit for exactly that qty (this also fixes a latent bug — the OTO leg was sized to the requested qty, so a partial fill over-covered). Belt: _reconcile_stop_coverage now repairs a naked long from the stop recorded on its last BUY (the level PM/RM approved — no longer "unknown", which was the original objection to auto-repair), refusing to place at/above the live price. This retroactively protects anything the old bug left naked.

MAJOR

Defect Effect
finalize persisted the post-sell residual into the position_qty_before_sell column The drain re-derives pre − fill from the same order → fill subtracted twice → for an exact fill the residual hit 0, took the "full exit, nothing to protect" path, reported success and deleted the WAL row. The residual stayed naked forever.
get_session_close() always returned None Calendar.close is a naive datetime, not a time, so datetime.combine() raised TypeError on every call → the early-close guard was dead code and midday/close ran against a shut market on half-days. The test that should have caught it built a MagicMock with a time.
_get_sector() returned "Unknown" for every ETF yfinance .info has no sector for ETFs → max_sector_pct was skipped entirely on an ETF BUY, and a held XLV contributed $0 to Healthcare for an LLY BUY. Both directions of the cap dead, on a ~20%-ETF universe.
Constructor emitted a gross weight delta as allocation_pct Every consumer spends it as raw notional: a 6% SQQQ (3x) target deployed $6k = 18% gross, then the next session saw 18 vs target 6 and SOLD 67% of the hedge PM wanted held, repeating until raw ≈2%.
A single NaN market_value silently disabled max_total_position_pct + max_sector_pct NaN comparisons are all False → the caps switched off for the whole session, on exactly the broken-snapshot day they matter most.
macro sector_guidance was never persisted Every macro_sector_stance / macro_sector_tailwind was permanently "unknown" — the evening thesis-health step and every missed-opportunity snapshot rendered "Macro sector stance: unknown" nightly while macro was emitting OW/UW calls. Three stacked breaks (not persisted; reader wants a dict, model carries a list; overweight vs bullish).
parse_json scored a top-level list as 0 tech_analyst returns an array of per-symbol analyses; whenever the model wrapped it in prose the scan returned only the last element — a 25-symbol chunk could silently collapse to 1 with 24 discarded and no error. Reproduced.
ma_200 was unconditionally None in production lookback_days is calendar days → 120 yielded ~82 bars and len(df) >= 200 never held. The tech_analyst prompt rendered MA200=None for every symbol every day.
compute_trade_calibration ignored filled TRAIL_STOP exits Every stop-out left a phantom open lot and closed nothing (LLY read as 14 shares held while flat). The 8 real stop-outs move win_rate 22.2%→30.0%, avg_return −2.79%→−2.18% — these feed PM as facts and the reviewer as calibration_note.
earnings record_failure() marked the filing seen After the first transient LLM failure the filing was never re-queued, the 3-strike budget never reached abandoned, and PM was served the prior quarter's analysis labelled "[from cache]" as if current. Hit every symbol with a same-form analysis on disk — the whole universe in steady state.

MINOR (18)

Ex-div stop adjustment could never fire for a Monday ex-div (calendar +1 day is never a Monday for a Mon–Fri session; Friday computed Saturday) · _force_delever booked proceeds only after insert_trade, so a DB hiccup on a live SELL force-sold the next position to cover a deficit already covered · cancel_snapshotted_stops discarded the rollback's failed_specs · correlation cluster omitted the BUY symbol's own existing position (40k NVDA + 10k AVGO + 5k add scored 15%, not 55%) · target_weight_pct=0 (CLOSE) swallowed by the churn filter into a permanent HOLD · stop validated unrounded but shipped rounded (could round up to exactly the entry) · ExecutionStage discarded the fresh price_map (the 5% entry-staleness guard compared against research-time prices) · position_reviewer given raw cash while the sweep vehicle was stripped from its positions · the evening Telegram snapshot counted parked SGOV as deployed capital · reviewer prompt says "think in ATRs" but build_user_message never rendered atr_pct/stop_distance_atrs · the same SELL re-graded 2–3 nights running counted as independent sells · the same-day trim guard failed open on a partially-filled-then-canceled sell · queued-earnings cap bounded the delta, not the resulting weight · change_30d_bps measured ~57–60 days (sign-flipped on live data today) · plus the tests that pinned two of these bugs.

Three of these were regressions from today's own PR #101 (reviewer cash, ATR rendering, and the emergency×park interaction), caught by the same audit.

Notable refutations (16)

Verification earned its keep: news symbol matching already uses word boundaries; every _locked_write caller passes a single statement; the Anthropic cache accounting is correct despite a stale comment; the FIFO lot matcher handles partial exits correctly; the stuck-WAL scenario is mechanically unreachable; the retry-deadline ordering is fine given the 300s HTTP timeout.

Test plan

  • Full suite: 1351 passed (was 1310 on main; +41 regression tests, each naming the bug it locks out)
  • SDK-level verification of the critical bug (StopLossRequest field list) + reproduction of parse_json, get_session_close, and the ma_200 gap
  • Read-only live-account check confirming the naked-overnight mechanism
  • Watch the first live morning: entries followed by a GTC stop per fill, and any coverage repair on the existing book

🤖 Generated with Claude Code

https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR

yebof and others added 4 commits July 16, 2026 19:01
…tions naked overnight

Found by the 2026-07-16 full-codebase audit; confirmed against the SDK, the
production logs, and the live account.

alpaca-py's StopLossRequest has NO time_in_force field of its own, so an OTO
child leg inherits the PARENT's tif. The entry parent is (correctly) DAY — an
unfilled entry limit must die at the close rather than fill into a stale
thesis the next morning — which silently made every BUY-attached protective
stop a DAY order too. Alpaca expired it at 16:00 ET the same session. Any
position bought in the morning and not later handed a midday/close TRAIL_STOP
(the only path that used the GTC _submit_stop_limit_order) sat COMPLETELY
UNPROTECTED overnight — exactly when the gap risk the stop exists for happens.

Production proof: VST bought 2026-06-26 09:47 ET with SL=$158.75; the SAME
evening's coverage reconcile logged 'VST held=31.0000 but only 0.0000 covered';
the gap persisted until an LLM SELL exited at $152.77 on 07-01 — ~$185 worse
than the stop would have capped. Control: the GTC trail-stops on GE survived
day boundaries in the same account. Live check today: all 3 open stops are
GTC stop-limits from the trail path; not one BUY-attached stop exists.

This also contradicted the close-session prompt, which tells the reviewer to
hold overnight BECAUSE a broker stop is standing watch.

Fix (the leg's tif cannot be set independently, so decouple):
- submit_order no longer attaches an OTO leg; the entry stays a plain DAY
  limit/market and returns pending_stop_price so the caller owes a stop.
- broker.place_entry_protection(): waits for the entry to reach terminal,
  reads the ACTUAL fill, and places a GTC stop-limit for exactly that qty.
  This also fixes a latent bug — the OTO leg was sized to the REQUESTED qty,
  so a partial entry fill left a stop covering shares we never owned.
- ExecutionStage protects every filled entry after the submission burst.
- Belt: _reconcile_stop_coverage now REPAIRS a naked long instead of only
  flagging it, using the stop recorded on its last BUY (the level PM/RM
  approved — no longer 'unknown', which was the original objection to
  auto-repair). Guards: never place at/above the live price (that would fire
  instantly — an exit decision belongs to the reviewer), never invent a level.
  This retroactively protects anything the old bug left naked and covers a
  crash between an entry fill and the stop placement.

Tests: entry carries no OTO leg + stays DAY; protection is GTC and sized to
the actual fill; no-fill places nothing; stop-submit failure is swallowed;
repair from the recorded BUY stop (full/partial/refuse-above-price/no-stop/
failure/covered). 1320 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
All confirmed by an independent skeptic pass; each has a regression test.

RISK ENGINE
- NaN market_value silently DISABLED max_total_position_pct + max_sector_pct
  for the whole session (NaN comparisons are all False, so `total_pct > cap`
  evaluated False). Now blocks with a synthetic hard violation, mirroring the
  total_value guard — a broken snapshot is the day the caps matter most.
- correlation_cluster omitted the BUY symbol's OWN existing position:
  `highly_correlated_peers` (correctly) excludes the symbol itself, so an ADD
  to the biggest name in a cluster counted only the ADD. 40k NVDA + 10k AVGO
  + 5k add scored 15%, not 55%.

CONSTRUCTOR
- allocation_pct was a GROSS weight delta but every consumer (rules.py,
  ExecutionStage) spends it as RAW notional: a 6% SQQQ (3x) target deployed
  $6k = 18% gross, and the next session saw 18 vs target 6 and SOLD 67% of
  the hedge PM wanted held, repeating until raw ~2%. Convert once at the
  source; no-op at multiplier 1.0. (SELL is a ratio of gross weights — the
  multiplier cancels — so it was already right.)
- target_weight_pct=0 means CLOSE, but the min_trade_weight_delta churn
  filter turned a small dreg's explicit close into a HOLD forever.
- The stop was validated unrounded and shipped rounded, so a stop that rounds
  UP to exactly the entry passed the `stop < entry` check → risk_per_share=0
  and a stop that fires on the first tick down.

BROKER
- get_session_close() ALWAYS returned None against the real SDK:
  Calendar.close is a naive datetime, not a time, so datetime.combine() raised
  TypeError every call → the early-close guard was dead code and midday/close
  ran against a shut market on half-days. The test that should have caught it
  built a MagicMock with a `time`; it now constructs the real Calendar model.
- _get_sector() returned "Unknown" for EVERY ETF (yfinance .info has no
  sector for ETFs), which both skipped max_sector_pct entirely on an ETF BUY
  and made a held XLV contribute $0 to Healthcare for an LLY BUY.
  Deterministic _ETF_SECTORS table, consulted before the network fetch.
- cancel_snapshotted_stops discarded _restore_stop_orders' failed_specs, so a
  rollback that itself failed left shrunk coverage reported as a bare False.

PIPELINE
- finalize persisted the POST-sell residual into the position_qty_before_sell
  column; the drain recomputes `pre - fill` from the same order, so the fill
  was subtracted twice — for an exact fill the residual hit 0, took the "full
  exit, nothing to protect" early return, reported success and DELETED the WAL
  row. The residual position stayed naked forever.
- Ex-div stop adjustment could NEVER fire for a Monday ex-div: it compared
  against calendar `today + 1 day`, which for a Mon-Fri session is never a
  Monday (Friday computes Saturday). Now uses the next TRADING day.
- _force_delever booked projected proceeds only AFTER insert_trade succeeded,
  so a DB hiccup on a live SELL made the loop force-sell the NEXT position to
  cover a deficit already covered — liquidating holdings over a bookkeeping
  failure.

DATA
- macro sector_guidance was never persisted by save_last_state, so every
  macro_sector_stance / macro_sector_tailwind was PERMANENTLY "unknown" — the
  evening thesis-health step and every missed-opportunity snapshot rendered
  "Macro sector stance: unknown" nightly while macro was emitting OW/UW calls.
  Three stacked breaks (not persisted; reader wants a dict, model carries a
  list; overweight/underweight vs bullish/bearish) — normalized on write. The
  round-trip test had PINNED the bug by asserting the key was dropped; its
  real intent (keep the snapshot tiny) is honoured by storing the map without
  the `reason` prose.

1336 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
…ted views

Second pass over the audit's unverified backlog (the verify fleet hit the
session limit mid-run, so I verified these by hand — each is reproduced).

SILENT DATA LOSS
- parse_json scored a top-level LIST as 0, so whenever the model wrapped its
  JSON in any prose the candidate scan compared the array against its own
  elements and returned the LAST ELEMENT. tech_analyst returns an ARRAY of
  per-symbol analyses (`items = parsed if isinstance(parsed, list) else
  [parsed]`), so a 25-symbol chunk could silently collapse to 1 analysis with
  24 discarded and no error anywhere. Reproduced; a container is now scored by
  the SUM of its elements so it strictly outranks any element it contains.

DEAD INDICATOR
- ma_200 was unconditionally None in production: lookback_days is CALENDAR
  days (`start = today - timedelta(days=lookback_days)`), so 120 yielded ~82
  bars and technical.py's `len(df) >= 200` never held. The tech_analyst prompt
  rendered "MA200=None" for every symbol every day — the analyst was asked to
  judge trend with the most-used long-term reference permanently absent.
  lookback_days 120 -> 320 (~220 bars), with a test that pins the config
  against every indicator window the code advertises.

STALE / INVERTED VIEWS
- ExecutionStage discarded the fresh price_map from the no-SELL refresh, so
  for an ADD to a held name the 5% entry-staleness guard and the order sizing
  both used research-time prices from 5-10 minutes earlier.
- position_reviewer was handed RAW cash while the sweep vehicle was stripped
  from its positions — it saw an all-in book with a few hundred dollars spare
  while most of the equity sat parked and instantly available. Its de-lever
  mandate and weight reasoning key off that number. (Regression from the
  cash-sweep work: DecisionStage already credited it for the PM.)
- The evening Telegram position snapshot counted parked SGOV as deployed
  capital, reporting a ~99%-deployed book on a night the money was entirely in
  T-bills, and listing SGOV among the P&L movers. Parked value is now reported
  separately as what it is.
- position_reviewer's prompt instructs "think in ATRs" and the pipeline pays
  for an ATR fetch per position, but build_user_message never rendered
  atr_pct / stop_distance_atrs — the instruction referred to data the model
  could not see. (Also a regression from the exit-quality work.)

1336 tests green (+9 regressions here).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
A second verify pass (14 skeptics, one per remaining candidate) confirmed 6
and refuted 8 — the refuted set included several plausible-looking claims that
did not survive contact with the code (news symbol matching already uses word
boundaries; every _locked_write caller passes a single statement; the Anthropic
cache accounting is right even though its comment is stale).

MAJOR
- compute_trade_calibration ignored FILLED TRAIL_STOP exits, so every stop-out
  left a phantom open lot and closed nothing: LLY BUY8 -> stop-filled 8 ->
  BUY6 -> stop-filled 6 read as 14 shares still held and zero LLY trades
  closed, while the position was flat. The 8 real stop-outs in the ledger move
  the numbers materially (a typical window: win_rate 22.2% -> 30.0%,
  avg_return -2.79% -> -2.18%) — and these feed PM as facts and the reviewer as
  calibration_note. The filled-guard mirrors _build_post_exit_reality: a placed
  but unfilled stop is protection, not an exit.
- earnings record_failure() wrote the new filing_date into the manifest, and
  _check_symbol treats that as "already processed" — so after the FIRST
  transient LLM failure the filing was never re-queued, the 3-strike budget
  never reached `abandoned`, and PM was served the PRIOR quarter's analysis
  labelled "[from cache]" as if it were current. It bit every symbol with a
  same-form analysis already on disk (12 today; the whole universe in steady
  state). "Already processed" now means SUCCEEDED — confirm_filing() zeroes
  failed_attempts, so attempts 1-2 retry as designed and the 3rd abandons.

MINOR
- _build_trade_grade_summary counted each re-grade of the same SELL as an
  independent sell (the 2-day grading window has no already-graded filter, so
  evening re-grades a trade 2-3 nights running), inflating the premature/wrong
  counts that drive the reviewer's patience tilt. Deduped on (symbol,
  sell_date); a malformed row without a date is counted rather than collapsed.
- _symbols_already_trimmed_today failed OPEN on a partially-filled-then-
  canceled sell: shares left the book at midday, but close saw a clean slate
  and could trim the same name again on the same soft flag — the 2026-05-04
  AMZN double-trim this guard exists to stop. Now uses the codebase's existing
  _trade_executed_or_pending contract (a zero-fill rejection still allows a
  retry).
- _clamp_queued_earnings_buys capped the constructor's DELTA, not the RESULTING
  weight, so a name held at 15% with an unread filing could be topped up to 20%
  because the add itself was <= 5% — while the prompt and the docstring both
  promise a cap on the resulting position.
- credit_spread's "change_30d_bps" measured ~57-60 days: the reference was the
  head of a 60-calendar-day fetch rather than an observation ~30 days back
  (inherited from the earlier MONTHLY series fetcher, where it was harmless).
  On live FRED data today it reported -11.0 bps where the true 30-day change
  was +6.0 — a sign flip on a risk-off input to macro.

1341 tests green (+15 regressions here).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
@yebof
yebof merged commit 7c9b0b1 into main Jul 16, 2026
1 check passed
@yebof
yebof deleted the fix/audit-2026-07-16 branch July 16, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant