fix: full-codebase audit — 25 defects (1 critical: stops expired at the close, positions naked overnight) - #102
Merged
Merged
Conversation
…tions naked overnight Found by the 2026-07-16 full-codebase audit; confirmed against the SDK, the production logs, and the live account. alpaca-py's StopLossRequest has NO time_in_force field of its own, so an OTO child leg inherits the PARENT's tif. The entry parent is (correctly) DAY — an unfilled entry limit must die at the close rather than fill into a stale thesis the next morning — which silently made every BUY-attached protective stop a DAY order too. Alpaca expired it at 16:00 ET the same session. Any position bought in the morning and not later handed a midday/close TRAIL_STOP (the only path that used the GTC _submit_stop_limit_order) sat COMPLETELY UNPROTECTED overnight — exactly when the gap risk the stop exists for happens. Production proof: VST bought 2026-06-26 09:47 ET with SL=$158.75; the SAME evening's coverage reconcile logged 'VST held=31.0000 but only 0.0000 covered'; the gap persisted until an LLM SELL exited at $152.77 on 07-01 — ~$185 worse than the stop would have capped. Control: the GTC trail-stops on GE survived day boundaries in the same account. Live check today: all 3 open stops are GTC stop-limits from the trail path; not one BUY-attached stop exists. This also contradicted the close-session prompt, which tells the reviewer to hold overnight BECAUSE a broker stop is standing watch. Fix (the leg's tif cannot be set independently, so decouple): - submit_order no longer attaches an OTO leg; the entry stays a plain DAY limit/market and returns pending_stop_price so the caller owes a stop. - broker.place_entry_protection(): waits for the entry to reach terminal, reads the ACTUAL fill, and places a GTC stop-limit for exactly that qty. This also fixes a latent bug — the OTO leg was sized to the REQUESTED qty, so a partial entry fill left a stop covering shares we never owned. - ExecutionStage protects every filled entry after the submission burst. - Belt: _reconcile_stop_coverage now REPAIRS a naked long instead of only flagging it, using the stop recorded on its last BUY (the level PM/RM approved — no longer 'unknown', which was the original objection to auto-repair). Guards: never place at/above the live price (that would fire instantly — an exit decision belongs to the reviewer), never invent a level. This retroactively protects anything the old bug left naked and covers a crash between an entry fill and the stop placement. Tests: entry carries no OTO leg + stays DAY; protection is GTC and sized to the actual fill; no-fill places nothing; stop-submit failure is swallowed; repair from the recorded BUY stop (full/partial/refuse-above-price/no-stop/ failure/covered). 1320 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
All confirmed by an independent skeptic pass; each has a regression test. RISK ENGINE - NaN market_value silently DISABLED max_total_position_pct + max_sector_pct for the whole session (NaN comparisons are all False, so `total_pct > cap` evaluated False). Now blocks with a synthetic hard violation, mirroring the total_value guard — a broken snapshot is the day the caps matter most. - correlation_cluster omitted the BUY symbol's OWN existing position: `highly_correlated_peers` (correctly) excludes the symbol itself, so an ADD to the biggest name in a cluster counted only the ADD. 40k NVDA + 10k AVGO + 5k add scored 15%, not 55%. CONSTRUCTOR - allocation_pct was a GROSS weight delta but every consumer (rules.py, ExecutionStage) spends it as RAW notional: a 6% SQQQ (3x) target deployed $6k = 18% gross, and the next session saw 18 vs target 6 and SOLD 67% of the hedge PM wanted held, repeating until raw ~2%. Convert once at the source; no-op at multiplier 1.0. (SELL is a ratio of gross weights — the multiplier cancels — so it was already right.) - target_weight_pct=0 means CLOSE, but the min_trade_weight_delta churn filter turned a small dreg's explicit close into a HOLD forever. - The stop was validated unrounded and shipped rounded, so a stop that rounds UP to exactly the entry passed the `stop < entry` check → risk_per_share=0 and a stop that fires on the first tick down. BROKER - get_session_close() ALWAYS returned None against the real SDK: Calendar.close is a naive datetime, not a time, so datetime.combine() raised TypeError every call → the early-close guard was dead code and midday/close ran against a shut market on half-days. The test that should have caught it built a MagicMock with a `time`; it now constructs the real Calendar model. - _get_sector() returned "Unknown" for EVERY ETF (yfinance .info has no sector for ETFs), which both skipped max_sector_pct entirely on an ETF BUY and made a held XLV contribute $0 to Healthcare for an LLY BUY. Deterministic _ETF_SECTORS table, consulted before the network fetch. - cancel_snapshotted_stops discarded _restore_stop_orders' failed_specs, so a rollback that itself failed left shrunk coverage reported as a bare False. PIPELINE - finalize persisted the POST-sell residual into the position_qty_before_sell column; the drain recomputes `pre - fill` from the same order, so the fill was subtracted twice — for an exact fill the residual hit 0, took the "full exit, nothing to protect" early return, reported success and DELETED the WAL row. The residual position stayed naked forever. - Ex-div stop adjustment could NEVER fire for a Monday ex-div: it compared against calendar `today + 1 day`, which for a Mon-Fri session is never a Monday (Friday computes Saturday). Now uses the next TRADING day. - _force_delever booked projected proceeds only AFTER insert_trade succeeded, so a DB hiccup on a live SELL made the loop force-sell the NEXT position to cover a deficit already covered — liquidating holdings over a bookkeeping failure. DATA - macro sector_guidance was never persisted by save_last_state, so every macro_sector_stance / macro_sector_tailwind was PERMANENTLY "unknown" — the evening thesis-health step and every missed-opportunity snapshot rendered "Macro sector stance: unknown" nightly while macro was emitting OW/UW calls. Three stacked breaks (not persisted; reader wants a dict, model carries a list; overweight/underweight vs bullish/bearish) — normalized on write. The round-trip test had PINNED the bug by asserting the key was dropped; its real intent (keep the snapshot tiny) is honoured by storing the map without the `reason` prose. 1336 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
…ted views Second pass over the audit's unverified backlog (the verify fleet hit the session limit mid-run, so I verified these by hand — each is reproduced). SILENT DATA LOSS - parse_json scored a top-level LIST as 0, so whenever the model wrapped its JSON in any prose the candidate scan compared the array against its own elements and returned the LAST ELEMENT. tech_analyst returns an ARRAY of per-symbol analyses (`items = parsed if isinstance(parsed, list) else [parsed]`), so a 25-symbol chunk could silently collapse to 1 analysis with 24 discarded and no error anywhere. Reproduced; a container is now scored by the SUM of its elements so it strictly outranks any element it contains. DEAD INDICATOR - ma_200 was unconditionally None in production: lookback_days is CALENDAR days (`start = today - timedelta(days=lookback_days)`), so 120 yielded ~82 bars and technical.py's `len(df) >= 200` never held. The tech_analyst prompt rendered "MA200=None" for every symbol every day — the analyst was asked to judge trend with the most-used long-term reference permanently absent. lookback_days 120 -> 320 (~220 bars), with a test that pins the config against every indicator window the code advertises. STALE / INVERTED VIEWS - ExecutionStage discarded the fresh price_map from the no-SELL refresh, so for an ADD to a held name the 5% entry-staleness guard and the order sizing both used research-time prices from 5-10 minutes earlier. - position_reviewer was handed RAW cash while the sweep vehicle was stripped from its positions — it saw an all-in book with a few hundred dollars spare while most of the equity sat parked and instantly available. Its de-lever mandate and weight reasoning key off that number. (Regression from the cash-sweep work: DecisionStage already credited it for the PM.) - The evening Telegram position snapshot counted parked SGOV as deployed capital, reporting a ~99%-deployed book on a night the money was entirely in T-bills, and listing SGOV among the P&L movers. Parked value is now reported separately as what it is. - position_reviewer's prompt instructs "think in ATRs" and the pipeline pays for an ATR fetch per position, but build_user_message never rendered atr_pct / stop_distance_atrs — the instruction referred to data the model could not see. (Also a regression from the exit-quality work.) 1336 tests green (+9 regressions here). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
A second verify pass (14 skeptics, one per remaining candidate) confirmed 6 and refuted 8 — the refuted set included several plausible-looking claims that did not survive contact with the code (news symbol matching already uses word boundaries; every _locked_write caller passes a single statement; the Anthropic cache accounting is right even though its comment is stale). MAJOR - compute_trade_calibration ignored FILLED TRAIL_STOP exits, so every stop-out left a phantom open lot and closed nothing: LLY BUY8 -> stop-filled 8 -> BUY6 -> stop-filled 6 read as 14 shares still held and zero LLY trades closed, while the position was flat. The 8 real stop-outs in the ledger move the numbers materially (a typical window: win_rate 22.2% -> 30.0%, avg_return -2.79% -> -2.18%) — and these feed PM as facts and the reviewer as calibration_note. The filled-guard mirrors _build_post_exit_reality: a placed but unfilled stop is protection, not an exit. - earnings record_failure() wrote the new filing_date into the manifest, and _check_symbol treats that as "already processed" — so after the FIRST transient LLM failure the filing was never re-queued, the 3-strike budget never reached `abandoned`, and PM was served the PRIOR quarter's analysis labelled "[from cache]" as if it were current. It bit every symbol with a same-form analysis already on disk (12 today; the whole universe in steady state). "Already processed" now means SUCCEEDED — confirm_filing() zeroes failed_attempts, so attempts 1-2 retry as designed and the 3rd abandons. MINOR - _build_trade_grade_summary counted each re-grade of the same SELL as an independent sell (the 2-day grading window has no already-graded filter, so evening re-grades a trade 2-3 nights running), inflating the premature/wrong counts that drive the reviewer's patience tilt. Deduped on (symbol, sell_date); a malformed row without a date is counted rather than collapsed. - _symbols_already_trimmed_today failed OPEN on a partially-filled-then- canceled sell: shares left the book at midday, but close saw a clean slate and could trim the same name again on the same soft flag — the 2026-05-04 AMZN double-trim this guard exists to stop. Now uses the codebase's existing _trade_executed_or_pending contract (a zero-fill rejection still allows a retry). - _clamp_queued_earnings_buys capped the constructor's DELTA, not the RESULTING weight, so a name held at 15% with an unread filing could be topped up to 20% because the add itself was <= 5% — while the prompt and the docstring both promise a cap on the resulting position. - credit_spread's "change_30d_bps" measured ~57-60 days: the reference was the head of a 60-calendar-day fetch rather than an observation ~30 days back (inherited from the earlier MONTHLY series fetcher, where it was harmless). On live FRED data today it reported -11.0 bps where the true 30-day change was +6.0 — a sign flip on a risk-off input to macro. 1341 tests green (+15 regressions here). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
Merged
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-codebase audit: 25 defects found and fixed (1 critical, 6 major, 18 minor)
A 20-unit audit workflow over the whole repo (src 24k lines / tests 31k / prompts 2.7k) — 15 subsystem finders + 5 cross-cutting lenses (money-arithmetic consistency, concurrency/crash-safety, time/timezone, tests-as-spec, prompt↔code drift, dead guards). 102 raw findings → adversarially verified one-by-one → 25 confirmed and fixed, 16 refuted.
🔴 CRITICAL — every BUY-attached stop expired at the close; positions were naked overnight
alpaca-py'sStopLossRequesthas notime_in_forcefield, so an OTO child leg inherits the parent's tif. The entry parent is (correctly)DAY— an unfilled entry limit must die at the close rather than fill into a stale thesis — which silently made every BUY-attached protective stop a DAY order too. Alpaca expired it at 16:00 ET the same session. Any position bought in the morning and not later handed a midday/closeTRAIL_STOP(the only path that used the GTC_submit_stop_limit_order) sat completely unprotected overnight — exactly when the gap risk the stop exists for happens.Production proof: VST bought 2026-06-26 09:47 ET with
SL=$158.75; the same evening's coverage reconcile loggedVST held=31.0000 but only 0.0000 covered; the gap persisted until an LLM SELL exited at $152.77 on 07-01 — ~$185 worse than the stop would have capped. Control: GTC trail-stops on GE survived day boundaries in the same account. Live check: all 3 open stops today are GTC stop-limits from the trail path; not one BUY-attached stop exists.It also contradicted the close-session prompt, which tells the reviewer to hold overnight because a broker stop is standing watch.
Fix (the leg's tif can't be set independently, so decouple): the entry stays a plain DAY limit;
broker.place_entry_protection()waits for terminal, reads the actual fill, and places a GTC stop-limit for exactly that qty (this also fixes a latent bug — the OTO leg was sized to the requested qty, so a partial fill over-covered). Belt:_reconcile_stop_coveragenow repairs a naked long from the stop recorded on its last BUY (the level PM/RM approved — no longer "unknown", which was the original objection to auto-repair), refusing to place at/above the live price. This retroactively protects anything the old bug left naked.MAJOR
finalizepersisted the post-sell residual into theposition_qty_before_sellcolumnpre − fillfrom the same order → fill subtracted twice → for an exact fill the residual hit 0, took the "full exit, nothing to protect" path, reported success and deleted the WAL row. The residual stayed naked forever.get_session_close()always returned NoneCalendar.closeis a naive datetime, not a time, sodatetime.combine()raised TypeError on every call → the early-close guard was dead code and midday/close ran against a shut market on half-days. The test that should have caught it built a MagicMock with atime._get_sector()returned"Unknown"for every ETF.infohas nosectorfor ETFs →max_sector_pctwas skipped entirely on an ETF BUY, and a held XLV contributed $0 to Healthcare for an LLY BUY. Both directions of the cap dead, on a ~20%-ETF universe.allocation_pctmarket_valuesilently disabledmax_total_position_pct+max_sector_pctsector_guidancewas never persistedmacro_sector_stance/macro_sector_tailwindwas permanently"unknown"— the evening thesis-health step and every missed-opportunity snapshot rendered "Macro sector stance: unknown" nightly while macro was emitting OW/UW calls. Three stacked breaks (not persisted; reader wants a dict, model carries a list; overweight vs bullish).parse_jsonscored a top-level list as 0ma_200was unconditionally None in productionlookback_daysis calendar days → 120 yielded ~82 bars andlen(df) >= 200never held. The tech_analyst prompt renderedMA200=Nonefor every symbol every day.compute_trade_calibrationignored filled TRAIL_STOP exitscalibration_note.record_failure()marked the filing seenabandoned, and PM was served the prior quarter's analysis labelled "[from cache]" as if current. Hit every symbol with a same-form analysis on disk — the whole universe in steady state.MINOR (18)
Ex-div stop adjustment could never fire for a Monday ex-div (calendar
+1 dayis never a Monday for a Mon–Fri session; Friday computed Saturday) ·_force_deleverbooked proceeds only afterinsert_trade, so a DB hiccup on a live SELL force-sold the next position to cover a deficit already covered ·cancel_snapshotted_stopsdiscarded the rollback'sfailed_specs· correlation cluster omitted the BUY symbol's own existing position (40k NVDA + 10k AVGO + 5k add scored 15%, not 55%) ·target_weight_pct=0(CLOSE) swallowed by the churn filter into a permanent HOLD · stop validated unrounded but shipped rounded (could round up to exactly the entry) · ExecutionStage discarded the freshprice_map(the 5% entry-staleness guard compared against research-time prices) · position_reviewer given raw cash while the sweep vehicle was stripped from its positions · the evening Telegram snapshot counted parked SGOV as deployed capital · reviewer prompt says "think in ATRs" butbuild_user_messagenever renderedatr_pct/stop_distance_atrs· the same SELL re-graded 2–3 nights running counted as independent sells · the same-day trim guard failed open on a partially-filled-then-canceled sell · queued-earnings cap bounded the delta, not the resulting weight ·change_30d_bpsmeasured ~57–60 days (sign-flipped on live data today) · plus the tests that pinned two of these bugs.Three of these were regressions from today's own PR #101 (reviewer cash, ATR rendering, and the emergency×park interaction), caught by the same audit.
Notable refutations (16)
Verification earned its keep: news symbol matching already uses word boundaries; every
_locked_writecaller passes a single statement; the Anthropic cache accounting is correct despite a stale comment; the FIFO lot matcher handles partial exits correctly; the stuck-WAL scenario is mechanically unreachable; the retry-deadline ordering is fine given the 300s HTTP timeout.Test plan
StopLossRequestfield list) + reproduction ofparse_json,get_session_close, and the ma_200 gap🤖 Generated with Claude Code
https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR