feat: returns optimization — cash sweep, exit guards, resume lane, feedback repair (forensics-driven) - #101
Merged
Merged
Conversation
…mand 2026-07-16 forensics: the account sat at ~84% idle cash for weeks (~$8.9k cash drag over the quarter, ~$300/mo of forgone risk-free carry). This adds a deterministic, zero-LLM sweep with a strict cash-equivalence contract: - CashSweeper (src/execution/cash_sweep.py): park_excess() buys the vehicle with cash above a reserve, minus open-BUY-order holds (Alpaca's cash field doesn't subtract them — sweeping that cash would starve pending fills; unknowable holds → park nothing). fund_buys() releases exactly enough parked cash before the BUY phase, via the same _submit_protected_sell + _finalize_pending_protections discipline as every other SELL path. - The vehicle is hidden from every LLM view (PM decision stage, position reviewer, evening builders) and its market value is credited as CASH in _filter_hard_risk_decisions — parked cash can never block a real BUY via the net-exposure/cash rules. - force_delever sells it FIRST (tier -1); _reconcile_stop_coverage exempts it (deliberately stopless); SWEEP_BUY/SWEEP_SELL action names keep it out of every grading/calibration consumer. - config: cash_sweep section (enabled/symbol/reserve_pct/min_order_usd), default-disabled for backwards compat; production settings.yaml enables SGOV with 1% reserve. - broker.open_buy_notional(): None-vs-0.0 distinction so a failed order query reads as 'unknowable', never as 'no pending buys'. Also pins QUANT_AGENT_MAX_RETRIES in the retry-deadline test (was sensitive to ambient env overrides). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
2026-07-16 sell autopsy: 28/53 realized exits since 5/1 were EARLY (stock >=5% higher within 20 days), $8.5k-16k left on the table; 5 trail-stop fills missed avg +30.7% post-exit; LLY was trail-whipsawed twice identically 4 weeks apart. The mechanical share of the leak gets code guards (prompts alone demonstrably didn't hold): - TRAIL_STOP noise-band clamp: a new stop inside 1.25×ATR14 of current price sits inside one day's normal range — rejected, old stop kept. A cited hard trigger bypasses (same philosophy as the same-day-trim gate). Unknowable ATR degrades open (no clamp). - TRAIL_STOP ratchet cooldown: at most one accepted tighten per ~2 trading days per symbol (the reviewer's >=1.02×old_stop min-bump rule made every accepted trail tighten >=2%; GE was ratcheted 325->350 in 8 sessions on one flag). Hard triggers bypass. - Live stop truth in position facts: prefer broker.get_current_stop_price over the stale-wide BUY-row stop (after any trail the reviewer saw a fat distance_to_stop and kept ratcheting), plus atr_pct / stop_distance_atrs so the reviewer reasons in vol units. - Entry ATR floor (ExecutionStage): a BUY stop closer than 1×ATR14 (computed from bars already fetched by research) is widened to 1×ATR; qty_by_risk sizes against the wider distance so per-trade $ risk is unchanged. No bars -> no floor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
… exit audit RC4 (2026-07-16 forensics): the learning loop couldn't emit 'we were wrong to sell' or 'we were wrong to skip': - value_entry_missed was code-filtered OUT of the recurring-miss digest — the exact category evening uses for actionable entries (SNDK flagged 16×, ORCL 7×; PM was shown '(no recurring missed themes)' every run since 7/01 while evening wrote 'zero execution — this is process failure'). - Misses were grouped by theme_if_any, LLM free text that never repeats verbatim (45 distinct themes, 0 recurring) — now grouped by SYMBOL, theme kept as annotation. - Sell grades are scored by the LLM at t+1..t+3 with thesis-rationalization framing (32/33 'correct' while the tape had 53% of exits ≥5% higher within 20d; grader error ~50% on checkable sells). A deterministic post-exit reality block (_build_post_exit_reality: trades × live prices, no LLM) now rides with the grade summary into the reviewer prompt, with a hard escalation line when over half of recent exits kept running. SWEEP_SELL and <2-day-old exits excluded. - Reviewer prompt: to_stop is advisory distance, never a trigger (GS 5/18 pre-empted its own stop); single-source bearish news on a >10% winner caps first-day action at REDUCE≤50% (AAPL 6/25); think in ATR units; documents the trail cooldown + noise-band enforcement from the exits commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
…nswered RC3 (2026-07-16 forensics): macro demanded 72-75% invested for three months (risk-on the whole window); realized invested% averaged 39.2% and declined monotonically 72.7%→15.4% over 13 weeks. Every layer shaved sizes independently (PM on calibration, RM on R/R, evening tilt) and nothing reconciled the compounded result against the target — ~$8.9k of cash drag, 81% of the SPY shortfall. - PMFacts gains macro_target_invested_pct + deployment_gap_pp; a >15pp under-target book renders a⚠️ DEPLOYMENT GAP section the PM prompt now requires be answered in the cash_target step: close the gap this session OR name a checkable blocker per unfilled slot. 'Staying cautious' without a named blocker is explicitly ruled out. Does not override RM, caps, or drawdown-halving — it forces the gap to be addressed, not auto-filled. - macro_exposure_deviation advisory is now direction-aware: for an UNDER- deployed book it tells RM NOT to scale the remaining BUYs down for this reason (the old symmetric 'consider scale_all_buys' amplified the drag it should have been correcting). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
… lane RC2 (2026-07-16 forensics): when LLM latency inflates, morning dies at the wrapper's outer timeout exactly at the PM→RM boundary — research (3 tech chunks + news + macro) plus PM eat the budget and the kill lands as RM starts. 61/61 PM BUY-proposal days between 6/30 and 7/15 were destroyed this way (zero RM vetoes, 100% mechanical attrition), and every 30-min retry tick re-burned the FULL research pipeline to die identically (run-57b752bd: 'Constructor: 5 targets → 4 BUY' logged at 1196s of a 1200s budget). The relay outage that triggered it is fixed, but the structural fragility — the only BUY-capable session being the first casualty of any latency inflation — is not. - src/decision_checkpoint.py: the plan is persisted (atomic tmp+rename) the moment DecisionStage produces decisions; ET-date-keyed, versioned, 90-min max age. - run_morning resume lane: after the FULL normal preamble (WAL drains, orphan sweep, coverage audit, stale-order cancel, force_delever, circuit breaker, fresh account snapshot), an unconsumed same-day checkpoint skips research+PM and re-enters at RiskStage — ~2 LLM calls instead of ~8. RM ALWAYS re-runs (no resume-past-RM variant, by design); the hard-risk filter re-runs on fresh cash/positions; ExecutionStage's 5% entry-staleness skip and pre-BUY daily-loss recheck guard stale prices. - At-most-once: the checkpoint is marked consumed BEFORE ExecutionStage submits and on ANY RiskStage early-exit — an RM-rejected plan is never re-offered (that would be a veto bypass), and a kill mid-execution is owned by the existing BUY write-ahead orphan sweep, never by re-running the plan. - Every checkpoint operation is best-effort: failures degrade to a normal full run, never a crashed session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
…check
RC5 (2026-07-16 forensics): a 1-day incident became 13+ days because both
observability layers were blind to kills:
(a) timeout's SIGTERM/SIGKILL skips python's finally-block notifier — 13
straight mornings died with ZERO Telegram pushes;
(b) evening's _expected_sessions_missing_today only checked that ANY run-
row existed — partial runs (research logged, then killed) satisfied it,
so the 🔴 banner never fired either.
- Wrapper: on violent deaths only (124/137/143 — the codes where python's
own notifier cannot run) push a Telegram message directly from bash;
ordinary non-zero exits stay python-owned to avoid duplicate pushes.
TELEGRAM_DISABLED kill switch honored.
- Wrapper: optional HEALTHCHECKS_URL ping on success, /fail on failure —
the external dead-man's switch CLAUDE.md has wishlisted since May; covers
total host death and evening-not-firing, which no in-process check can.
- Dead-man check: two sharper probes when 'run-' rows exist — (a) research
ran but portfolio_manager never logged (killed mid-research), (b) the
decision checkpoint exists unconsumed (killed at the PM→RM boundary, the
observed death mode). New db.agent_names_logged_on() supports the probe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
…park bookend Two gaps caught in self-review before the adversarial pass: - run_position_review computed the SGOV-free view but still passed raw positions into position facts / the reviewer / the action dispatcher — the reviewer would have hold-graded parked cash. - park_excess was hooked into morning only; midday/close SELL proceeds would have sat unswept overnight. Both sessions now park at the bookend (emergency paths return earlier and deliberately skip parking). Adds an end-to-end run_midday test pinning both behaviors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
…pass A 34-agent review workflow (5 lenses × independent skeptic per finding) confirmed 22 defects (7 claims refuted). Grouped fixes: - RM sweep view (major): RiskStage passed raw positions to the RiskManager — the VETO layer saw parked SGOV as an 84%-of-book position while PM saw the same dollars as cash, desynchronizing the two risk layers with veto power on the corrupted view. RM/correlation/has_book now use the scrubbed list; the hard filter keeps RAW positions (it derives the parked-cash credit from finding the vehicle itself). Regression test added. - Emergency×park oscillation (major): the post-review breach branch in run_position_review fell through to the park bookend — the system would buy SGOV with ~all equity minutes after force-selling everything, then the next intra tick would emergency-sell the fresh SGOV lot. Now returns emergency_sold immediately, mirroring the pre-review breaker. - Ex-div false cooldown (major): ex-div adjustments write TRAIL_STOP rows (stop LOWERING) — they no longer count as a 'tighten' for the ratchet cooldown, so dividend names don't get spurious 2-day trail freezes. - Resume-lane bars (major): the checkpoint deliberately omits symbols_bars; resume now rehydrates OHLCV for the plan's BUY symbols deterministically, restoring the entry ATR floor and the correlation advisory on resume. - Checkpoint lifecycle: mark_consumed is fail-CLOSED (falls back to deleting the file — unlink survives ENOSPC — and returns bool); both emergency-liquidation exits consume the checkpoint and record a status marker; legit no_data mornings record status too. - Dead-man accuracy: probes skip legitimately-PM-less mornings via the status marker; the notifier matches decorated 'morning (...)' entries so the sharpened diagnoses actually reach the 🔴 banner. - Sweep hygiene: evening today_trades excludes SWEEP_* churn (limit 30→20 after filter); post-exit reality skips the sweep symbol (emergency exits of SGOV are not decision-quality data); park_excess sizes qty against the LIMIT price so a padded fill can't overdraw a thin reserve. - Entry ATR floor: after widening, recompute R/R vs the unchanged target and skip the BUY below 1.2 — RM approved the tight-stop geometry, not a collapsed one. - PMFacts renders the OVER-deployment case instead of calling +40pp 'within band'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Returns optimization: forensics-driven fixes for the July underperformance
Why (the forensics, 2026-07-16)
A 7-agent diagnostic workflow root-caused the gap (NAV −0.5% vs SPY +2.5% over the 6/5–7/14 validation window; −3.5% vs SPY since 5/1; 84% idle cash, no BUY since 6/26):
to_stoptreated as a trigger, same-day full exits on single news stories (AAPL +18% sold on one component-cost story)value_entry_missedcode-filtered out of PM's view (SNDK flagged 16×, PM never saw it); misses grouped by never-repeating free-text themesrun-row as "morning ran"A separate 25-agent optimization hunt proposed 20 improvements; 19 survived adversarial judging. The top-scored, invariant-safe subset is implemented here; the rest are documented below as backlog.
What's in (8 commits)
feat(cash)SGOV idle-cash sweep — deterministic, zero-LLM. Excess cash above a 1% reserve parks into SGOV at every session bookend; released automatically before BUYs. Strict cash-equivalence contract: hidden from every LLM view, counted as cash by the risk filter, excluded from exposure math, stopless by design, first to liquidate in force_delever,SWEEP_*action names invisible to all grading/calibration consumers. Open-BUY-order holds subtracted before parking (unknowable → park nothing). ~$300/mo risk-free carry at current cash levels.feat(exits)deterministic exit-quality guards — TRAIL_STOP noise-band clamp (reject stops inside 1.25×ATR14 of price), ratchet cooldown (one accepted tighten per ~2 trading days), live broker stop in position facts (the stale-wide BUY-row stop kept feeding the ratchet), entry stop floor at 1×ATR with automatic size compensation. Hard-trigger citations bypass the clamps.feat(feedback)grading-loop repair —value_entry_missednow a real miss; misses grouped by symbol AND theme (dual-key); a deterministic post-exit reality block (trades × live prices, no LLM) rides into the reviewer prompt with an escalation line when >50% of recent exits kept running; reviewer prompt:to_stopis advisory, single-source news on a >10% winner caps first-day action at REDUCE≤50%, think in ATR units.feat(pm)deployment-gap convergence — PMFacts renders acash_target: close it or name a checkable blocker. Themacro_exposure_deviationadvisory is direction-aware (no more "consider scale_all_buys" on an UNDER-deployed book).feat(pipeline)decision checkpoint + resume lane — the PM's plan is persisted the moment it exists; a killed morning resumes next tick at the RiskStage (full preamble re-runs, RM always re-runs, consumed-before-execution gives at-most-once). Converts the RC2 death mode into delayed-by-one-tick execution and stops retry ticks from re-burning research.feat(obs)violent-death reporting — wrapper pushes Telegram on kill statuses (124/137/143) where python's notifier cannot run; optionalHEALTHCHECKS_URLexternal dead-man ping; the evening dead-man check now detects partial mornings (research-only rows, unconsumed checkpoint).fix(cash)— self-review fixes: reviewer view truly excludes the vehicle; midday/close park bookend.fix(review)— 22 confirmed findings from the adversarial review applied: RM sweep view (the veto layer saw SGOV as an 84% position while PM saw cash), emergency×park oscillation, ex-div false trail cooldown, resume-lane bar rehydration, fail-closed checkpoint consumption, dead-man probe accuracy + notifier banner matching, sweep hygiene in evening/learning views, post-widening R/R recheck.Hard invariants — untouched
cash_only / force_delever semantics, SELL protection discipline (SWEEP_SELL uses
_submit_protected_sell+ finalize like every other path), RM veto power (resume re-runs RM, rejected plans consumed), 3% breaker, caps, ET discipline,allocation_pctsemantics. The reviewer/PM prompt edits are operator edits (the auto-evolution protection is unaffected).Test plan
fix(review)commit), 7 refutedHEALTHCHECKS_URLin.envto activate the external dead-man switchDeferred backlog (judged KEEP, not implemented here)
Outcome-aware replay scoring (score replayed decisions against realized forward returns); SPY-trend + breadth deterministic regime block; conviction-tiered risk budget; re-entry radar for stopped-out names + deterministic re-entry tickets; memory-layer ablation in the replay harness; deployment-rate calibration persistence; sector relative-strength table; cluster-capacity pre-clamp; broker-side stop-out fills into evening grading; house-money pyramid adds; pre-market gap scan.
🤖 Generated with Claude Code
https://claude.ai/code/session_01PQkESoSTYx2bCy7WYnTPXR