Skip to content

feat(replay): add capture plan orchestration - #363

Open
PhilipJohnBasile wants to merge 2 commits into
youssofal:mainfrom
PhilipJohnBasile:feat/replay-orchestrator
Open

feat(replay): add capture plan orchestration#363
PhilipJohnBasile wants to merge 2 commits into
youssofal:mainfrom
PhilipJohnBasile:feat/replay-orchestrator

Conversation

@PhilipJohnBasile

Copy link
Copy Markdown
Contributor

Split from #336 as one independently reviewable system.

This PR adds engine-neutral replay orchestration only:

  • bounded capture discovery and deterministic seeded selection
  • metadata filters, public-fingerprint deduplication, and replayability classification
  • stale-plan validation against the complete current selection
  • atomic, private-by-default replay receipt persistence
  • no candidate execution, evaluators, request capture, promotion, or server wiring

The deterministic replay engine remains in #351. Privacy-safe request capture remains in #356. This PR imports neither implementation. A caller hands the orchestrator report and decision mappings after its replay engine finishes.

Measurement on this commit:

python scripts/bench_replay_orchestrator.py
1,000 captures, 128 selected cases
plan selection, 30 iterations: p50 50.151 ms, p95 51.384 ms, p99 51.676 ms
freshness validation, 100 iterations: p50 51.003 ms, p95 55.209 ms, p99 58.789 ms
atomic receipt write plus freshness check, 50 iterations: p50 54.083 ms, p95 58.031 ms, p99 59.897 ms
source digests across plans: 1
stale mutation rejected: yes, 57.013 ms
receipt files: 50, temporary files remaining: 0

Validation:

python -m pytest tests/test_replay_orchestrator.py -q
9 passed

ruff check mtplx/replay_orchestrator.py tests/test_replay_orchestrator.py scripts/bench_replay_orchestrator.py
All checks passed!

python -m compileall -q mtplx/replay_orchestrator.py tests/test_replay_orchestrator.py scripts/bench_replay_orchestrator.py
git diff --check

The existing macOS CI job now includes tests/test_replay_orchestrator.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant