Architecture analysis and cost/performance optimization context

[rica-v3]

Purpose

This report documents the current project objective, architecture, runtime workflow, and component responsibilities. It provides the baseline context for cost and performance optimization.

Analysis date: 2026-07-17
Validation baseline: main branch

Executive summary

teams-runtime is a file-backed, Discord-operated orchestration system that coordinates multiple AI roles through software planning, implementation, review, QA, version control, and sprint closeout.

The dominant cost is model-call amplification across sequential roles, compounded by expanding request histories and persistent model sessions. The main local performance costs are repeated directory scans, full Markdown regeneration, polling, Git subprocesses, and ten long-running role services.

The architecture prioritizes durability, explicit handoffs, traceability, and recoverability over throughput. Optimization should focus first on observability, model-call reduction, context compaction, and incremental persistence.

Project objective

The project turns Discord into a durable multi-agent software workspace:

Discord message
  -> intake and classification
  -> research and planning
  -> architecture guidance
  -> implementation
  -> architecture review
  -> QA validation
  -> Git commit
  -> sprint report and closeout

Primary objectives:

• Preserve request, backlog, sprint, and role state across process restarts.
• Route work using deterministic workflow rules rather than allowing models to select arbitrary agents.
• Maintain role-specific workspaces, prompts, skills, journals, and sessions.
• Convert Discord messages into traceable software tasks and artifacts.
• Support autonomous scheduled sprints and manually supervised daily sprints.
• Require evidence-based review and QA before work is declared complete.
• Produce auditable Markdown and JSON records for every stage.

See the README and architecture overview.

Project scale

Area	Approximate size
Production Python	37,600 lines
Test Python	34,900 lines
Test modules	45
Unit tests	754
Main TeamService coordinator	7,532 lines

The test-to-production ratio is strong, but the large coordinator remains a maintainability and profiling concern.

Structure

Layer	Main paths	Responsibility
Entry points	main.py, cli.py	Package execution and backward-compatible CLI facade
CLI adapters	adapters/cli/commands.py	init, run, start, stop, restart, status, configuration, and sprint commands
Discord adapters	adapters/discord	Discord gateway, message ingestion, chunking, retrying, transcripts, and process lifecycle
Shared contracts	shared/models.py, shared/config.py	Roles, envelopes, requests, backlog, sprint configuration, and serialization
Shared persistence	shared/paths.py, shared/persistence.py	Filesystem layout and JSON, JSONL, and Markdown persistence
Model runtime	runtime/base_runtime.py, runtime/codex_runner.py	Prompt construction, model subprocess execution, result repair, and session reuse
Specialized runtimes	runtime/internal, runtime/research_runtime.py	Research, semantic parsing, and backlog sourcing
Orchestration	workflows/orchestration	Composition, intake, routing, delegation, relay, reporting, and scheduling
Sprint workflows	workflows/sprints/lifecycle.py	Sprint creation, planning, execution, recovery, validation, and closeout
State stores	workflows/state	Request, backlog, and sprint persistence
Role policy	workflows/roles	Capabilities, routing signals, prompt rules, and workflow policy
Scaffolding	core/template.py, templates/scaffold	Generated role workspaces, prompts, skills, and shared sprint documents
Compatibility	core, discord, models.py	Legacy imports and re-exports
Tests	tests	Intake, routing, contracts, sprints, persistence, reporting, and recovery

Runtime topology

Seven public roles:

• orchestrator
• research
• planner
• designer
• architect
• developer
• qa

Three internal roles:

• parser
• sourcer
• version_controller

A full deployment starts one service process per role, producing ten resident processes. Each service owns role-specific workspace and session state, logging, Discord integration, and an internal relay consumer.

The orchestrator additionally owns request creation, deduplication, workflow transitions, requirement capture, the sprint scheduler, the backlog sourcing loop, and terminal reporting.

Each public TeamService also constructs local parser, sourcer, and version-controller runtime objects. Most helper behavior is only exercised by the orchestrator, so this adds construction and architectural overhead alongside the separate internal services.

Core data model

Entity	Responsibility
MessageEnvelope	Transport contract between roles, including request identity, scope, intent, artifacts, and target
RequestRecord	Lifecycle of a user or internal task, including workflow state, events, result, and ownership
BacklogItem	Candidate work, priority, evidence, status, and sprint association
Sprint state	Milestone, plan, requirements, selected backlog, todos, reports, commits, and closeout status
Role session	Model session identity, sprint identity, workspace location, and resume metadata
Runtime configuration	Discord credentials, model settings, scheduler configuration, and transport mode

Structured operational state is stored beneath .teams_runtime in request, backlog, sprint, event, operation, session, relay, agent, and archive directories.

Human and agent-facing projections live under shared_workspace and role-specific directories. JSON is the main operational source of truth. Markdown serves as a materialized view and collaboration surface, although some role-authored artifacts are synchronized back into sprint state.

Request workflow

1. A Discord message reaches a role listener.
2. Messages sent to non-orchestrator roles are forwarded to the orchestrator.
3. The orchestrator handles plan feedback, active-sprint requirement capture, deduplication, reopening, and request creation.
4. The orchestrator model runs first. Requests without an explicit governed workflow default to research-first planning.
5. Work is transported through a filesystem relay by default. Discord receives compact monitoring summaries.
6. The target role loads its model session, builds a structured prompt, invokes Codex or Gemini through a subprocess, validates the result contract, and returns a report envelope.
7. The orchestrator persists the result, updates projections, invokes the deterministic transition engine, and delegates the next role.
8. The request terminates as completed, blocked, or failed and a user-facing report is sent.

The deterministic transition engine is an important cost control. Models propose results and transitions, but code selects the next role.

Typical implementation sequence:

research_initial
  -> planner_draft
  -> optional designer or architect advisory
  -> planner_finalize
  -> architect_guidance
  -> developer_build
  -> architect_review
  -> developer_revision if necessary
  -> QA
  -> complete or reopen

The review cycle limit defaults to 20, allowing significant call amplification on difficult work.

Sprint workflow

Autonomous mode

The scheduler polls every 15 seconds. It selects actionable backlog items, creates a sprint, captures a Git baseline, creates planner-owned todos, and processes todos sequentially through the internal request workflow.

Selection considers all actionable pending items rather than enforcing a clear batch-size ceiling. A large backlog can therefore create an unexpectedly large sprint.

Manual daily mode

Manual startup runs:

research prepass
  -> milestone refinement
  -> artifact synchronization
  -> human plan confirmation
  -> backlog definition
  -> prioritization
  -> todo finalization

This requires five planner calls, one research decision, and normally one parser call before implementation begins. Manual execution also performs an additional planner review after each completed todo.

Todo execution and closeout

Each todo becomes an internal RequestRecord and passes through the normal planner, architecture, development, review, and QA workflow.

After successful QA:

• Git state is inspected.
• The version-controller model is invoked when changes exist.
• A commit is created.
• Backlog and todo state are updated.
• Sprint artifacts and Discord reports are refreshed.

Closeout validates requirement traceability, may create recovery work, verifies Git state, and invokes the planner once more to draft the terminal sprint report.

Model cost profile

Approximate lower bounds for healthy, no-retry paths:

Work path	Minimum model calls
Planning-only user request	3: orchestrator, research decision, planner
User-originated implementation	7: orchestrator, research decision, planner, architect guidance, developer, architect review, QA
Autonomous sprint todo	5: planner, architect guidance, developer, architect review, QA
Sprint todo with code commit	6: normal todo plus version controller
Manual sprint preparation	6 before confirmation; normally 7 including parser
Sprint closeout	1 additional planner call

Costs increase through external deep research, advisory passes, revision cycles, planner review after each manual todo, reopened requirements, closeout recovery, and contract repair calls.

A standard role result may be retried twice for invalid structured output. One logical role step can therefore consume up to three model executions.

Default settings use gpt-5.5 for all public roles:

Role	Default reasoning
Orchestrator	medium
Research	xhigh
Planner	xhigh
Designer	medium
Architect	xhigh
Developer	high
QA	medium

Parser, sourcer, and version-controller behavior does not have a clean independent low-cost model tier.

Performance pressure points

1. Full request records and envelopes are embedded in role prompts. Request events retain full earlier role results, so later calls repeatedly resend prior summaries, proposals, evidence, and transitions.
2. Long-lived sessions reduce cold starts but accumulate context. There is no explicit token-budget-based rollover or compaction policy.
3. Role handoffs and sprint todos are intentionally sequential. Throughput is bounded by the slowest model invocation.
4. Model subprocesses do not have a general execution timeout. A hung CLI can hold a workflow indefinitely.
5. Internal relay inboxes are scanned every 0.2 seconds. Ten services can approach 50 empty directory scans per second.
6. Every request save refreshes all seven role todo files and scans the request directory.
7. Saving one backlog item normally rescans the backlog and rewrites active and completed Markdown projections.
8. Sprint synchronization rewrites approximately eight Markdown artifacts. Both load and save paths can perform reconciliation and regeneration.
9. Markdown journal and history appends read and rewrite the entire file, producing increasing write amplification.
10. Git baseline, status, diff, commit, and closeout checks create subprocess amplification as repositories grow.
11. Detailed role and sprint reports can become multiple Discord sends with retries.
12. External research has long completion and callback timeouts and can dominate tail latency.

Correctness risks affecting cost

Persistence uses direct JSON and Markdown writes without an explicit atomic rename or shared cross-process transaction lock.

Potential consequences:

• Lost updates.
• Partially read files.
• Duplicate execution after reconnect or recovery.
• Temporary divergence between JSON state and Markdown artifacts.
• Repeated work caused by polling races.

Request fingerprints, persisted terminal checks, stale relay handling, and per-process active-request locks reduce risk, but do not provide cross-process transactional guarantees.

Existing cost controls

• Deterministic routing avoids an LLM router call at every transition.
• Request fingerprinting suppresses duplicate user work.
• Internal filesystem relay avoids using Discord for full payload transfer.
• Model sessions are reused within a role and sprint.
• Backlog sourcing skips model execution when no findings exist.
• Sourcer output has heuristic fallbacks and duplicate filtering.
• Status commands read persisted state instead of invoking a model.
• Models and reasoning levels are configurable per public role.
• Structured contracts prevent ambiguous output from silently advancing work.
• Sprint and requirement recovery preserve partial progress.

Observability gaps

The project cannot yet produce a reliable monetary cost baseline. General role execution does not consistently record:

• Input and output tokens.
• Cached versus uncached tokens.
• Per-call model price.
• Model subprocess duration.
• Queue and relay waiting time.
• Contract repair count.
• Session age and accumulated context size.
• File scan and rewrite counts.
• Discord send and retry counts.
• Git operation duration.
• End-to-end request and sprint critical-path latency.

The internal sourcer records elapsed time and prompt/output character counts, but this instrumentation is not shared across the main runtime.

Recommended optimization order

1. Add a common model-execution ledger keyed by request, sprint, todo, role, stage, session, model, reasoning, duration, repair attempt, and token usage.
2. Reduce call amplification with cheaper helper roles, batched planner reviews, and stricter review and reopen budgets.
3. Compact prompts to current state, relevant evidence, artifact references, and a bounded event window. Add session rollover thresholds.
4. Make request, backlog, role-todo, and sprint artifact updates incremental.
5. Add atomic writes and cross-process coordination, potentially through SQLite or a dedicated state service.
6. Bound autonomous sprint size by item count, estimated complexity, or token budget.
7. Replace aggressive polling with notifications, queues, or adaptive backoff.
8. Introduce concurrency only where work is independent. Developer parallelism requires isolated Git worktrees or equivalent conflict control.
9. Reassess whether all internal roles need independent Discord-connected processes and whether public services need duplicate local helper runtimes.

Validation

• python -m compileall -q teams_runtime: passed.
• python -m unittest discover -s teams_runtime/tests: 754 tests passed in 22.253 seconds.
• The analysis did not modify the worktree.

Proposed next step

Instrument the existing runtime before altering routing or quality gates. A measured per-request and per-sprint baseline will allow call reduction, model tiering, prompt compaction, and persistence changes to be evaluated safely.

[rica-v3]

Optimization status

• Observability foundation completed and merged in PR #3: privacy-safe model invocation telemetry, token/latency/retry tracking, optional cost estimates, metrics CLI, and operational documentation.
• First call-amplification optimization implemented in draft PR #4: manual sprint planner reviews are no longer forced after successful TODOs without pending requirement candidates; multiple candidates are batched into one checkpoint review.
• Current validation: 760 tests passed, focused sprint suites passed, compile and diff checks passed.
• Next planned task after PR Batch manual-sprint planner checkpoint reviews #4 merges: compact role prompt context while preserving full persisted audit history and workflow evidence.

[rica-v3]

Optimization status update

• Observability foundation remains merged in PR #3.
• Planner checkpoint-review batching is now merged in PR #4.
• Prompt event-history compaction is implemented in draft PR #5: prompts keep the recent tail, backfill the newest missing-role evidence, preserve the complete persisted request history, and support rollback with prompt_context.enabled: false.
• Documentation now defines compaction and backfill with an exact input/output example and telemetry comparison guidance.
• Validation: 769 full-suite tests and 9 focused prompt-context tests passed; package compilation and diff checks passed.

[rica-v3]

Full-sprint A/B benchmark implementation and first measurement

Report date: 2026-07-28 KST
Benchmark branch: enhancement/sprint-performance-benchmark
Baseline: merged prompt-compaction PR #5

Progress recap

• Observability and privacy-safe model telemetry are merged in PR #3.
• Planner checkpoint-review batching is merged in PR #4.
• Prompt event-history compaction and role-evidence backfill are merged in PR #5.
• A repeatable full-sprint Before/After benchmark pipeline is now implemented locally on enhancement/sprint-performance-benchmark; it has not yet been pushed or opened as a PR.

Benchmark pipeline delivered

• Adds python -m teams_runtime benchmark sprint-ab for isolated, repeatable full-sprint comparisons.
• Runs the same deterministic scenario with compaction disabled (Before) and enabled (After).
• Seeds 50 historical events into the canonical first planning request. With the PR Compact request event history in role prompts #5 policy, the compacted arm can retain 16 events and report 34 omitted events.
• Captures attempted/completed/timed-out/terminated provider calls, wall and provider duration, prompt size, native token usage, cache usage, tool calls, pricing coverage, and workflow/quality outcomes.
• Uses a durable invocation journal and identity reconciliation. Missing, duplicated, unmatched, or replaced invocation identities suppress token coverage and cost instead of producing misleading totals.
• Registers the provider process before release, bounds call/run time, terminates complete process groups, and verifies cleanup after timeout.
• Emits machine-readable JSON plus a Markdown execution report. Detailed methodology and Backfill/Compaction examples are documented outside the README in docs/performance_benchmarking.md.

First live Before/After run

Metric	Before	After	Delta
Attempted provider calls	8	8	0
Completed / timeout / terminated	5 / 2 / 1	5 / 2 / 1	unchanged
Telemetry records	7	7	0
Native token usage coverage	5/8 (62.5%)	5/8 (62.5%)	unchanged
Wall duration	1,800.030 s	1,800.017 s	-0.013 s
Provider duration	1,659.709 s	1,645.025 s	-14.684 s (-0.88%)
Prompt characters	394,728	393,621	-1,107 (-0.28%)
Input tokens	10,193,927	21,734,202	+11,540,275 (+113.21%)
Cached input tokens	9,597,824	20,191,744	+10,593,920
Uncached input tokens	596,103	1,542,458	+946,355
Output tokens	129,092	152,295	+23,203
Reasoning tokens	42,971	42,473	-498
Total tokens	10,323,019	21,886,497	+11,563,478 (+112.02%)
Tool calls	132	137	+5
Estimated monetary cost	N/A	N/A	no rate card configured

Interpretation and limitations

This first pair is inconclusive and must not be used as evidence that compaction improved or regressed cost:

• Both arms reached the 30-minute run timeout during planning.
• Neither arm completed a TODO, implementation, commit, QA, or sprint closeout.
• The original fixture did not reach the provider with enough history to trigger PR Compact request event history in role prompts #5 compaction, so the measured prompt delta is not attributable to compaction.
• Native usage was available for only 5 of 8 attempted calls in each arm. The corrected benchmark uses the attempted-call denominator, 62.5%; the initial raw report's 71.43% telemetry-record denominator is not trusted for coverage or cost gating.
• No rate card was configured, so monetary cost remains unpriced.
• The large token delta is therefore treated as run-to-run/session variance, not an optimization result.

The scenario has since been corrected so Backfill is applied exactly once to the canonical first planning request before the first research/planner provider call. Future pairs can verify that the Before arm includes all 50 events while the After arm includes 16 and reports 34 omitted.

Validation and safety

• Full test suite: 824 passed, 1 skipped.
• Focused benchmark/accounting/cleanup suites passed.
• git diff --check and package compilation passed.
• Independent final race/identity audit found no high- or medium-severity issue.
• All 19 retained JSON/JSONL artifacts parsed successfully.
• No credential-like content, symlink, world-readable artifact file, or surviving benchmark/provider process was found.
• Worktree is clean; the branch is 6 commits ahead of origin/release/enhancement and is currently unpushed.

Next measurement gate

Run one new v2 live pair only after the benchmark branch is reviewed/pushed. Accept an optimization conclusion only when compaction is observed, workflow outcomes are comparable, invocation identities reconcile, token coverage is disclosed, and pricing coverage is complete when monetary cost is reported.