** Intelligent Retry Strategy Engine with Failure Pattern Recognition and Adaptive Escalation

[github-actions]

Strategic Improvement

** Current retry logic is simple (fixed count). Build a pattern-recognition system that analyzes failure types (API timeout, test flake, merge conflict, context exhaustion) and applies failure-specific retry strategies: exponential backoff for rate limits, immediate retry for flakes, model escalation for context issues, human escalation for persistent failures. Learns optimal retry counts per pattern from historical data.

Acceptance Criteria

• Failure classifier categorizes errors into 8+ types (timeout, flake, conflict, quota, etc.)
• Retry strategy router applies type-specific logic (backoff, model switch, skip stage, escalate)
• Historical success rate per (failure_type, retry_strategy) tracked in database
• Adaptive retry count based on historical P(success | failure_type, retry_n)
• Cost savings metric: wasted retries avoided vs successful recoveries
• Integration test showing 3 failure types handled with different strategies
• Documentation update in CLAUDE.md with retry decision flowchart

Context

• Priority: ** P0
• Complexity: ** standard
• Generated by: Strategic Intelligence Agent
• Strategy alignment: ** P0: Reliability & Success Rate - "Smarter retry strategies based on failure patterns"

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	standard
Max Iterations	20
Triage	Strategic reliability improvement with clear acceptance criteria; well-scoped modular design across pipeline retry, database tracking, and intelligence layers.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	30083052307

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (575 lines)

Implementation Plan: Intelligent Retry Strategy Engine with Failure Pattern Recognition

Analysis & Design Decisions

Requirements Clarity

Minimum viable change: Integrate failure classification + adaptive retry strategy selection into the build loop. The current sw-loop.sh uses fixed retry counts; we need to:

1. Classify failures into 8+ types
2. Store success rates per (failure_type, strategy)
3. Route to type-specific retry logic
4. Adapt retry counts based on historical P(success)

Implicit requirements identified:

• Must integrate with existing SQLite database (sw-db.sh)
• Must not break current loop functionality—feature-flag behind config
• Must provide decision audit trail for transparency
• Must work with bash 3.2 constraints (no assoc arrays, readarray, etc.)

Design Alternatives Considered

Approach A: Unified Classifier + Router (CHOSEN)

• Single module classifies failures, routes to strategies
• Historical data stored in SQLite with schema: (failure_type, strategy, success_count, total_attempts)
• Strategies: exponential backoff (rate limits), immediate retry (flakes), model escalation (context), skip/escalate (persistent)
• Pros: Cohesive, testable, leverages existing db layer
• Cons: Slightly more upfront complexity, requires schema migration
• Blast radius: Contained to loop stage; gated by config flag
• Maintainability: High—explicit decision logic, auditable

Approach B: Inline Classification (Rejected)

• Add case statements directly in loop.sh to classify failures
• Pros: Minimal new files
• Cons: Loop.sh becomes bloated, hard to test, violates single-responsibility, difficult to extend with new failure types
• Blast radius: High—touches loop core logic
• Maintainability: Low—mixed concerns

Approach C: External Service (Rejected)

• Spawn separate agent/service for retry decisions
• Pros: Clean separation
• Cons: Overkill complexity, adds latency, requires inter-process communication
• Blast radius: Moderate—new infrastructure
• Maintainability: Moderate—adds operational complexity

Why Approach A: Balances complexity, testability, and maintainability. Failure classification is deterministic business logic—perfect for unit testing in bash. SQLite integration is proven (sw-db.sh exists). Can be feature-flagged and rolled out safely.

---

Task Decomposition (with Dependencies)

1. Create failure classifier module (scripts/lib/retry-classifier.sh)
   • Regex patterns for 8+ failure types (timeout, flake, conflict, quota, context_exhaustion, network, auth, persistent)

✅ Task Checklist

• Task 1: Create scripts/lib/retry-classifier.sh with 8 failure type patterns
• Task 2: Extend scripts/sw-db.sh with retry_outcomes schema and query functions
• Task 3: Create scripts/lib/retry-router.sh with strategy selection logic
• Task 4: Integrate classifier + router into scripts/sw-loop.sh (behind feature flag)
• Task 5: Create scripts/lib/retry-auditor.sh for decision logging
• Task 6: Create scripts/sw-retry-classifier-test.sh with 40+ test cases
• Task 7: Create scripts/sw-retry-router-test.sh with 30+ test cases
• Task 8: Create scripts/sw-retry-engine-test.sh with 3 integration scenarios
• Task 9: Update scripts/sw-cost.sh to track retry cost savings
• Task 10: Add feature flag to .claude/daemon-config.json
• Task 11: Update .claude/CLAUDE.md with flowchart and decision tables
• Task 12: Create ARCHITECTURE.md section documenting design decisions
• Validation: Run full test suite (npm test) — all 100+ test suites pass
• Validation: Run integration test (bash scripts/sw-retry-engine-test.sh) — verify 3 scenarios
• Validation: Verify database schema exists after first pipeline run
• Validation: Verify retry decisions logged to .claude/pipeline-artifacts/retry-decisions.jsonl
• Failure classifier categorizes errors into 8+ types (timeout, flaky, conflict, quota, context_exhaustion, network, auth, persistent)
• Retry strategy router applies type-specific logic (exponential_backoff, immediate, escalate_model, escalate_human, skip_stage)
• Historical success rate tracked in SQLite (retry_outcomes table, indexed by failure_type + strategy)
• Adaptive retry count based on P(success | failure_type, strategy, retry_n) — query and use highest-confidence strategy

---

Generated by shipwright pipeline at 2026-07-24T09:41:25Z

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (167 lines)

I now have enough grounding. Two findings materially change the design, so the ADR corrects the plan rather than rubber-stamping it:

• classify_failure() already exists in scripts/lib/daemon-failure.sh:22 — it classifies into 6 classes (auth_error, api_error, invalid_issue, context_exhaustion, build_failure, unknown) and is already wired into daemon_on_failure() at line 196 (the memory note saying otherwise is stale). The plan's new classify_failure in retry-classifier.sh would shadow it and fork the taxonomy.
• db_record_outcome() already exists (sw-db.sh:994) using _db_exec + db_available + _SQL_SQ escaping. The plan's raw '$failure_type' interpolation and hand-rolled BEGIN EXCLUSIVE locking contradict the established DB conventions (WAL + .timeout 5000 are already set at sw-db.sh:132,114).

---

Design: Intelligent Retry Strategy Engine with Failure Pattern Recognition and Adaptive Escalation

Context

The build loop (scripts/sw-loop.sh) and daemon (scripts/lib/daemon-failure.sh) currently retry failures with fixed, undifferentiated logic: MAX_RESTARTS (sw-loop.sh:106), a single default strategy="retry_with_context" (sw-loop.sh:874), and exit-code/log-text classification that already exists but does not feed strategy selection or learn from outcomes. Every failure class gets the same escalation path, wasting cost on unwinnable retries (auth, persistent) and under-retrying transient ones (rate limits, flakes).

Constraints imposed by the codebase:

• Bash 3.2 — no associative arrays, readarray, or case-conversion expansions. (Note: the "Project Context" header says TypeScript/vitest; that is the fixture project the loop operates on. Shipwright itself is bash — this ADR is bash.)
• Existing, wired classifier. classify_failure() (daemon-failure.sh:22) is called from daemon_on_failure() (:196). We extend it, not replace it. Introducing a second classify_failure in a sourced lib would shadow the first depending on source order — a latent, order-dependent bug.
• Established DB layer. sw-db.sh provides _db_exec, db_available, _SQL_SQ single-quote escaping, WAL mode (:132), and a 5s busy timeout (:114). db_record_outcome() (:994) is the pattern to mirror. Raw string interpolation into SQL is a documented pitfall in .claude/CLAUDE.md.
• Feature-flag discipline. New pipeline behavior must be config-gated and default-off until validated.

Decision

Build the engine as two new sourced libs plus targeted extensions to existing files — no forked taxonomy, no parallel classifier.

1. Unify, don't duplicate, the classifier. Extend the existing classify_failure() taxonomy in daemon-failure.sh to emit the additional discriminated types the router needs (quota, network, flaky, merge_conflict, persistent) alongside today's auth_error, api_error, build_failure, context_exhaustion, invalid_issue, unknown. The router consumes the string type — it does not care which lib produced it. This keeps one source of truth for "what kind of failure is this."

2. scripts/lib/retry-router.sh — pure decision function. get_retry_strategy(failure_type, prev_attempts, max_retries) → JSON. Returns {strategy, retry_count, backoff_seconds, confidence, reason}. It is a pure function of (its arguments + DB reads); it performs no retries, no sleeps, no side effects. This is the single most testable unit — mockable against sqlite3 :memory:.

Default strategy table (used when history < min_historical_samples):

failure_type	strategy	retry_count	backoff
api_error/quota	exponential_backoff	3	2^n capped 32s
network	exponential_backoff	3	2^n capped 32s
flaky	immediate	2	0
context_exhaustion	escalate_model	1	0
merge_conflict/auth_error/persistent/unknown	escalate_human	0	—

When history ≥ min_historical_samples and confidence ≥ threshold, pick argmax(success_rate), tiebreak to lowest cost. Otherwise fall back to the table. Fail-open: if db_available is false, always use defaults — the engine must never block the loop.

3. scripts/sw-db.sh — add retry_outcomes table + two functions, mirroring db_record_outcome:

• db_record_retry_outcome(...) via _db_exec with _SQL_SQ escaping (NOT '$var' interpolation).
• db_get_success_rate(failure_type, strategy) with a 30-day decay window and a min_historical_samples floor, returning {total, success, rate, confidence}.
• Concurrency: rely on existing WAL + .timeout 5000, not BEGIN EXCLUSIVE. EXCLUSIVE serializes writers and defeats WAL; the busy timeout already handles SQLITE_BUSY. Inserts are single statements — atomic by definition.

4. scripts/lib/retry-auditor.sh — append-only decision log to .claude/pipeline-artifacts/retry-decisions.jsonl built with jq -n --arg (never string-built JSON). Escalation state (budget/cooldown) lives in .claude/pipeline-artifacts/, not /tmp (per-repo isolation, no world-writable race, survives across the run).

5. scripts/sw-loop.sh — extend the existing failure branch, gated by retry_engine.enabled. Slot into the current decision point near sw-loop.sh:874/:2179 where strategy and root-cause diagnosis already live; do not add a parallel control path.

Data flow:

test failure → classify_failure(type) → get_retry_strategy(type,...)
   → [apply: backoff sleep | escalate_model | escalate_human | immediate]
   → run outcome → db_record_retry_outcome → log_retry_decision

Alternatives Considered

1. New standalone retry-classifier.sh with its own classify_failure (the plan as written) — Pros: self-contained, easy to test in isolation. Cons: shadows the existing wired function (source-order-dependent bug), forks the failure taxonomy into two drifting copies, and the memory system already flagged classifier/retry wiring as a past failure area. Rejected.
2. Inline case classification in sw-loop.sh (plan's Approach B) — Pros: no new files. Cons: bloats loop core, untestable in isolation, mixes concerns. Rejected.
3. External decision service/agent (plan's Approach C) — Pros: clean separation. Cons: IPC latency and operational weight for a deterministic lookup. Rejected.

---

Generated by shipwright pipeline design stage at 2026-07-24T09:45:33Z

[github-actions]

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 20 max iterations, 1 agent(s), model: opus

[github-actions]

[github-actions]