** Memory Module Decomposition into Focused, Testable Components

[github-actions]

Strategic Improvement

** sw-memory.sh is 2240 lines (2nd largest script after sw-loop.sh which has issue #779). Decompose into focused modules: lib/memory-capture.sh (failure pattern recording), lib/memory-query.sh (pattern retrieval), lib/memory-cleanup.sh (retention/archival), lib/cost-tracking.sh (budget/spend). Improves maintainability, testability, and follows AGI criterion of reducing monolithic structures.

Acceptance Criteria

• lib/memory-capture.sh created (<400 lines): capture_failure_pattern, emit_learning_event, validate_pattern
• lib/memory-query.sh created (<400 lines): query_patterns, find_similar_failures, get_success_rate
• lib/memory-cleanup.sh created (<300 lines): archive_old_patterns, enforce_retention, vacuum_db
• lib/cost-tracking.sh created (<400 lines): record_cost, check_budget, get_spending_report
• sw-memory.sh becomes thin router (<200 lines) sourcing libs
• All existing memory tests pass, add 20+ new tests for lib functions
• Backwards compatible: all CLI commands work unchanged
• Platform health next scan shows sw-memory.sh <300 lines

Context

• Priority: ** P6
• Complexity: ** standard
• Generated by: Strategic Intelligence Agent
• Strategy alignment: ** P6: Platform Self-Improvement (AGI-Level Readiness)

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	standard
Max Iterations	20
Triage	Strategic refactoring to decompose 2240-line memory module into focused, testable libraries improves maintainability and aligns with Shipwright's AGI-readiness goals; well-scoped with clear acceptance criteria and backwards compatibility requirements.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	30083021647

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (239 lines)

Implementation Plan — Memory Module Decomposition (Issue #806)

Decompose scripts/sw-memory.sh (2240 lines) into focused, testable lib/ modules
behind a thin router, with 100% backwards compatibility for the 16+ scripts that
source it.

---

Reality Check (Brainstorming: Challenge the Assumptions)

The codebase was inspected against the issue's acceptance criteria. Three AC assumptions
do not match the code; corrected decisions are documented here (autonomous mode — no
questions asked).

AC assumption	Reality in sw-memory.sh	Decision
Functions named capture_failure_pattern, query_patterns, record_cost, …	Real functions are memory_capture_failure, memory_search, memory_update_metrics, … (all memory_*-prefixed)	Keep existing names. 16+ scripts call them (memory_capture_failure × 11, memory_finalize_pipeline × 7, memory_inject_context × 5, …). Renaming breaks every caller. Treat AC names as role descriptions, not literal identifiers; expose them as thin delegating wrappers where the AC demands the public surface.
lib/cost-tracking.sh extracts record_cost/check_budget/get_spending_report	No cost/budget logic exists here. Real cost/budget lives in sw-cost.sh (1056 lines). The only "cost" is a token-cost metric on pipeline runs (memory_update_metrics, A/B report).	Create lib/cost-tracking.sh housing the metrics + A/B cost-delta functions. Do not duplicate sw-cost.sh. Add AC-named wrappers (record_cost, check_budget, get_spending_report) that delegate.
Each of 4 libs fits <400/<300 lines	memory_inject_context alone = 247 lines; memory_analyze_failure (133) + memory_capture_pattern (160) are heavy. A naïve 4-way split lands 2 files at 600–900 lines.	Refine to 6 focused libs + 1 shared memory-common.sh. Satisfies the intent (<400/file, single responsibility). The 4 AC-named libs still exist; two are split further because the functions are genuinely large.

Minimum viable change: move function bodies verbatim into lib/ files sourced by a thin
sw-memory.sh; zero behavior change, zero rename. Everything else (wrappers, new tests) supports
that invariant.

---

Component Diagram

              scripts/sw-memory.sh  (thin router, target <200 lines)
              │  sources libs in dependency order; keeps CLI case-dispatch
              ▼
   ┌──────────────────────────────────────────────────────────────┐
   │  lib/memory-common.sh   (shared foundation, ~120 lines)       │
   │  MEMORY_ROOT/GLOBAL_MEMORY, repo_hash, repo_name,             │
   │  repo_memory_dir, ensure_memory_dir, _memory_content_hash,    │
   │  _expand_domain_keywords     ← depended on by ALL below       │
   └──────────────────────────────────────────────────────────────┘
        ▲         ▲          ▲          ▲          ▲          ▲
   ┌────┘    ┌────┘     ┌────┘     ┌────┘     ┌────┘     ┌────┘
┌──┴──────┐┌─┴────────┐┌┴────────┐┌┴────────┐┌┴────────┐┌┴──────────────┐
│capture  ││analysis  ││query    ││inject   ││cleanup  ││cost-tracking  │
│(record) ││(enrich)  ││(retrieve)││(display)││(retain) ││(metrics)      │
└─────────┘└──────────┘└─────────┘└─────────┘└─────────┘└───────────────┘

Dependencies point inward: every module sources only memory-common.sh +
canonical lib/helpers.sh/lib/compat.sh. No module sources a sibling → no circular
deps. analysis/inject may call query functions at runtime (all sourced by the router
before dispatch), but never source each other.

✅ Task Checklist

• Task 1: Baseline tests green + snapshot public function list (before.txt). (blocks all)
• Task 2: Create lib/memory-common.sh (paths/helpers + load guard).
• Task 3: Create lib/memory-capture.sh (verbatim, sources common). (needs T2)
• Task 4: Create lib/memory-analysis.sh. (needs T2)
• Task 5: Create lib/memory-query.sh. (needs T2)
• Task 6: Create lib/memory-inject.sh. (needs T2; calls query at runtime)
• Task 7: Create lib/memory-cleanup.sh + retention wrappers. (needs T2)
• Task 8: Create lib/cost-tracking.sh + cost/budget wrappers. (needs T2)
• Task 9: Rewrite sw-memory.sh as thin router, ordered sourcing, unchanged dispatch. (needs T3–T8; blocks T10)
• Task 10: Diff before.txt vs after.txt — assert zero function-name drift.
• Task 11: sw-lib-memory-capture-test.sh (5+ tests).
• Task 12: sw-lib-memory-query-test.sh (5+ tests).
• Task 13: sw-lib-memory-cleanup-test.sh (5+ tests).
• Task 14: sw-lib-cost-tracking-test.sh (5+ tests).
• Task 15: Register tests in package.json; shipwright docs sync; full regression; verify line-count + version-check.
• scripts/sw-memory.sh is a thin router < 300 lines (target < 200): header + ordered sourcing + case dispatch only.
• lib/memory-capture.sh, memory-query.sh, cost-tracking.sh each < 400; memory-cleanup.sh < 300; memory-common.sh/memory-analysis.sh/memory-inject.sh each < 400.
• AC-named public surface exists as delegating wrappers: record_cost, check_budget, get_spending_report, archive_old_patterns, enforce_retention, vacuum_db.
• Zero function-name drift: diff before.txt after.txt empty.
• Existing sw-memory-test.sh (95 assertions) + sw-memory-effectiveness-test.sh pass unchanged.

---

Generated by shipwright pipeline at 2026-07-24T09:43:28Z

[github-actions]

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (186 lines)

Design: ** Memory Module Decomposition into Focused, Testable Components

Context

scripts/sw-memory.sh is a 2240-line monolith defining ~40 functions across six distinct responsibilities (capture, analysis, retrieval, injection, cleanup/retention, metrics). It is a library-and-CLI hybrid: 16 non-test scripts source it to obtain memory_* functions (verified callers include sw-loop.sh, sw-pipeline.sh, sw-predictive.sh, and 8 lib/pipeline-*/lib/daemon-* modules), and it dispatches a CLI case when executed directly.

Constraints that dominate the design:

1. Backwards compatibility is non-negotiable. Every memory_* name in /tmp/mem-funcs-before.txt must remain defined after refactor, with identical signatures. A rename or a missed source line is a silent "command not found" mid-pipeline in fleet mode.
2. The issue's AC names are fictional. Real functions are memory_*-prefixed (memory_capture_failure, not capture_failure_pattern); record_cost/check_budget logic does not exist here — real budgeting lives in sw-cost.sh (1056 lines). AC names must be honored as thin delegating wrappers, never as renames.
3. A naïve 4-way split violates the <400-line target. memory_inject_context is 247 lines; query+inject combined ≈ 600. Decomposition must be 6 focused libs + 1 shared foundation.
4. Established precedent exists. scripts/lib/ already holds decomposed families (pipeline-*, daemon-*, memory-effectiveness.sh) using an idempotent load-guard idiom ([[ -n "${_X_LOADED:-}" ]] && return 0; _X_LOADED=1) and a helper-fallback block. This design mirrors that convention exactly.

Language context says TypeScript/vitest, but the actual artifact is Bash — the API-design "REST/HTTP" framing is not applicable (see the explicit note in Endpoint Specification below); the public API here is a set of sourced Bash functions.

Decision

Decompose into 6 focused libraries + 1 shared foundation, sourced by a thin router that retains only the header, ordered source block, and CLI case dispatch.

Module boundaries (one reason to change each):

Module	Responsibility	Public functions (verbatim bodies)
lib/memory-common.sh	Paths + shared primitives	repo_hash, repo_name, repo_memory_dir, ensure_memory_dir, _memory_content_hash, _expand_domain_keywords + MEMORY_ROOT/GLOBAL_MEMORY
lib/memory-capture.sh	Write learnings	memory_capture_pipeline, memory_capture_failure, memory_capture_failure_from_log, memory_record_fix_outcome, memory_track_fix, memory_finalize_pipeline, memory_capture_decision, memory_store_for_embedding
lib/memory-analysis.sh	Claude-CLI enrichment	memory_analyze_failure, memory_capture_pattern
lib/memory-query.sh	Retrieval/ranking	memory_ranked_search, memory_semantic_search, _has_embeddings, memory_query_fix_for_error, memory_closed_loop_inject, memory_inject_goal_context, memory_search, memory_search_weighted, memory_get_actionable_failures
lib/memory-inject.sh	Display/context bundle	memory_inject_context, memory_show, memory_stats, show_help
lib/memory-cleanup.sh	Retention/archival I/O	memory_decay_old, memory_forget, _memory_aggregate_global, memory_export, memory_import + wrappers archive_old_patterns/enforce_retention/vacuum_db
lib/cost-tracking.sh	Metrics + A/B cost-delta	memory_update_metrics, memory_get_baseline, memory_get_dora_baseline, memory_ab_assign_group, memory_ab_record_assignment, memory_ab_record_result, cmd_memory_ab_report + wrappers record_cost/check_budget/get_spending_report

Dependency rule — dependencies point inward. Every module sources only memory-common.sh + canonical lib/helpers.sh/lib/compat.sh. No module sources a sibling → no circular deps. analysis and inject call query functions at runtime, which is safe because the router sources all libs before any dispatch.

Load-guard idiom (matches memory-effectiveness.sh): each lib opens with [[ -n "${_MEMORY_<X>_LOADED:-}" ]] && return 0; _MEMORY_<X>_LOADED=1, so double-sourcing memory-common.sh from six siblings is a no-op after the first.

Error handling / invariants preserved verbatim during the move (zero behavior change):

• Atomic writes: tmp file + mv (never echo > file) — critical under fleet-mode decay/capture concurrency.
• JSON via jq --arg (never string interpolation); numeric fields default 0.
• Graceful degradation: Claude-CLI-absent → memory_analyze_failure exits 0 with no partial write; corrupt/empty failures.json → query returns [] exit 0; empty memory → memory_inject_context emits a valid empty bundle.
• $NO_GITHUB guards retained where present.

Router source order (fixed, in sw-memory.sh): memory-common → memory-capture → memory-analysis → memory-query → memory-inject → memory-cleanup → cost-tracking. The if [[ "${BASH_SOURCE[0]}" == "$0" ]] execute-vs-source guard and existing sw-db.sh/sw-intelligence.sh/memory-effectiveness.sh sourcing stay put.

Alternatives Considered

1. Literal 4-module split per AC names, renaming functions (memory_capture_failure→capture_failure_pattern) — Pros: matches the issue text verbatim. Cons: breaks all 16 sourcing callers (memory_capture_failure×11, memory_finalize_pipeline×7, memory_inject_context×5); maximal blast radius, zero benefit. Rejected.
2. 4 modules, ignore the <400-line target — Pros: fewer files, closer to AC's module count. Cons: query+inject ≈ 600 lines, defeating the single-responsibility/maintainability goal that motivates the issue. Rejected in favor of 6 libs + common.
3. Extract real cost/budget logic from sw-cost.sh into lib/cost-tracking.sh — Pros: would make record_cost/check_budget "real." Cons: out of scope; large blast radius on a second 1056-line script; duplicates a working budget system. Rejected — this issue owns memory metrics only; AC names become delegating wrappers (check_budget reads ~/.shipwright/budget.json, delegating to sw-cost.sh if sourced, else no-op returning 0).
4. Split by data file (one lib per failures.json/patterns.json/metrics.json) — Pros: clean data ownership. Cons: capture and query both touch failures.json; would force circular calls or shared-state coupling across libs. Rejected — responsibility (verb) is a cleaner seam than data (noun) here.

Implementation Plan

Files to create:

• scripts/lib/memory-common.sh (~120)
• scripts/lib/memory-capture.sh (<400)
• scripts/lib/memory-analysis.sh (~300)
• scripts/lib/memory-query.sh (<400)
• scripts/lib/memory-inject.sh (~380)
• scripts/lib/memory-cleanup.sh (<300)
• scripts/lib/cost-tracking.sh (<400)

---

Generated by shipwright pipeline design stage at 2026-07-24T09:46:26Z

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 20 max iterations, 1 agent(s), model: opus

[github-actions]

[github-actions]