Hardcoded Policy Migration Engine with Adaptive Override Framework

[github-actions]

Strategic Improvement

Platform health scan shows 48 hardcoded values blocking true self-improvement. Build a scanner that extracts hardcoded tunables (timeouts, thresholds, limits, retry counts) from scripts, migrates them to config/policy.json with defaults, and adds runtime override support. Each value gets: config key, default, env var override, adaptive hint (which intelligence module might tune it). This is foundational for the platform to improve itself - current hardcoding prevents learning-based optimization. Targets top 20 highest-impact tunables first (identified via git blame frequency + intelligence module references).

Acceptance Criteria

• Scanner identifies hardcoded numeric thresholds, timeouts, limits in all scripts
• Generates config/policy.json with structured schema (section → key → {default, env_var, adaptive_hint, rationale})
• Refactors top 20 highest-impact scripts to read from config with fallback to env vars
• Helper function _policy_int <section> <key> [default] added to lib/compat.sh
• Documents migration path for remaining 28 values
• All tests pass after migration (validates backward compatibility)
• Adds config validation to shipwright doctor

Context

• Priority: P6
• Complexity: full
• Generated by: Strategic Intelligence Agent
• Strategy alignment: P6: Platform Self-Improvement (AGI-Level Readiness)

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	full
Max Iterations	30
Triage	Foundational infrastructure enabling Shipwright's self-optimization — extracts hardcoded values across 20+ scripts into a policy layer with env override support, touching config layer, helper functions, multiple test suites, and doctor validation.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	27483724637

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (61 lines)

I've created a comprehensive implementation plan for issue #660: Hardcoded Policy Migration Engine with Adaptive Override Framework.

Plan Summary

The plan (saved to .claude/IMPLEMENTATION_PLAN.md) addresses all acceptance criteria with:

Architecture Decisions

• Centralized Config: All 48 hardcoded values → config/policy.json (single source of truth)
• Helper Function: _policy_int <section> <key> [default] in scripts/lib/compat.sh with 3-level fallback chain (env var → config → parameter)
• Naming Convention: SW_POLICY_<SECTION>_<KEY> for environment overrides
• Backward Compatibility: 3-phase migration (20 now, 28 later) with fallback defaults at every call site

Files to Modify (20+ files)

Create:

• scripts/lib/policy-scanner.sh — Discover remaining hardcoded values
• config/policy.schema.json — JSON schema for validation
• .claude/policy-migration-guide.md — Roadmap for remaining 28 values
• Test suites for scanner and policy validation

Modify:

• scripts/lib/compat.sh — Add _policy_int helper
• config/policy.json — Extend with top 20 hardcoded values
• scripts/sw-doctor.sh — Add policy validation check
• Top 20 highest-impact scripts: sw-loop.sh, sw-daemon.sh, sw-adaptive.sh, sw-pipeline.sh, sw-cost.sh, sw-feedback.sh, sw-recruit.sh, sw-auth.sh, sw-predictive.sh, sw-intelligence.sh, and 10 others

19-Task Checklist

Granular tasks from analysis → implementation → testing → documentation

Testing Strategy (Test Pyramid)

• Unit tests (65%): _policy_int validation, scanner detection patterns
• Integration tests (30%): config.json schema, script behavior with overrides
• E2E tests (5%): daemon/pipeline respect policy, doctor validation works

Failure Mode Analysis (5 Critical Scenarios)

Most critical: Config File Missing/Malformed

• Mitigation: Every script has hardcoded fallback default (3rd param to _policy_int)
• Detection: Doctor check validates JSON validity first
• Recovery: Automatic restore from git or --fix option

Also covered:

• Invalid environment variable values (non-numeric) → safely ignored
• Incomplete migration (inconsistent behavior) → scanner + phased roadmap
• Out-of-bounds values → constraints enforced, warnings logged
• Version mismatch → schema versioning + backward compatibility

Definition of Done (24 checkboxes)

Including code quality, testing, documentation, and acceptance criteria validation

---

Generated by shipwright pipeline at 2026-06-14T00:48:38Z

[github-actions]

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (153 lines)

I now have the real picture. The plan has a significant flaw: it proposes building parallel infrastructure (_policy_int in compat.sh, an inline-metadata schema, a new SW_POLICY_* convention) that duplicates and would break the already-existing scripts/lib/policy.sh (policy_get), the flat-value config/policy.json (v2), and the ~14 scripts already consuming it. The ADR below corrects course.

---

Design: Hardcoded Policy Migration Engine with Adaptive Override Framework

Context

Problem. ~48 numeric tunables (timeouts, thresholds, retry counts, limits) are hardcoded across the bash scripts. Adaptive agents (sw-adaptive.sh, sw-self-optimize.sh, sw-policy-learner) cannot discover or tune what they cannot see, so the platform cannot self-optimize. Issue #660 asks us to centralize these values and make them overridable at runtime.

Critical constraint the plan missed — this is brownfield, not greenfield. The repository already ships the core of this system:

• config/policy.json (version "2") — already the single source of truth, with nested sections (daemon, pipeline, quality, decision, recruit, strategic, mergePolicy, riskTierRules, …). Values are stored as plain scalars ("poll_interval_seconds": 60).
• scripts/lib/policy.sh — already exposes policy_get <jq_path> [default] and is sourced by ~14 scripts (sw-daemon.sh, sw-decide.sh, sw-hygiene.sh, sw-incident.sh, sw-recruit.sh, sw-model-router.sh, sw-strategic.sh, …).
• config/policy.schema.json and scripts/sw-policy-e2e-test.sh ("Verify config/policy.json is honored") already exist.
• _smart_int in compat.sh exists but reads daemon-config.json (a different file) via SW_<KEY> env override — a separate concern that must not be merged into policy.

The actual gap is narrow: (1) policy_get has no environment-variable override — adaptive agents and operators can't override a policy value at runtime without editing the JSON; (2) there is no numeric validation / bounds clamping; (3) the remaining ~28 unmigrated call sites still hardcode values; (4) there is no scanner to find them or doctor check to guard the file.

The plan's proposed inline-metadata schema ({"default":N,"env_var":...,"min":...,"max":...} per value) is incompatible with the current flat structure — adopting it rewrites every value and breaks all 14 existing policy_get consumers. We reject it (see Alternatives).

Decision

Extend the existing policy.sh, do not build a parallel system in compat.sh. Keep policy.json values as flat scalars. Add a thin, backward-compatible override + validation layer and keep all metadata (bounds, adaptive hints) in a sidecar block, never wrapping the live values.

Component decomposition

┌──────────────────────────────────────────────────────────────┐
│  config/policy.json (v2, EXTENDED)                            │
│   • flat scalar values (unchanged shape) ← 14 consumers safe  │
│   • new "_meta" sidecar: { "daemon.poll_interval_seconds":    │
│       {min,max,adaptive_hint,tags} }  (optional, non-breaking) │
└───────────────┬──────────────────────────────────────────────┘
                │ read
        ┌───────▼─────────┐      env: SW_POLICY_<PATH>
        │ scripts/lib/    │◄──────────────────────────────┐
        │ policy.sh       │                                │
        │  policy_get()   │  (unchanged signature)         │
        │  policy_int()   │  NEW: env→json→default + clamp │
        └───────┬─────────┘                                │
     source     │ called by                                │
   ┌────────────┼───────────────┐                          │
   ▼            ▼               ▼                           │
sw-daemon.sh  sw-loop.sh*   sw-pipeline.sh*  …18 scripts   │
   │                                                        │
   ▼ writes override                                        │
sw-adaptive.sh / sw-policy-learner  ──────────────────────┘
   (set SW_POLICY_* or patch policy.json atomically)

   ┌─────────────────────────┐   ┌──────────────────────────┐
   │ scripts/lib/            │   │ sw-doctor.sh             │
   │ policy-scanner.sh (NEW) │   │  Check N: policy valid?  │
   │  finds remaining N=[0-9]+│   │  json ok · in bounds ·   │
   │  cross-refs policy.json  │   │  unknown SW_POLICY_* warn│
   └─────────────────────────┘   └──────────────────────────┘

* = scripts to migrate this sprint. Dependencies point inward: scripts depend on policy.sh; policy.sh depends only on policy.json + jq. No script depends on the scanner or doctor.

---

Generated by shipwright pipeline design stage at 2026-06-14T00:51:53Z

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 30 max iterations, 1 agent(s), model: opus

[github-actions]

[github-actions]