diff --git a/skills/agency-health/SKILL.md b/skills/agency-health/SKILL.md new file mode 100644 index 000000000..3687de08f --- /dev/null +++ b/skills/agency-health/SKILL.md @@ -0,0 +1,127 @@ +--- +name: agency-health +description: Assemble a typed health bundle for one running agency case by composing the registry-pinned data-store read_projection with cross-run ledger aggregates, grade findings against a declared baseline, and seal a read-only health_verdict plus named intervention findings; the lane itself moves no money, grants no authority, and routes consequences through separate governed runs. +runx: + category: agency-ops +--- + +# Agency Health + +`agency-health` reads one running agency end to end, grades its operational +health, and seals a typed verdict plus named intervention findings. It is +**read-only**: it appends nothing to the case stream, sends nothing, executes +nothing, and consumes no effect. Operators (or downstream drivers) use the +sealed findings to launch separate, scoped governed runs (a `policy-author` +tighten, an `improve-skill` debug, a `human ops` escalation); this skill never +mints, settles, sends, or widens authority itself. + +Use this skill when one of your agencies has been running for at least a +couple of days and you need an at-a-glance signal about whether its stream is +healthy (turns advancing, refusals steady, spend well below cap) versus +needing an intervention (stuck turns, refusal spikes, cap pressure). Do not +use this skill to *resolve* an incident, refund a customer, or rewrite a +policy; the verdict names a lane to call next. `run-history-analyst` audits +the whole receipt ledger; `receipt-auditor` walks one receipt; this skill +walks one live agency case end to end. + +## Operating model + +The skill is composed of one read-only graph that runs in order: + +1. **inspect-case** — read the agency case state by composing the + registry-pinned `data-store` read_projection (C2) keyed on the agency + case, returning events folded in turn order with each event's + `case_id`, `turn`, `driver_id`, `event_kind`, and `version`. +2. **aggregate-ledger** — read cross-run aggregates (seal rate, refusal + spikes) by composition with the `ledger` read runner (C7), referencing + receipts by `id-stub` only (the ledger is audit-only and can never be a + domain-keyed state read). +3. **grade** — fold the case projection and the ledger aggregates, grade + the four canonical signals (seal_rate, stuck_case_count, cap_usage_pct, + escalation_backlog) against `health_baseline` thresholds or the supplied + defaults, and emit one `health_verdict.status` (healthy / degraded / + at_risk) plus a list of typed `findings` that each ground a metric in a + folded case_id and turn number or a referenced ledger id-stub. +4. **name-lane** — for each warrant of intervention, attach a + `target_lane` (one of `policy-author`, `improve-skill`, `human ops`) + and a one-line `reason` that the operator can act on. Lanes that would + widen a cap or authority, and any critical finding, escalate to the + `human ops` lane. +5. **seal** — emit a thin review act over the verdict (status `sealed` + for `ready` and `sealed` for `needs_more_evidence` STOP case) without + composing anything in the case stream. + +The two harness cases correspond to the canonical decision branches: + +- `concerning-agency-sealed` — running agency with stuck turns and cap + pressure yields `decision=ready`, `health_verdict.status=degraded`, + graded findings, and three typed intervention findings naming + `policy-author`, `improve-skill`, and `human ops` lanes; expect + `status: sealed`. +- `no-case-events-stop` — an `agency_ref` with no readable case events + over the period yields `decision=needs_more_evidence`, no findings + graded, no intervention emitted, and `status: sealed` for the + deterministic conflict that still seals. + +## Skill chain + +- **Upstream:** the agency itself (the case stream this skill reads is the + same stream the agency's runners append to). +- **Downstream:** `policy-author` (to tighten a policy or timeout), + `improve-skill` (to debug a member behind a refusal spike), `human ops` + (to escalate cap-widening or critical findings). Use `vault-unseal` to + audit the cases this skill flags as `at_risk`. + +## Inputs + +The runner accepts these typed inputs: + +```text +data_source_ref: string Provider reference, e.g. "registry-pinned/0.6.14". +store_id: string Registry store id (the pinned data-store). +agency_ref: object {case_id, driver_id, agency_charter_id, period}. +period: object? Optional {start_turn, end_turn, allow_partial=false}. +case_id: string? Optional override; defaults to agency_ref.case_id. +health_baseline: object? Optional {threshold_days_stuck=3, cap_pressure_pct=80, refusal_spike_rate=0.10}. +``` + +## Outputs + +The runner returns one typed decision: + +```text +decision: + ready|needs_more_evidence|needs_human +health_verdict: + status: healthy|degraded|at_risk + period: {start_turn, end_turn} + findings: + - signal: seal_rate | stuck_case_count | cap_usage_pct | escalation_backlog + assessment: pass | warn | fail + measured: + baseline: + grounded_in: + case_id: + turn: # for case projection reads + ledger_id_stub: # for ledger aggregate reads + reason: +intervention_findings: + - target_lane: policy-author | improve-skill | human ops + reason: + grounded_in: { case_id, turn, ledger_id_stub } + severity: warn | critical +``` + +This lane moves no money and grants no authority. The `human ops` lane is +the only escalation route for any cap-widening or authority-widening +remedy; this skill refuses to widen a cap or grant access itself. + +## Refusals + +The skill refuses to grade a signal not grounded in the folded case +projection or a ledger id-stub aggregate, refuses to invent a cap or +threshold it cannot read from the agency charter snapshot or the supplied +`health_baseline`, and never invents a turn state the sealed event order +does not show. A `needs_more_evidence` decision is a **deterministic** +sealed case, never a soft skip; the operator sees the refuse reason and +extends the period or supplies the missing baseline. diff --git a/skills/agency-health/X.yaml b/skills/agency-health/X.yaml new file mode 100644 index 000000000..c91b3fadf --- /dev/null +++ b/skills/agency-health/X.yaml @@ -0,0 +1,221 @@ +skill: agency-health +version: "0.1.0" + +catalog: + kind: graph + audience: operator + visibility: public + role: canonical + execution: read + completion: review + requires_adapter: false + approval: none + +runners: + assess: + default: true + type: graph + inputs: + data_source_ref: + type: string + required: true + description: Provider reference, e.g. registry-pinned/0.6.14. + store_id: + type: string + required: true + description: Registry store id (the pinned data-store). + agency_ref: + type: json + required: true + description: Agency reference containing case_id, driver_id, agency_charter_id, and period. + period: + type: json + required: false + description: Optional period with start_turn, end_turn, allow_partial. + case_id: + type: string + required: false + description: Optional override, defaults to agency_ref.case_id. + health_baseline: + type: json + required: false + description: Optional thresholds (threshold_days_stuck, cap_pressure_pct, refusal_spike_rate). + graph: + name: agency-health + result_from: [seal] + steps: + - id: inspect-case + label: read the agency case projection in folded turn order + tool: runx.data_store.read_projection + scopes: + - data_store:read + inputs: + store_id: $input.store_id + case_id: $input.agency_ref.case_id + period: $input.period + projection: agency-case-event-stream + artifacts: + wrap_as: case_projection + packet: runx.data_store.case_projection.v1 + - id: aggregate-ledger + label: read cross-run aggregates by receipt id-stub + tool: runx.ledger.read_aggregates + scopes: + - ledger:read_aggregates + inputs: + data_source_ref: $input.data_source_ref + filter: agency=$input.agency_ref.case_id + metrics: [seal_rate, refusal_count] + artifacts: + wrap_as: ledger_aggregates + packet: runx.ledger.aggregates.v1 + - id: grade + label: grade the four canonical signals against the baseline + run: + type: agent-task + agent: health-grader + task: agency-health-grade + outputs: + health_verdict: object + allowed_tools: + - data_store.read + - ledger.read_aggregates + context: + case_projection: inspect-case.case_projection.data + ledger_aggregates: aggregate-ledger.ledger_aggregates.data + inputs: + health_baseline: $input.health_baseline + agency_charter_id: $input.agency_ref.agency_charter_id + case_id: $input.agency_ref.case_id + period: $input.period + artifacts: + wrap_as: health_verdict + packet: runx.agency.health_verdict.v1 + - id: name-lane + label: attach a target lane to each warranted intervention + run: + type: agent-task + agent: lane-namer + task: agency-health-name-lane + outputs: + intervention_findings: object + allowed_tools: + - data_store.read + context: + health_verdict: grade.health_verdict.data + inputs: + agency_ref: $input.agency_ref + case_id: $input.agency_ref.case_id + artifacts: + wrap_as: intervention_findings + packet: runx.agency.intervention_findings.v1 + - id: seal + label: seal the assessment verdict without case-stream mutation + tool: runx.receipt.seal_review + scopes: + - receipt:seal + inputs: + packet_kind: runx.agency.assessment.v1 + seal_kind: review + context: + health_verdict: grade.health_verdict.data + intervention_findings: name-lane.intervention_findings.data + +harness: + cases: + - name: concerning-agency-sealed + runner: assess + inputs: + data_source_ref: registry-pinned/0.6.14 + store_id: store-agency-runa-001 + agency_ref: | + { + "case_id": "agc-001", + "driver_id": "drv-001", + "agency_charter_id": "charter-agc-001", + "period": {"start_turn": 1, "end_turn": 12} + } + period: {"start_turn": 1, "end_turn": 12, "allow_partial": false} + health_baseline: {"threshold_days_stuck": 3, "cap_pressure_pct": 80, "refusal_spike_rate": 0.10} + caller: + answers: + agent_task.agency-health-grade.output: + health_verdict: + schema: runx.agency.health_verdict.v1 + status: degraded + period: {"start_turn": 1, "end_turn": 12} + findings: + - signal: stuck_case_count + assessment: warn + measured: 4 + baseline: 3 + grounded_in: {"case_id": "agc-001", "turn": 8, "ledger_id_stub": "r/le-77d1ab"} + reason: 4 turns parked awaiting_approval over 3-day baseline. + - signal: cap_usage_pct + assessment: warn + measured: 78 + baseline: 80 + grounded_in: {"case_id": "agc-001", "turn": 11, "ledger_id_stub": "r/cap-77d1ab"} + reason: payment_ceiling reached 78 percent of charter cap. + - signal: seal_rate + assessment: pass + measured: 0.92 + baseline: 0.85 + grounded_in: {"case_id": "agc-001", "turn": 12, "ledger_id_stub": "r/seal-77d1ab"} + reason: 92 percent of sealed turns in period vs 85 baseline. + - signal: escalation_backlog + assessment: pass + measured: 0 + baseline: 0 + grounded_in: {"case_id": "agc-001", "turn": 12, "ledger_id_stub": "r/esc-77d1ab"} + reason: zero escalations pending. + agent_task.agency-health-name-lane.output: + intervention_findings: + schema: runx.agency.intervention_findings.v1 + findings: + - target_lane: improve-skill + reason: A member is behind a refusal spike on await-input. + grounded_in: {"case_id": "agc-001", "turn": 8} + severity: warn + - target_lane: policy-author + reason: Tighten the timeout for awaiting_approval on charter-agc-001. + grounded_in: {"case_id": "agc-001", "turn": 11} + severity: warn + - target_lane: human ops + reason: Flag the cap-usage trend for charter review. + grounded_in: {"case_id": "agc-001", "turn": 12} + severity: warn + expect: + status: sealed + + - name: no-case-events-stop + runner: assess + inputs: + data_source_ref: registry-pinned/0.6.14 + store_id: store-agency-runa-001 + agency_ref: | + { + "case_id": "agc-empty", + "driver_id": "drv-empty", + "agency_charter_id": "charter-agc-empty", + "period": {"start_turn": 1, "end_turn": 30} + } + period: {"start_turn": 1, "end_turn": 30, "allow_partial": false} + health_baseline: {"threshold_days_stuck": 3, "cap_pressure_pct": 80, "refusal_spike_rate": 0.10} + caller: + answers: + agent_task.agency-health-grade.output: + health_verdict: + schema: runx.agency.health_verdict.v1 + status: healthy + period: {"start_turn": 1, "end_turn": 30} + findings: [] + reason: No readable case events over the period, nothing to grade. + agent_task.agency-health-name-lane.output: + intervention_findings: + schema: runx.agency.intervention_findings.v1 + findings: [] + expect: + status: sealed + refusal: + contains: "no case events readable over the period"