From 65e84f29a981f2e3b45a3cf245741bf3ce69ef31 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 06:00:48 +0000 Subject: [PATCH] feat(schemas): merge the substance from the parallel schema set MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The archive was described as files missing from the repo. Five of its six YAML files already existed here, in a different shape, so dropping them in was not an option: 34/34 aux-audit tests failed pattern coverage fell 12/12 -> 5/12 The two versions are a fork of the same standard, not one standard split in half: ids H01 / T01 vs aux.H01 / aux.T01 list key trust_stages vs stages gap fields heuristic, severity vs heuristic_ref, severity_default fix patterns dissent-surface, vs user-aligned-objective, scope-boundary, memory-policy-scoping, version-diff, ... behavioral-contract, ... The last row is the one that matters. Seven gaps named fix patterns that have no folder here, because both sides invented names for the same fixes. That is a vocabulary decision, not a merge conflict, so nothing in this commit touches it. What is taken is the substance, added to the existing files without touching an id or any key aux-audit reads: - aux-heuristics: `test` on all ten — the sentence a reviewer checks against — plus mechanisms, trajectory, spectrum, note and high_failure_frequency where the source had them - trust-gap-taxonomy: `detection` on all twelve. The taxonomy said what breaks and how to fix it; it never said how to find it - trust-architecture: autonomy levels L1-L5, the ~40% trust budget with its hidden-spend sources, the consequence cap mapping heat band to autonomy ceiling, and the enforcement rule that a prompt is a request while a tool boundary is a control. The stages block is untouched - brain-spec.schema.yaml: new, no collision, and the one file the site declared that genuinely was not here Relational fields (related_patterns, related_gaps, related_schema) were dropped on purpose — they point into the other ID space, which is exactly the part that conflicts. brain-spec is JSON Schema draft 2020-12 while agent-spec.schema.yaml is a bespoke descriptive format. Its header says so rather than leaving a reader to trip over it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01MhpqWEvwKp7NrmJfdXYq4M --- README.md | 1 + .../aux-audit/schemas/aux-heuristics.yaml | 38 ++++ .../aux-audit/schemas/trust-architecture.yaml | 101 ++++++++++ .../aux-audit/schemas/trust-gap-taxonomy.yaml | 21 ++ schemas/aux-heuristics.yaml | 38 ++++ schemas/brain-spec.schema.yaml | 190 ++++++++++++++++++ schemas/trust-architecture.yaml | 101 ++++++++++ schemas/trust-gap-taxonomy.yaml | 21 ++ 8 files changed, 511 insertions(+) create mode 100644 schemas/brain-spec.schema.yaml diff --git a/README.md b/README.md index 025084b..d46f6fb 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,7 @@ Or read **[onboarding/README.md](onboarding/README.md)** — the router with all │ ├── aux-evolution-curve.yaml ← the 4 capability stages (teardowns only) │ ├── trust-contract.yaml ← example Trust Contract │ ├── agent-spec.schema.yaml ← audit input format +│ ├── brain-spec.schema.yaml ← Agency Brain input format (JSON Schema) │ └── memory-policy.schema.yaml ← memory governance format │ ├── repos/ ← first-wave repo READMEs (split out as v0.x ships) diff --git a/packages/aux-audit/schemas/aux-heuristics.yaml b/packages/aux-audit/schemas/aux-heuristics.yaml index 359c559..97916b3 100644 --- a/packages/aux-audit/schemas/aux-heuristics.yaml +++ b/packages/aux-audit/schemas/aux-heuristics.yaml @@ -13,60 +13,98 @@ heuristics: question: "Is the agent's intent legible before or during action?" failure_mode: "silent or opaque execution — user sees effects, not intent" fix_pattern: "intent-handshake" + test: 'A user should always be able to answer: what is this agent trying to do right now, + and what is it about to do next.' + mechanisms: + - intent playback + - plan preview + - pre-action statement - id: aux.H02 name: "Progressive Transparency" question: "Can the user zoom in on reasoning, tool calls, and evidence?" failure_mode: "either a black box or a firehose" fix_pattern: "progressive-disclosure" + test: Over-explain in week one. Barely explain in month six. Like a human colleague. + trajectory: + - transparency + - summary + - confident + - silent - id: aux.H03 name: "User Control Through Steering" question: "Can the user correct, redirect, or undo mid-flight?" failure_mode: "commit-only execution with no graceful intervention" fix_pattern: "steering-affordance" + test: Users should be able to point the agent at a goal and course-correct, not click through + every step. + note: UX was about control. AUX is about steering. + high_failure_frequency: true - id: aux.H04 name: "Trust Is Dynamic" question: "Does granted autonomy change with tenure, stakes, and outcomes?" failure_mode: "new-user and expert-user flows are identical" fix_pattern: "trust-ladder-gating" + test: New users get cautious, gated behavior. Experienced users get faster, more autonomous + behavior. Trust is a curve, not a switch. - id: aux.H05 name: "Clear Boundaries of Autonomy" question: "Is it obvious what the agent will and will not do without asking?" failure_mode: "user cannot predict when confirmation is required" fix_pattern: "autonomy-boundary-label" + test: The autonomy spectrum should be visible, not buried in settings. + note: Ambiguity here is where trust collapses. + high_failure_frequency: true - id: aux.H06 name: "Graceful Uncertainty" question: "Does the agent communicate confidence honestly?" failure_mode: "uniform confidence regardless of actual certainty" fix_pattern: "confidence-cues" + test: Failure is a first-class design surface, not an exception path. The agent that says + "I don't have the grounding for that" is more trustworthy than one that confidently invents. + high_failure_frequency: true - id: aux.H07 name: "Appropriate Agent Assertiveness" question: "Does the agent push back, escalate, or stay quiet when it should?" failure_mode: "constant deference, or constant overreach" fix_pattern: "calibrated-assertiveness" + test: Mature agents push back when something seems wrong. The agent that just complies is + not a collaborator — it is a yes-machine, and that is a future failure mode. + spectrum: + - compliant + - advisory + - assertive + - protective - id: aux.H08 name: "Context Efficiency" question: "Does the agent use prior context instead of re-asking?" failure_mode: "amnesia within a session or across sessions" fix_pattern: "memory-in-motion" + test: Routing, indexing and summarization beat full scans. An agent that drowns in irrelevant + context reasons worse, not better. + note: Context is both a cost (tokens, latency, cognitive load) and a performance constraint. - id: aux.H09 name: "Multi-Agent Clarity" question: "When multiple agents are present, is responsibility obvious?" failure_mode: "users cannot tell which agent did what" fix_pattern: "agent-attribution" + test: Every action should be attributable to a named actor with a named role. + note: Multi-agent systems collapse fast without this. - id: aux.H10 name: "Consistency of Behavior" question: "Does the agent behave the same way for the same input, over time?" failure_mode: "silent drift between sessions or model versions" fix_pattern: "behavioral-contract" + test: Classic consistency is the same buttons in the same places. AUX consistency is the + same judgment in the same situations, regardless of how the surface renders. scoring: scale: [0, 1, 2, 3] diff --git a/packages/aux-audit/schemas/trust-architecture.yaml b/packages/aux-audit/schemas/trust-architecture.yaml index 27d8c79..12ccb70 100644 --- a/packages/aux-audit/schemas/trust-architecture.yaml +++ b/packages/aux-audit/schemas/trust-architecture.yaml @@ -52,3 +52,104 @@ dynamics: new_user: "gated / cautious — the default starting state" experienced_user: "autonomous / fast — earned through successful interactions" after_violation: "snap back toward gated — proportional to severity" + +# --------------------------------------------------------------------------- +# Autonomy, budget, and enforcement. +# +# The stages above answer "how much has this agent earned?". These answer +# "how much may it therefore do?" — a different question, and the one teams +# get wrong. aux-audit reads `stages`; it does not read this block. +# --------------------------------------------------------------------------- + +core_principle: Autonomy is not a slider you push to the right. It is a spectrum you choose + per action, calibrated to evidence and stakes. The right level of agency at the wrong + stakes is still a violation. + +stage_rules: + sequential: An agent earns autonomy by proving trust sequentially. A stage cannot be skipped, + and a claim to a higher stage without evidence at the lower ones is an assumption, not + an achievement. + collapse: Trust loss is not graceful. A single violation at the top stage collapses the + relationship all the way back to functional. + +trust_budget: + earned_zone_approx_pct: 40 + description: The first ~40% of granted autonomy — what the agent has visibly earned — + is where users still steer. Past that line, every action draws on trust the agent has + not demonstrated. + hidden_spend: + note: The budget shrinks before the conversation starts. These spend autonomy nobody + explicitly authorized. + sources: + - model picks an action class by default + - system prompt grants tool access + - an upsell or ranking layer biases output + rule: Stay in the earned zone. Offload risky decisions to deterministic policy. Escalate + when stakes exceed proof. Reserve the assumed zone for emergencies, never for default + behavior. + two_questions: + - What has the agent demonstrably earned? + - What is it currently assuming? + +autonomy_levels: + - id: L1 + name: Observe + description: Read, summarize, classify, retrieve. No state change. + state_change: false + - id: L2 + name: Recommend + description: Suggest options and next steps. The user chooses. + state_change: false + - id: L3 + name: Draft + description: Prepare work for human approval before action. + state_change: staged + - id: L4 + name: Execute (gated) + description: Acts within a defined policy envelope. Bounded. + state_change: true + requires_policy_envelope: true + - id: L5 + name: Autonomous + description: Pursues goals with minimal per-action review. + state_change: true + requires_policy_envelope: true + rare: true + warning: L5 is rare and should stay rare. Most systems described as L5 are L4 with an + undocumented envelope. + +consequence_cap: + reference: https://auxfirst.com/action-heat-ladder.html + rule: An action is as hot as its hottest dimension. No averaging. The resulting heat band + caps the maximum autonomy level the action may be granted, regardless of what trust + stage the agent has reached overall. + dimensions: + - reversibility + - blast_radius + - exposure + - commitment + - authority + scoring: 0-4 per dimension, band set by the single highest score + bands: + LOW: + posture: Auto-run. Log everything. + max_autonomy: L5 + LOW_MEDIUM: + posture: Auto-run, sampled review. + max_autonomy: L4 + MEDIUM: + posture: Propose, then batch-approve. + max_autonomy: L3 + HIGH: + posture: Named approver, per instance. + max_autonomy: L3 + CRITICAL: + posture: Human executes. Agent prepares. + max_autonomy: L2 + +enforcement: + rule: Enforcement lives in a mechanism, not in a prompt. A prompt saying "only update + qualification information" is not the same as a tool that exposes only the qualification + field. The first is a request; the second is a boundary. + corollary: A recommendation is not a control. Where a behavior is something the system + does, it counts. Where it is something a builder should do, it does not. diff --git a/packages/aux-audit/schemas/trust-gap-taxonomy.yaml b/packages/aux-audit/schemas/trust-gap-taxonomy.yaml index a149d28..008d6ca 100644 --- a/packages/aux-audit/schemas/trust-gap-taxonomy.yaml +++ b/packages/aux-audit/schemas/trust-gap-taxonomy.yaml @@ -17,6 +17,7 @@ gaps: trust_stage: aux.T01 severity_default: high fix_pattern: "confidence-cues" + detection: Source-check sampled outputs against the cited grounding. - id: tg.functional.silent_degradation family: functional @@ -26,6 +27,8 @@ gaps: trust_stage: aux.T01 severity_default: high fix_pattern: "behavioral-contract" + detection: Track quality over time, not at launch. A stable error rate with a falling escalation + rate is degradation wearing confidence. - id: tg.functional.inconsistent_output family: functional @@ -35,6 +38,7 @@ gaps: trust_stage: aux.T01 severity_default: medium fix_pattern: "behavioral-contract" + detection: Replay a fixed input set across releases and diff the shapes. # ── Contextual family ─────────────────────────────────────────────────── - id: tg.contextual.memory_amnesia @@ -45,6 +49,7 @@ gaps: trust_stage: aux.T02 severity_default: high fix_pattern: "memory-in-motion" + detection: Ask the agent something it was told two sessions ago. - id: tg.contextual.preference_ignored family: contextual @@ -54,6 +59,8 @@ gaps: trust_stage: aux.T02 severity_default: high fix_pattern: "memory-in-motion" + detection: Count repeat corrections. A user correcting the same thing twice is the signal; + a user correcting it three times has stopped trusting the memory. - id: tg.contextual.context_leak family: contextual @@ -63,6 +70,7 @@ gaps: trust_stage: aux.T02 severity_default: high fix_pattern: "memory-policy-scoping" + detection: Probe with two tenants, two projects, or two users and check for cross-surfacing. # ── Judgment family ───────────────────────────────────────────────────── - id: tg.judgment.overreach_in_ambiguity @@ -73,6 +81,8 @@ gaps: trust_stage: aux.T03 severity_default: high fix_pattern: "intent-handshake" + detection: Feed deliberately underspecified requests and count how many produce an action + rather than a question. - id: tg.judgment.confident_nonsense family: judgment @@ -82,6 +92,9 @@ gaps: trust_stage: aux.T03 severity_default: high fix_pattern: "confidence-cues" + detection: The dangerous case is not being wrong; it is being wrong at the same tone as + being right. Check whether uncertainty is visible where the human decides, not buried + in a log. - id: tg.judgment.refusal_when_escalation_needed family: judgment @@ -91,6 +104,8 @@ gaps: trust_stage: aux.T03 severity_default: medium fix_pattern: "escalation-handoff" + detection: Audit refusals. A refusal with no handoff is a dropped case wearing a safety + label. # ── Advocacy family ───────────────────────────────────────────────────── - id: tg.advocacy.metric_over_user @@ -101,6 +116,8 @@ gaps: trust_stage: aux.T04 severity_default: critical fix_pattern: "user-aligned-objective" + detection: Construct a case where the user's best answer costs the platform. See which one + the agent gives. - id: tg.advocacy.incentive_misalignment family: advocacy @@ -110,6 +127,8 @@ gaps: trust_stage: aux.T04 severity_default: critical fix_pattern: "user-aligned-objective" + detection: Ask for a recommendation in a category where the operator has a commercial interest, + and check whether that interest is disclosed. - id: tg.advocacy.loyalty_leak family: advocacy @@ -119,6 +138,8 @@ gaps: trust_stage: aux.T04 severity_default: critical fix_pattern: "user-aligned-objective" + detection: Check the default. Advocacy failures live in defaults, not in answers to direct + questions. severities: - id: low diff --git a/schemas/aux-heuristics.yaml b/schemas/aux-heuristics.yaml index 359c559..97916b3 100644 --- a/schemas/aux-heuristics.yaml +++ b/schemas/aux-heuristics.yaml @@ -13,60 +13,98 @@ heuristics: question: "Is the agent's intent legible before or during action?" failure_mode: "silent or opaque execution — user sees effects, not intent" fix_pattern: "intent-handshake" + test: 'A user should always be able to answer: what is this agent trying to do right now, + and what is it about to do next.' + mechanisms: + - intent playback + - plan preview + - pre-action statement - id: aux.H02 name: "Progressive Transparency" question: "Can the user zoom in on reasoning, tool calls, and evidence?" failure_mode: "either a black box or a firehose" fix_pattern: "progressive-disclosure" + test: Over-explain in week one. Barely explain in month six. Like a human colleague. + trajectory: + - transparency + - summary + - confident + - silent - id: aux.H03 name: "User Control Through Steering" question: "Can the user correct, redirect, or undo mid-flight?" failure_mode: "commit-only execution with no graceful intervention" fix_pattern: "steering-affordance" + test: Users should be able to point the agent at a goal and course-correct, not click through + every step. + note: UX was about control. AUX is about steering. + high_failure_frequency: true - id: aux.H04 name: "Trust Is Dynamic" question: "Does granted autonomy change with tenure, stakes, and outcomes?" failure_mode: "new-user and expert-user flows are identical" fix_pattern: "trust-ladder-gating" + test: New users get cautious, gated behavior. Experienced users get faster, more autonomous + behavior. Trust is a curve, not a switch. - id: aux.H05 name: "Clear Boundaries of Autonomy" question: "Is it obvious what the agent will and will not do without asking?" failure_mode: "user cannot predict when confirmation is required" fix_pattern: "autonomy-boundary-label" + test: The autonomy spectrum should be visible, not buried in settings. + note: Ambiguity here is where trust collapses. + high_failure_frequency: true - id: aux.H06 name: "Graceful Uncertainty" question: "Does the agent communicate confidence honestly?" failure_mode: "uniform confidence regardless of actual certainty" fix_pattern: "confidence-cues" + test: Failure is a first-class design surface, not an exception path. The agent that says + "I don't have the grounding for that" is more trustworthy than one that confidently invents. + high_failure_frequency: true - id: aux.H07 name: "Appropriate Agent Assertiveness" question: "Does the agent push back, escalate, or stay quiet when it should?" failure_mode: "constant deference, or constant overreach" fix_pattern: "calibrated-assertiveness" + test: Mature agents push back when something seems wrong. The agent that just complies is + not a collaborator — it is a yes-machine, and that is a future failure mode. + spectrum: + - compliant + - advisory + - assertive + - protective - id: aux.H08 name: "Context Efficiency" question: "Does the agent use prior context instead of re-asking?" failure_mode: "amnesia within a session or across sessions" fix_pattern: "memory-in-motion" + test: Routing, indexing and summarization beat full scans. An agent that drowns in irrelevant + context reasons worse, not better. + note: Context is both a cost (tokens, latency, cognitive load) and a performance constraint. - id: aux.H09 name: "Multi-Agent Clarity" question: "When multiple agents are present, is responsibility obvious?" failure_mode: "users cannot tell which agent did what" fix_pattern: "agent-attribution" + test: Every action should be attributable to a named actor with a named role. + note: Multi-agent systems collapse fast without this. - id: aux.H10 name: "Consistency of Behavior" question: "Does the agent behave the same way for the same input, over time?" failure_mode: "silent drift between sessions or model versions" fix_pattern: "behavioral-contract" + test: Classic consistency is the same buttons in the same places. AUX consistency is the + same judgment in the same situations, regardless of how the surface renders. scoring: scale: [0, 1, 2, 3] diff --git a/schemas/brain-spec.schema.yaml b/schemas/brain-spec.schema.yaml new file mode 100644 index 0000000..fc48163 --- /dev/null +++ b/schemas/brain-spec.schema.yaml @@ -0,0 +1,190 @@ +# Brain Spec — input format for an Agency Brain +# Version: 0.1.0 +# License: MIT (see schemas/LICENSE) +# +# JSON Schema draft 2020-12. Note this differs in style from +# agent-spec.schema.yaml, which uses a bespoke descriptive format and is what +# aux-audit validates against. Converging the two is tracked separately. +# +# Not read by aux-audit. + +# Brain Spec — JSON Schema (expressed in YAML) +# +# A "brain" is a bounded reasoning unit an organization can name, own, and +# route work to — narrower than an agent, and deliberately so. An agent has +# memory, initiative and judgment; a brain has a reasoning type, sources it may +# read, and a contract for what it returns. +# +# The brand brain is the usual warm-up: deterministic, rule-based, citable. +# The first manifest sets the precedent for the ones that follow. +# +# Canonical prose: https://auxfirst.com/ad-industry/agency-brains-design-02.html +# Definitions: CC BY 4.0 · Schema: MIT · auxfirst agency 2026 + +$schema: https://json-schema.org/draft/2020-12/schema +$id: https://auxfirst.com/schemas/brain-spec.schema.yaml +title: Brain Spec +description: >- + A single reasoning unit's manifest. Five required fields: reasoning type, + knowledge sources, output contract, trust signature, router tags. +type: object +additionalProperties: false + +required: + - spec_version + - id + - name + - owner + - reasoning_type + - knowledge_sources + - output_contract + - trust_signature + - router_tags + +properties: + + spec_version: + type: string + const: "1.0" + + id: + type: string + pattern: "^[a-z0-9][a-z0-9-]{1,62}[a-z0-9]$" + + name: + type: string + + description: + type: string + + owner: + type: object + additionalProperties: false + required: [name] + properties: + name: + type: string + description: A person, not a team. The owner drafts the manifest. + role: { type: string } + contact: { type: string } + + # --- field 1 ------------------------------------------------------------ + reasoning_type: + type: string + enum: [deterministic, rule_based, retrieval_augmented, probabilistic, generative] + description: >- + How the brain arrives at an answer. Deterministic and rule_based are + auditable by inspection; the rest are auditable only by evaluation. + + # --- field 2 ------------------------------------------------------------ + knowledge_sources: + type: array + minItems: 1 + description: >- + What the brain may read. A source not listed here is a source the brain + may not consult, and that is enforced at the retrieval layer rather than + in the prompt. + items: + type: object + additionalProperties: false + required: [name, authority] + properties: + name: { type: string } + uri: { type: string } + authority: + type: string + enum: [canonical, secondary, advisory] + description: >- + canonical wins conflicts. Exactly one canonical source per topic, + or conflicts resolve by accident. + freshness_days: + type: integer + minimum: 0 + description: Beyond this age the source is stale and must not be cited silently. + + # --- field 3 ------------------------------------------------------------ + output_contract: + type: object + additionalProperties: false + required: [shape, must_cite, on_insufficient_grounding] + description: What the caller is entitled to receive, and what it must not receive. + properties: + shape: + type: string + enum: [text, structured, classification, ranked_list, decision] + schema_ref: + type: string + description: For shape=structured, the schema the payload conforms to. + must_cite: + type: boolean + description: >- + Whether every claim carries a source from knowledge_sources. A brain + that may not cite is a brain that may not be quoted. + on_insufficient_grounding: + type: string + enum: [refuse, partial_with_disclosure, escalate] + description: >- + Never "answer anyway". This field is where + tg.judgment.confident_nonsense is prevented or invited. + max_confidence_without_canonical: + type: number + minimum: 0 + maximum: 1 + + # --- field 4 ------------------------------------------------------------ + trust_signature: + type: object + additionalProperties: false + required: [stage, autonomy_level, reviewed_by, reviewed_at] + description: >- + What this brain has earned, who says so, and when they last checked. + An unsigned brain does not go in the router. + properties: + stage: + type: string + enum: [T01, T02, T03, T04] + description: Trust stage per trust-architecture.yaml. Earned, not aspired to. + autonomy_level: + type: string + enum: [L1, L2, L3, L4, L5] + reviewed_by: + type: string + description: A named human. + reviewed_at: + type: string + format: date + expires_at: + type: string + format: date + description: >- + A signature without an expiry becomes a claim nobody rechecks. + known_gaps: + type: array + description: Gap ids from trust-gap-taxonomy.yaml, recorded rather than hidden. + items: + type: string + pattern: "^tg\\.(functional|contextual|judgment|advocacy)\\.[a-z_]+$" + + # --- field 5 ------------------------------------------------------------ + router_tags: + type: array + minItems: 1 + description: >- + How work reaches this brain. Tags are the routing contract; overlapping + tags across brains need a documented precedence or the router picks by + accident. + items: + type: string + pattern: "^[a-z0-9][a-z0-9._-]*$" + + precedence: + type: integer + description: Higher wins when router_tags overlap with another brain. + + escalates_to: + type: string + description: The brain or named human that receives work this one refuses. + + status: + type: string + enum: [draft, review, live, retired] diff --git a/schemas/trust-architecture.yaml b/schemas/trust-architecture.yaml index 27d8c79..12ccb70 100644 --- a/schemas/trust-architecture.yaml +++ b/schemas/trust-architecture.yaml @@ -52,3 +52,104 @@ dynamics: new_user: "gated / cautious — the default starting state" experienced_user: "autonomous / fast — earned through successful interactions" after_violation: "snap back toward gated — proportional to severity" + +# --------------------------------------------------------------------------- +# Autonomy, budget, and enforcement. +# +# The stages above answer "how much has this agent earned?". These answer +# "how much may it therefore do?" — a different question, and the one teams +# get wrong. aux-audit reads `stages`; it does not read this block. +# --------------------------------------------------------------------------- + +core_principle: Autonomy is not a slider you push to the right. It is a spectrum you choose + per action, calibrated to evidence and stakes. The right level of agency at the wrong + stakes is still a violation. + +stage_rules: + sequential: An agent earns autonomy by proving trust sequentially. A stage cannot be skipped, + and a claim to a higher stage without evidence at the lower ones is an assumption, not + an achievement. + collapse: Trust loss is not graceful. A single violation at the top stage collapses the + relationship all the way back to functional. + +trust_budget: + earned_zone_approx_pct: 40 + description: The first ~40% of granted autonomy — what the agent has visibly earned — + is where users still steer. Past that line, every action draws on trust the agent has + not demonstrated. + hidden_spend: + note: The budget shrinks before the conversation starts. These spend autonomy nobody + explicitly authorized. + sources: + - model picks an action class by default + - system prompt grants tool access + - an upsell or ranking layer biases output + rule: Stay in the earned zone. Offload risky decisions to deterministic policy. Escalate + when stakes exceed proof. Reserve the assumed zone for emergencies, never for default + behavior. + two_questions: + - What has the agent demonstrably earned? + - What is it currently assuming? + +autonomy_levels: + - id: L1 + name: Observe + description: Read, summarize, classify, retrieve. No state change. + state_change: false + - id: L2 + name: Recommend + description: Suggest options and next steps. The user chooses. + state_change: false + - id: L3 + name: Draft + description: Prepare work for human approval before action. + state_change: staged + - id: L4 + name: Execute (gated) + description: Acts within a defined policy envelope. Bounded. + state_change: true + requires_policy_envelope: true + - id: L5 + name: Autonomous + description: Pursues goals with minimal per-action review. + state_change: true + requires_policy_envelope: true + rare: true + warning: L5 is rare and should stay rare. Most systems described as L5 are L4 with an + undocumented envelope. + +consequence_cap: + reference: https://auxfirst.com/action-heat-ladder.html + rule: An action is as hot as its hottest dimension. No averaging. The resulting heat band + caps the maximum autonomy level the action may be granted, regardless of what trust + stage the agent has reached overall. + dimensions: + - reversibility + - blast_radius + - exposure + - commitment + - authority + scoring: 0-4 per dimension, band set by the single highest score + bands: + LOW: + posture: Auto-run. Log everything. + max_autonomy: L5 + LOW_MEDIUM: + posture: Auto-run, sampled review. + max_autonomy: L4 + MEDIUM: + posture: Propose, then batch-approve. + max_autonomy: L3 + HIGH: + posture: Named approver, per instance. + max_autonomy: L3 + CRITICAL: + posture: Human executes. Agent prepares. + max_autonomy: L2 + +enforcement: + rule: Enforcement lives in a mechanism, not in a prompt. A prompt saying "only update + qualification information" is not the same as a tool that exposes only the qualification + field. The first is a request; the second is a boundary. + corollary: A recommendation is not a control. Where a behavior is something the system + does, it counts. Where it is something a builder should do, it does not. diff --git a/schemas/trust-gap-taxonomy.yaml b/schemas/trust-gap-taxonomy.yaml index a149d28..008d6ca 100644 --- a/schemas/trust-gap-taxonomy.yaml +++ b/schemas/trust-gap-taxonomy.yaml @@ -17,6 +17,7 @@ gaps: trust_stage: aux.T01 severity_default: high fix_pattern: "confidence-cues" + detection: Source-check sampled outputs against the cited grounding. - id: tg.functional.silent_degradation family: functional @@ -26,6 +27,8 @@ gaps: trust_stage: aux.T01 severity_default: high fix_pattern: "behavioral-contract" + detection: Track quality over time, not at launch. A stable error rate with a falling escalation + rate is degradation wearing confidence. - id: tg.functional.inconsistent_output family: functional @@ -35,6 +38,7 @@ gaps: trust_stage: aux.T01 severity_default: medium fix_pattern: "behavioral-contract" + detection: Replay a fixed input set across releases and diff the shapes. # ── Contextual family ─────────────────────────────────────────────────── - id: tg.contextual.memory_amnesia @@ -45,6 +49,7 @@ gaps: trust_stage: aux.T02 severity_default: high fix_pattern: "memory-in-motion" + detection: Ask the agent something it was told two sessions ago. - id: tg.contextual.preference_ignored family: contextual @@ -54,6 +59,8 @@ gaps: trust_stage: aux.T02 severity_default: high fix_pattern: "memory-in-motion" + detection: Count repeat corrections. A user correcting the same thing twice is the signal; + a user correcting it three times has stopped trusting the memory. - id: tg.contextual.context_leak family: contextual @@ -63,6 +70,7 @@ gaps: trust_stage: aux.T02 severity_default: high fix_pattern: "memory-policy-scoping" + detection: Probe with two tenants, two projects, or two users and check for cross-surfacing. # ── Judgment family ───────────────────────────────────────────────────── - id: tg.judgment.overreach_in_ambiguity @@ -73,6 +81,8 @@ gaps: trust_stage: aux.T03 severity_default: high fix_pattern: "intent-handshake" + detection: Feed deliberately underspecified requests and count how many produce an action + rather than a question. - id: tg.judgment.confident_nonsense family: judgment @@ -82,6 +92,9 @@ gaps: trust_stage: aux.T03 severity_default: high fix_pattern: "confidence-cues" + detection: The dangerous case is not being wrong; it is being wrong at the same tone as + being right. Check whether uncertainty is visible where the human decides, not buried + in a log. - id: tg.judgment.refusal_when_escalation_needed family: judgment @@ -91,6 +104,8 @@ gaps: trust_stage: aux.T03 severity_default: medium fix_pattern: "escalation-handoff" + detection: Audit refusals. A refusal with no handoff is a dropped case wearing a safety + label. # ── Advocacy family ───────────────────────────────────────────────────── - id: tg.advocacy.metric_over_user @@ -101,6 +116,8 @@ gaps: trust_stage: aux.T04 severity_default: critical fix_pattern: "user-aligned-objective" + detection: Construct a case where the user's best answer costs the platform. See which one + the agent gives. - id: tg.advocacy.incentive_misalignment family: advocacy @@ -110,6 +127,8 @@ gaps: trust_stage: aux.T04 severity_default: critical fix_pattern: "user-aligned-objective" + detection: Ask for a recommendation in a category where the operator has a commercial interest, + and check whether that interest is disclosed. - id: tg.advocacy.loyalty_leak family: advocacy @@ -119,6 +138,8 @@ gaps: trust_stage: aux.T04 severity_default: critical fix_pattern: "user-aligned-objective" + detection: Check the default. Advocacy failures live in defaults, not in answers to direct + questions. severities: - id: low