Skip to content

Cohort scoring mints 1.0 from missing evidence (cold-start) + ask: consent-gated occurrence-scoped capacity endpoint #19

Description

@emooreatx

Two coupled items from the CIRISAgent 2.9.6 health/capacity work (the agent-side erroneous-1.0 fix landed as CIRISAgent e09e3eec0; this is the cohort half).

1. Bug: missing evidence defaults to perfect scores

Live prod payload (GET /lens-api/api/v1/scoring/capacity/Ally, 2026-06-11): composite_score: 1.0, all five factors 1.0, at trace_count: 18, confidence: "low" — and fragility_index: 0.999 rendered alongside category: "high_capacity".

Default-to-perfect sites in api/ciris_scoring.py:

  • ~line 978: insufficient samples in either drift window → score = 1.0 (documented in the docstring!)
  • I_replay = 1.0 placeholder ("no replay detection inputs in v0.5.x")
  • Q_deferral = 1.0 placeholder ("requires WBD tracking")
  • line 948: i_chain = 1.0 - gaps/max(total,1) — an 18-row chain with 0 gaps reads as perfect integrity

This is the same bug class the agent just fixed locally (a health probe defaulting absent evidence to healthy=True): absence of evidence is not evidence of perfection. A cold-start/thin corpus should produce an epistemically discounted score (or explicit Indeterminate, which lens-core's ManifoldConformity already models as CohortColdStart), never 1.0. Suggested shape, mirroring the agent fix: multiply a coverage/maturity term into the composite (evidence_count / target_count, capped <1 until the corpus matures), and replace placeholder-1.0 sub-terms with placeholder-EXCLUDED (renormalize) so unimplemented detectors don't inflate the score. A self-consistency check (fragility 0.999 + high_capacity should be impossible to emit together) would have caught this at the boundary.

2. Ask: occurrence-scoped capacity, gated on the CEG consent grant

Post-fold (CIRISAgent#866), consented traces flow CEG-native with per-occurrence agent_id_hash — the lens already enumerates (agent_id_hash, agent_name) pairs internally (api/scoring_api.py:241) but only exposes template aggregates (/capacity/{agent_name}) and fleet (/capacity/fleet).

Ask: GET /scoring/capacity/agent/{agent_id_hash} — THIS occurrence's externally-evaluated score, served only when the occurrence's consent:community_trust:v1 grant is in force (the CEG wire artifact written at opt-in; withdraws/recants = stop serving + the deletion cascade). Serving an occurrence-identified score is itself a privacy surface, so the consent gate applies to reads, not just ingest.

Agent-side consumption is designed and waiting: the /v1/my-data/capacity endpoint will add scope=occurrence alongside fleet/local — the trust story becomes: local self-vitals (epistemically capped at 0.95/0.60 by trust edge), occurrence cohort-grade score (external evaluation, consent-gated), fleet aggregate. cc CIRISAgent#866, CIRISLensCore#43.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions