From 08b39397e7a3b90577bbd50c1f97d9d59f7b952e Mon Sep 17 00:00:00 2001 From: Miley Chandonnet Date: Wed, 2 Sep 2026 20:31:34 -0500 Subject: [PATCH 1/2] AMPR-321 #735: ProbeEvent.VerdictReached; register 14 dark event types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add `ProbeEvent.VerdictReached` to `ampere-core`'s sealed `Event` hierarchy: `probeId`, `subjectId`, the four-valued `Verdict`, and a small free-form `detail` map. Primitives plus `Verdict` by construction, because a consumer's Probe may judge a subject type Ampere cannot name, so the subject itself never crosses the boundary. `ProbeSuite` gains an optional `eventBus` (plus `eventSource`, `now`, `idGenerator` for test pinning) and publishes one event per report, in probe order, after every Probe has run; left null, evaluation stays pure. `TraceRecorder` captures it with no change. Writing the tripwire the ticket asked for surfaced 14 already-declared events missing from `EventRegistry.allEventTypes`: all 6 `GitEvent`, all 4 `PlanEvent`, all 3 `BenchEvent`, and `RoutingEvent.RouteFloorUnmet`. Unregistered means invisible to `subscribeToAll`, the relay, and every recorded trace — a whole Git workflow and a whole bench run were dark. All 14 are now registered, and `EventRegistryCompletenessTest` walks the sealed hierarchy in both directions so the next omission fails there instead of appearing as a hole in a trace. Three exhaustive `when (event)` blocks now handle verdicts by shape: `SignificanceAwareEventLogger` and the CLI `EventCategorizer` treat `Holds` as routine and every other outcome as significant, and `EventRenderer` gives `Undetermined` its own colour. An `Undetermined` is never a soft pass, which is also what the new Oscilloscope rendering note in `docs/ampere/events.md` records. I wrote this commit; Miley reviewed it. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 39 +++++ .../watch/presentation/EventCategorizer.kt | 12 ++ .../socket/ampere/renderer/EventRenderer.kt | 10 ++ .../agents/domain/event/EventRegistry.kt | 27 ++++ .../ampere/agents/domain/event/ProbeEvent.kt | 74 ++++++++++ .../utils/SignificanceAwareEventLogger.kt | 12 ++ .../link/socket/ampere/probe/ProbeSuite.kt | 49 +++++- .../agents/domain/event/ProbeEventTest.kt | 127 ++++++++++++++++ .../ampere/probe/ProbeSuiteEventTest.kt | 138 +++++++++++++++++ .../event/EventRegistryCompletenessTest.kt | 139 ++++++++++++++++++ .../ampere/eval/trace/TraceRecorderTest.kt | 71 +++++++++ docs/ampere/events.md | 52 +++++++ 12 files changed, 748 insertions(+), 2 deletions(-) create mode 100644 ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/ProbeEvent.kt create mode 100644 ampere-core/src/commonTest/kotlin/link/socket/ampere/agents/domain/event/ProbeEventTest.kt create mode 100644 ampere-core/src/commonTest/kotlin/link/socket/ampere/probe/ProbeSuiteEventTest.kt create mode 100644 ampere-core/src/jvmTest/kotlin/link/socket/ampere/agents/domain/event/EventRegistryCompletenessTest.kt diff --git a/CHANGELOG.md b/CHANGELOG.md index a360d724..c3e7e7ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,19 @@ The project is pre-1.0; breaking changes are acceptable and explicitly called ou ### Fixed +- **14 event types were declared but never registered, making them invisible** + ([AMPR-321](https://linear.app/miley/issue/AMPR-321)). + + `EventRegistry.allEventTypes` is hand-maintained and is what + `EnvironmentService.subscribeToAll`, `EventRelayServiceImpl`, and + `TraceRecorder` enumerate. Every `GitEvent` (6), every `PlanEvent` (4), every + `BenchEvent` (3), and `RoutingEvent.RouteFloorUnmet` had been left out of it, + so they reached no subscriber that did not name their type explicitly and + appeared in no recorded trace — a whole Git workflow and a whole bench run + were dark. All 14 are now registered. `EventRegistryCompletenessTest` walks + the sealed `Event` hierarchy and fails on the next omission rather than + leaving it to be found as a hole in a trace. + - **`BatchIssueCreator` reported `success = true` on a cyclic batch** ([AMPR-322](https://linear.app/miley/issue/AMPR-322)). @@ -47,6 +60,32 @@ The project is pre-1.0; breaking changes are acceptable and explicitly called ou ### Added +- **`ProbeEvent.VerdictReached`, and an optional event bus on `ProbeSuite`** + ([AMPR-321](https://linear.app/miley/issue/AMPR-321)). + + A Probe's verdict was returned to its caller and nowhere else, so it could + not be read back from a trace. The closest existing event, + `BenchEvent.ProbeGraded`, carries `probeId`/`passed`/`meanScore` and no + subject id at all — "task T3 depends on T7, which is scheduled after it" was + unrecoverable. `ProbeEvent.VerdictReached` puts one Probe's judgement of one + identified subject on the `EventSerialBus`: `probeId`, `subjectId`, the + four-valued `Verdict`, and a small free-form `detail` map. It is + primitives-plus-`Verdict` by construction, because a consumer's Probe may + judge a subject type Ampere cannot name; the subject itself never crosses the + boundary. It lives beside `BenchEvent` in `ampere-core` for the same reason + that event does — `Event` is sealed, and Kotlin requires sealed subtypes to + share module and package with the base type. + + `ProbeSuite` gains an optional `eventBus` (plus `eventSource`, `now`, and + `idGenerator`, all manually injected as everywhere else in Ampere). When it + is set, `evaluate` publishes one event per report, in probe order, after + every Probe has run. Left null — the default — evaluation is pure and + nothing is published, so Bench fixtures and unit tests need no bus. + `TraceRecorder` captures the new event with no change, since it subscribes to + `EventRegistry.allEventTypes`. Rendering rules, including why an + `Undetermined` must never share a treatment with `Holds`, are in + `docs/ampere/events.md`. + - **`ampereSqliteOpenHelperFactory()` (Android)** ([AMPR-324](https://linear.app/miley/issue/AMPR-324)). diff --git a/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/cli/watch/presentation/EventCategorizer.kt b/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/cli/watch/presentation/EventCategorizer.kt index 54b4d857..92dd7339 100644 --- a/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/cli/watch/presentation/EventCategorizer.kt +++ b/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/cli/watch/presentation/EventCategorizer.kt @@ -17,6 +17,7 @@ import link.socket.ampere.agents.domain.event.MessageEvent import link.socket.ampere.agents.domain.event.NotificationEvent import link.socket.ampere.agents.domain.event.PlanEvent import link.socket.ampere.agents.domain.event.PermissionDeniedEvent +import link.socket.ampere.agents.domain.event.ProbeEvent import link.socket.ampere.agents.domain.event.ProviderCallCompletedEvent import link.socket.ampere.agents.domain.event.ProviderCallStartedEvent import link.socket.ampere.agents.domain.event.ProductEvent @@ -25,6 +26,7 @@ import link.socket.ampere.agents.domain.event.SparkEvent import link.socket.ampere.agents.domain.event.TaskEvent import link.socket.ampere.agents.domain.event.TicketEvent import link.socket.ampere.agents.domain.event.ToolEvent +import link.socket.ampere.probe.Verdict /** * Determines the significance of events for observation purposes. @@ -139,6 +141,16 @@ object EventCategorizer { // A rung floor with no satisfying model is a terminal routing failure: // the call cannot proceed, so it warrants immediate human awareness. is RoutingEvent.RouteFloorUnmet -> EventSignificance.CRITICAL + + // A Probe verdict: a clean pass is routine, every other outcome is a + // decision worth surfacing. An Undetermined is never a quiet pass. + is ProbeEvent.VerdictReached -> when (event.verdict) { + is Verdict.Holds -> EventSignificance.ROUTINE + is Verdict.Warn, + is Verdict.Violated, + is Verdict.Undetermined, + -> EventSignificance.SIGNIFICANT + } }.let { significance -> if (event is ProviderCallCompletedEvent && !event.success) { EventSignificance.SIGNIFICANT diff --git a/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/renderer/EventRenderer.kt b/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/renderer/EventRenderer.kt index 214bdb65..8fd15db9 100644 --- a/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/renderer/EventRenderer.kt +++ b/ampere-cli/src/jvmMain/kotlin/link/socket/ampere/renderer/EventRenderer.kt @@ -30,6 +30,7 @@ import link.socket.ampere.agents.domain.event.MessageEvent import link.socket.ampere.agents.domain.event.NotificationEvent import link.socket.ampere.agents.domain.event.PlanEvent import link.socket.ampere.agents.domain.event.PermissionDeniedEvent +import link.socket.ampere.agents.domain.event.ProbeEvent import link.socket.ampere.agents.domain.event.ProviderCallCompletedEvent import link.socket.ampere.agents.domain.event.ProviderCallStartedEvent import link.socket.ampere.agents.domain.event.ProductEvent @@ -41,6 +42,7 @@ import link.socket.ampere.agents.domain.event.SparkRemovedEvent import link.socket.ampere.agents.domain.event.TaskEvent import link.socket.ampere.agents.domain.event.TicketEvent import link.socket.ampere.agents.domain.event.ToolEvent +import link.socket.ampere.probe.Verdict /** * Renders events to terminal with color coding and formatting. @@ -186,6 +188,14 @@ class EventRenderer( is LinkEvent.LinkResolutionFailed -> "🔌" to red // Asset resolution: out-of-band, mirroring Link's icon family is AssetAccessEvent -> "🖼" to green + // Probe verdicts: the colour is the verdict. Undetermined is never + // green — it is not decided, and must not read as a pass. + is ProbeEvent.VerdictReached -> "⚖" to when (event.verdict) { + is Verdict.Holds -> green + is Verdict.Warn -> yellow + is Verdict.Violated -> red + is Verdict.Undetermined -> magenta + } } } diff --git a/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/EventRegistry.kt b/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/EventRegistry.kt index 232bc908..25ae3cfa 100644 --- a/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/EventRegistry.kt +++ b/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/EventRegistry.kt @@ -9,6 +9,10 @@ import link.socket.ampere.agents.domain.event.EventRegistry.allEventTypes * 1. Add it to the [allEventTypes] list * 2. It will automatically be available in EventTypeParser and EnvironmentService * + * The list is hand-maintained, so `EventRegistryCompletenessTest` walks the sealed + * [Event] hierarchy and fails when a subtype is missing from it. An event that is not + * here is invisible to `subscribeToAll`, the relay, and every recorded trace. + * * Benefits: * - Single place to maintain event type list * - Prevents inconsistencies between different parts of the system @@ -103,6 +107,29 @@ object EventRegistry { RoutingEvent.RouteSelected.EVENT_TYPE, RoutingEvent.RouteFallback.EVENT_TYPE, RoutingEvent.RouteResolved.EVENT_TYPE, + RoutingEvent.RouteFloorUnmet.EVENT_TYPE, + + // PlanEvent types + PlanEvent.PlanStepStarted.EVENT_TYPE, + PlanEvent.PlanStepCompleted.EVENT_TYPE, + PlanEvent.TaskAssigned.EVENT_TYPE, + PlanEvent.MonitoringStarted.EVENT_TYPE, + + // GitEvent types + GitEvent.BranchCreated.EVENT_TYPE, + GitEvent.Committed.EVENT_TYPE, + GitEvent.Pushed.EVENT_TYPE, + GitEvent.PullRequestCreated.EVENT_TYPE, + GitEvent.FilesStaged.EVENT_TYPE, + GitEvent.OperationFailed.EVENT_TYPE, + + // BenchEvent types + BenchEvent.BenchRunStarted.EVENT_TYPE, + BenchEvent.ProbeGraded.EVENT_TYPE, + BenchEvent.BenchRunCompleted.EVENT_TYPE, + + // ProbeEvent types + ProbeEvent.VerdictReached.EVENT_TYPE, // TelemetryEvent types ProviderCallStartedEvent.EVENT_TYPE, diff --git a/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/ProbeEvent.kt b/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/ProbeEvent.kt new file mode 100644 index 00000000..ee4157c7 --- /dev/null +++ b/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/ProbeEvent.kt @@ -0,0 +1,74 @@ +package link.socket.ampere.agents.domain.event + +import kotlinx.datetime.Instant +import kotlinx.serialization.Serializable +import link.socket.ampere.agents.domain.Urgency +import link.socket.ampere.probe.Probe +import link.socket.ampere.probe.ProbeId +import link.socket.ampere.probe.ProbeSuite +import link.socket.ampere.probe.Verdict + +/** + * Verdicts reached by a [Probe], carried on the `EventSerialBus` so a decision + * about a plan, a manifest, or a recalled fact is visible in the trace rather + * than only returned to whoever asked (AMPR-321). + * + * Lives alongside [BenchEvent] for the same reason it does: `Event` is a sealed + * interface, and Kotlin requires sealed subtypes to share both module and + * package with the base type. That constraint is also why the payload is + * primitives plus [Verdict] — a consumer's Probe may judge a subject type + * Ampere cannot name, so the subject itself never crosses this boundary. + */ +@Serializable +sealed interface ProbeEvent : Event { + + /** The Probe that reached the verdict. */ + val probeId: ProbeId + + /** + * One Probe reached a verdict on one identified subject. + * + * [subjectId] is caller-supplied (see [ProbeSuite]) because a Probe's + * subject type is unconstrained and the SPI cannot ask a subject for its + * own identity. [detail] is free-form key/value for the Oscilloscope; keep + * it small — it is stored in every trace that captures this event. + */ + @Serializable + data class VerdictReached( + override val eventId: EventId, + override val eventSource: EventSource, + override val timestamp: Instant, + override val probeId: ProbeId, + val subjectId: String, + val verdict: Verdict, + val detail: Map = emptyMap(), + override val urgency: Urgency = Urgency.LOW, + ) : ProbeEvent { + + override val eventType: EventType = EVENT_TYPE + + override fun getSummary( + formatUrgency: (Urgency) -> String, + formatSource: (EventSource) -> String, + ): String = buildString { + append("Probe ${probeId.value} on $subjectId: ${verdict.label()}") + verdict.reason?.let { append(" — $it") } + append(" ${formatUrgency(urgency)}") + } + + companion object { + const val EVENT_TYPE: EventType = "VerdictReached" + } + } +} + +/** + * Short, stable name for a [Verdict] in a summary line. `Undetermined` reads as + * itself and never as a soft pass — the whole point of the fourth value. + */ +private fun Verdict.label(): String = when (this) { + is Verdict.Holds -> "holds" + is Verdict.Warn -> "warn" + is Verdict.Violated -> "violated" + is Verdict.Undetermined -> "undetermined(${cause.name})" +} diff --git a/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/events/utils/SignificanceAwareEventLogger.kt b/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/events/utils/SignificanceAwareEventLogger.kt index 38ff4e2b..53eddab8 100644 --- a/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/events/utils/SignificanceAwareEventLogger.kt +++ b/ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/events/utils/SignificanceAwareEventLogger.kt @@ -21,6 +21,7 @@ import link.socket.ampere.agents.domain.event.MessageEvent import link.socket.ampere.agents.domain.event.NotificationEvent import link.socket.ampere.agents.domain.event.PermissionDeniedEvent import link.socket.ampere.agents.domain.event.PlanEvent +import link.socket.ampere.agents.domain.event.ProbeEvent import link.socket.ampere.agents.domain.event.ProductEvent import link.socket.ampere.agents.domain.event.ProviderCallCompletedEvent import link.socket.ampere.agents.domain.event.ProviderCallStartedEvent @@ -30,6 +31,7 @@ import link.socket.ampere.agents.domain.event.TaskEvent import link.socket.ampere.agents.domain.event.TicketEvent import link.socket.ampere.agents.domain.event.ToolEvent import link.socket.ampere.agents.events.subscription.Subscription +import link.socket.ampere.probe.Verdict /** * Filters events based on significance and displays rich event details. @@ -187,6 +189,16 @@ class SignificanceAwareEventLogger( is BenchEvent.ProbeGraded -> EventSignificance.ROUTINE is BenchEvent.BenchRunCompleted -> EventSignificance.SIGNIFICANT + // Probe verdicts - a clean pass is routine; anything else is a decision + // a human may need to see. An Undetermined is never a quiet pass. + is ProbeEvent.VerdictReached -> when (event.verdict) { + is Verdict.Holds -> EventSignificance.ROUTINE + is Verdict.Warn, + is Verdict.Violated, + is Verdict.Undetermined, + -> EventSignificance.SIGNIFICANT + } + // Link lifecycle - resolution is routine, but anything that changes or // denies a Plug's access to a wire is a consent-visible fact. is LinkEvent.LinkResolved -> EventSignificance.ROUTINE diff --git a/ampere-core/src/commonMain/kotlin/link/socket/ampere/probe/ProbeSuite.kt b/ampere-core/src/commonMain/kotlin/link/socket/ampere/probe/ProbeSuite.kt index 03fa38fb..6c4e0675 100644 --- a/ampere-core/src/commonMain/kotlin/link/socket/ampere/probe/ProbeSuite.kt +++ b/ampere-core/src/commonMain/kotlin/link/socket/ampere/probe/ProbeSuite.kt @@ -1,21 +1,66 @@ package link.socket.ampere.probe +import kotlinx.datetime.Clock +import kotlinx.datetime.Instant +import link.socket.ampere.agents.domain.event.EventSource +import link.socket.ampere.agents.domain.event.ProbeEvent +import link.socket.ampere.agents.events.bus.EventSerialBus +import link.socket.ampere.agents.events.utils.generateUUID + /** * Runs an ordered list of Probes over one subject. * * The caller supplies [evaluate]'s `subjectId` because [S] is unconstrained * and the SPI cannot ask the subject for its own identity. + * + * Pass an [eventBus] to make the verdicts visible in the trace: [evaluate] then + * publishes one [ProbeEvent.VerdictReached] per report, in probe order, after + * every Probe has run. Left null — the default for Bench fixtures and unit + * tests — evaluation is pure and nothing is published. + * + * Wiring is manual, as everywhere else in Ampere: [eventSource], [now], and + * [idGenerator] are constructor parameters so a test can pin what a published + * event carries. */ class ProbeSuite( private val probes: List>, + private val eventBus: EventSerialBus? = null, + private val eventSource: EventSource = EventSource.Agent(DEFAULT_SOURCE_ID), + private val now: () -> Instant = { Clock.System.now() }, + private val idGenerator: () -> String = { generateUUID() }, ) { - suspend fun evaluate(subjectId: String, subject: S): List = - probes.map { probe -> + suspend fun evaluate(subjectId: String, subject: S): List { + val reports = probes.map { probe -> ProbeReport( probeId = probe.id, subjectId = subjectId, verdict = probe.evaluate(subject), ) } + + // Published after the whole suite runs, so a subscriber never sees a + // partial verdict set from a suite that threw halfway through. + eventBus?.let { bus -> + reports.forEach { report -> + bus.publish( + ProbeEvent.VerdictReached( + eventId = idGenerator(), + eventSource = eventSource, + timestamp = now(), + probeId = report.probeId, + subjectId = report.subjectId, + verdict = report.verdict, + ), + ) + } + } + + return reports + } + + companion object { + /** Attribution for verdicts published by a suite the caller did not name. */ + const val DEFAULT_SOURCE_ID: String = "ampere.probe-suite" + } } diff --git a/ampere-core/src/commonTest/kotlin/link/socket/ampere/agents/domain/event/ProbeEventTest.kt b/ampere-core/src/commonTest/kotlin/link/socket/ampere/agents/domain/event/ProbeEventTest.kt new file mode 100644 index 00000000..cb703ac2 --- /dev/null +++ b/ampere-core/src/commonTest/kotlin/link/socket/ampere/agents/domain/event/ProbeEventTest.kt @@ -0,0 +1,127 @@ +package link.socket.ampere.agents.domain.event + +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertIs +import kotlin.test.assertTrue +import kotlinx.datetime.Instant +import kotlinx.serialization.json.Json +import link.socket.ampere.agents.domain.Urgency +import link.socket.ampere.probe.ProbeId +import link.socket.ampere.probe.UndeterminedCause +import link.socket.ampere.probe.Verdict + +/** + * Serialization samples for [ProbeEvent.VerdictReached] (AMPR-321 task 2). + * + * A verdict is only useful in a trace if it survives the round-trip whole: + * `subjectId` says *what* was judged, and the [Verdict] subtype says how. Both + * are pinned here, across all four verdict values. + */ +class ProbeEventTest { + + private val json = Json { + prettyPrint = false + encodeDefaults = true + classDiscriminator = "type" + ignoreUnknownKeys = true + } + + private val timestamp = Instant.fromEpochMilliseconds(1_700_000_000_000) + + private fun verdictReached( + verdict: Verdict, + subjectId: String = "plan-7", + detail: Map = emptyMap(), + ): ProbeEvent.VerdictReached = ProbeEvent.VerdictReached( + eventId = "11111111-1111-1111-1111-111111111111", + eventSource = EventSource.Agent("agent-planner"), + timestamp = timestamp, + probeId = ProbeId("ampere.sequence"), + subjectId = subjectId, + verdict = verdict, + detail = detail, + ) + + private fun verdictSamples(): List = listOf( + Verdict.Holds(), + Verdict.Warn(reason = "T3 is scheduled tight against T7"), + Verdict.Violated(reason = "cycle: T3 -> T7 -> T3"), + Verdict.Undetermined( + reason = "no schedule on T7", + cause = UndeterminedCause.EVIDENCE_ABSENT, + ), + ) + + @Test + fun `every verdict survives the polymorphic Event round-trip`() { + verdictSamples().forEach { verdict -> + val original: Event = verdictReached(verdict) + + val encoded = json.encodeToString(Event.serializer(), original) + val decoded = json.decodeFromString(Event.serializer(), encoded) + + assertIs(decoded) + assertEquals("plan-7", decoded.subjectId, "subjectId was lost for $verdict") + assertEquals(verdict, decoded.verdict, "verdict changed shape") + assertEquals(original, decoded) + } + } + + @Test + fun `detail round-trips and defaults to empty`() { + val withDetail = verdictReached( + verdict = Verdict.Violated(reason = "cycle"), + detail = mapOf("edge" to "T3 -> T7"), + ) + + val decoded = json.decodeFromString( + Event.serializer(), + json.encodeToString(Event.serializer(), withDetail), + ) + + assertIs(decoded) + assertEquals(mapOf("edge" to "T3 -> T7"), decoded.detail) + assertEquals(emptyMap(), verdictReached(Verdict.Holds()).detail) + } + + /** + * Both names are frozen the moment a trace is written: the class name is the + * polymorphic discriminator a recorded event decodes through, and `EVENT_TYPE` + * is what subscribers and [EventRegistry] address. Renaming either makes older + * traces undecodable — the reason `BenchEvent.ProbeGraded` still has its name. + */ + @Test + fun `the class discriminator and event type are pinned`() { + val encoded = json.encodeToString(Event.serializer(), verdictReached(Verdict.Holds())) + + assertTrue( + encoded.contains("ProbeEvent.VerdictReached"), + "expected the pinned class discriminator; got $encoded", + ) + assertEquals("VerdictReached", ProbeEvent.VerdictReached.EVENT_TYPE) + assertTrue(ProbeEvent.VerdictReached.EVENT_TYPE in EventRegistry.allEventTypes) + } + + @Test + fun `an undetermined verdict never reads as a pass in the summary`() { + val summary = verdictReached( + verdict = Verdict.Undetermined( + reason = "no schedule on T7", + cause = UndeterminedCause.EVIDENCE_ABSENT, + ), + ).getSummary( + formatUrgency = { "[${it.name}]" }, + formatSource = { it.getIdentifier() }, + ) + + assertTrue(summary.contains("undetermined(EVIDENCE_ABSENT)"), summary) + assertTrue(summary.contains("plan-7"), summary) + assertTrue(!summary.contains("holds"), summary) + } + + @Test + fun `urgency defaults to low`() { + assertEquals(Urgency.LOW, verdictReached(Verdict.Violated(reason = "cycle")).urgency) + } +} diff --git a/ampere-core/src/commonTest/kotlin/link/socket/ampere/probe/ProbeSuiteEventTest.kt b/ampere-core/src/commonTest/kotlin/link/socket/ampere/probe/ProbeSuiteEventTest.kt new file mode 100644 index 00000000..ce03d82d --- /dev/null +++ b/ampere-core/src/commonTest/kotlin/link/socket/ampere/probe/ProbeSuiteEventTest.kt @@ -0,0 +1,138 @@ +package link.socket.ampere.probe + +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue +import kotlinx.coroutines.ExperimentalCoroutinesApi +import kotlinx.coroutines.test.runCurrent +import kotlinx.coroutines.test.runTest +import kotlinx.datetime.Instant +import link.socket.ampere.agents.domain.event.EventSource +import link.socket.ampere.agents.domain.event.ProbeEvent +import link.socket.ampere.agents.events.api.EventHandler +import link.socket.ampere.agents.events.bus.EventSerialBus + +/** + * AMPR-321 task 3 validation: a suite handed a bus makes its verdicts visible in + * the trace, and a suite without one stays pure. + */ +@OptIn(ExperimentalCoroutinesApi::class) +class ProbeSuiteEventTest { + + private val holdsProbe = object : Probe { + override val id = ProbeId("holds") + + override suspend fun evaluate(subject: String): Verdict = Verdict.Holds() + } + + private val violatesProbe = object : Probe { + override val id = ProbeId("violates") + + override suspend fun evaluate(subject: String): Verdict = + Verdict.Violated(reason = "cycle: T3 -> T7 -> T3") + } + + private val undeterminedProbe = object : Probe { + override val id = ProbeId("undetermined") + + override suspend fun evaluate(subject: String): Verdict = Verdict.Undetermined( + reason = "T7 publishes no schedule", + cause = UndeterminedCause.EVIDENCE_ABSENT, + ) + } + + private val timestamp = Instant.fromEpochMilliseconds(1_700_000_000_000) + + private fun suite(bus: EventSerialBus?): ProbeSuite = ProbeSuite( + probes = listOf(holdsProbe, violatesProbe, undeterminedProbe), + eventBus = bus, + eventSource = EventSource.Agent("agent-planner"), + now = { timestamp }, + idGenerator = { "evt-fixed" }, + ) + + @Test + fun `evaluate publishes one verdict event per probe in probe order`() = runTest { + val bus = EventSerialBus(scope = backgroundScope) + val received = mutableListOf() + bus.subscribeSuspending( + agentId = "oscilloscope", + eventType = ProbeEvent.VerdictReached.EVENT_TYPE, + handler = EventHandler { event, _ -> received += event as ProbeEvent.VerdictReached }, + ) + + val reports = suite(bus).evaluate(subjectId = "plan-7", subject = "a plan graph") + runCurrent() + + assertEquals(3, received.size) + assertEquals( + listOf(ProbeId("holds"), ProbeId("violates"), ProbeId("undetermined")), + received.map { it.probeId }, + ) + assertEquals(reports.map { it.verdict }, received.map { it.verdict }) + assertTrue(received.all { it.subjectId == "plan-7" }) + assertTrue(received.all { it.timestamp == timestamp }) + assertEquals(EventSource.Agent("agent-planner"), received.first().eventSource) + } + + @Test + fun `a suite with no bus publishes nothing and still reports`() = runTest { + val bus = EventSerialBus(scope = backgroundScope) + val received = mutableListOf() + bus.subscribeSuspending( + agentId = "oscilloscope", + eventType = ProbeEvent.VerdictReached.EVENT_TYPE, + handler = EventHandler { event, _ -> received += event as ProbeEvent.VerdictReached }, + ) + + val reports = suite(bus = null).evaluate(subjectId = "plan-7", subject = "a plan graph") + runCurrent() + + assertEquals(3, reports.size) + assertTrue(received.isEmpty()) + } + + @Test + fun `an undetermined verdict is published as itself and not as a pass`() = runTest { + val bus = EventSerialBus(scope = backgroundScope) + val received = mutableListOf() + bus.subscribeSuspending( + agentId = "oscilloscope", + eventType = ProbeEvent.VerdictReached.EVENT_TYPE, + handler = EventHandler { event, _ -> received += event as ProbeEvent.VerdictReached }, + ) + + ProbeSuite(probes = listOf(undeterminedProbe), eventBus = bus) + .evaluate(subjectId = "plan-7", subject = "a plan graph") + runCurrent() + + assertEquals( + Verdict.Undetermined( + reason = "T7 publishes no schedule", + cause = UndeterminedCause.EVIDENCE_ABSENT, + ), + received.single().verdict, + ) + } + + @Test + fun `each published verdict carries its own event id`() = runTest { + val bus = EventSerialBus(scope = backgroundScope) + val received = mutableListOf() + bus.subscribeSuspending( + agentId = "oscilloscope", + eventType = ProbeEvent.VerdictReached.EVENT_TYPE, + handler = EventHandler { event, _ -> received += event as ProbeEvent.VerdictReached }, + ) + + var next = 0 + ProbeSuite( + probes = listOf(holdsProbe, violatesProbe), + eventBus = bus, + idGenerator = { "evt-${next++}" }, + ).evaluate(subjectId = "plan-7", subject = "a plan graph") + runCurrent() + + assertEquals(listOf("evt-0", "evt-1"), received.map { it.eventId }) + } +} diff --git a/ampere-core/src/jvmTest/kotlin/link/socket/ampere/agents/domain/event/EventRegistryCompletenessTest.kt b/ampere-core/src/jvmTest/kotlin/link/socket/ampere/agents/domain/event/EventRegistryCompletenessTest.kt new file mode 100644 index 00000000..356821a8 --- /dev/null +++ b/ampere-core/src/jvmTest/kotlin/link/socket/ampere/agents/domain/event/EventRegistryCompletenessTest.kt @@ -0,0 +1,139 @@ +package link.socket.ampere.agents.domain.event + +import java.lang.reflect.Modifier +import kotlin.reflect.KClass +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * The tripwire for `EventRegistry.allEventTypes` (AMPR-321 task 1). + * + * That list is hand-maintained, and an event missing from it is silently + * invisible: `EnvironmentService.subscribeToAll`, `EventRelayServiceImpl`, and + * `TraceRecorder` all enumerate it, so an unregistered event reaches no + * subscriber that did not name its type and appears in no recorded trace. This + * test walks the sealed [Event] hierarchy instead of trusting the list, so the + * next omission fails here rather than as a hole in a trace. + * + * JVM-only because it needs `sealedSubclasses`; the hierarchy it checks is + * declared in `commonMain`, so covering it once on one target is enough. + */ +class EventRegistryCompletenessTest { + + /** + * Concrete events that no walk of the hierarchy can reach. [SparkEvent] is a + * plain interface, not a sealed one — its implementations are top-level + * classes in the same file — and reflection cannot enumerate the implementors + * of an open interface. `SparkEvent is the only Event branch a walk cannot + * enumerate` below fails if a second such branch appears, which is what keeps + * this hand-written list from going quietly stale. + */ + private val openBranchEvents: List> = listOf( + SparkAppliedEvent::class, + SparkRemovedEvent::class, + CognitiveStateSnapshot::class, + ) + + @Test + fun `every sealed Event subtype is registered`() { + val missing = declaredEventTypes() + .filterValues { it !in EventRegistry.allEventTypes } + .map { (klass, eventType) -> "${klass.name} ($eventType)" } + .sorted() + + assertTrue( + missing.isEmpty(), + "Event subtypes missing from EventRegistry.allEventTypes — they are invisible to " + + "subscribeToAll, the relay, and every recorded trace:\n${missing.joinToString("\n")}", + ) + } + + @Test + fun `every registered event type is declared by an Event subtype`() { + val declared = declaredEventTypes().values.toSet() + val unknown = EventRegistry.allEventTypes.filterNot { it in declared }.sorted() + + assertTrue( + unknown.isEmpty(), + "EventRegistry.allEventTypes names types no Event subtype declares:\n${unknown.joinToString("\n")}", + ) + } + + @Test + fun `no event type is registered twice`() { + val duplicates = EventRegistry.allEventTypes + .groupingBy { it } + .eachCount() + .filterValues { it > 1 } + .keys + .sorted() + + assertTrue(duplicates.isEmpty(), "Duplicate entries in allEventTypes: $duplicates") + } + + @Test + fun `every concrete Event subtype declares an EVENT_TYPE constant`() { + val withoutConstant = concreteEventClasses() + .filter { it.eventTypeConstant() == null } + .map { it.name } + .sorted() + + assertTrue( + withoutConstant.isEmpty(), + "Event subtypes with no EVENT_TYPE constant on themselves or a supertype, which this " + + "tripwire cannot check and EventRegistry cannot name:\n${withoutConstant.joinToString("\n")}", + ) + } + + /** + * A non-sealed interface under [Event] is a hole in the hierarchy: its + * implementors can live anywhere, so nothing — not this test, not the + * compiler — can enumerate them. One exists ([SparkEvent]) and its three + * implementations are listed in [openBranchEvents]. A second one has to be + * listed there too, or its events go unchecked. + */ + @Test + fun `SparkEvent is the only Event branch a walk cannot enumerate`() { + val openBranches = Event::class.sealedSubclasses + .filter { it.java.isInterface && !it.isSealed } + .map { it.java.name } + + assertEquals(listOf(SparkEvent::class.java.name), openBranches) + } + + @Test + fun `the probe verdict event is registered`() { + assertTrue( + ProbeEvent.VerdictReached.EVENT_TYPE in EventRegistry.allEventTypes, + "VerdictReached must be registered for a trace to capture it", + ) + } + + /** Concrete [Event] classes mapped to the `EVENT_TYPE` each one resolves. */ + private fun declaredEventTypes(): Map, EventType> = + concreteEventClasses().mapNotNull { klass -> + klass.eventTypeConstant()?.let { klass to it } + }.toMap() + + /** Every instantiable event in the hierarchy, plus the branch a walk cannot reach. */ + private fun concreteEventClasses(): List> = + (sealedLeaves(Event::class) + openBranchEvents) + .map { it.java } + .distinct() + .filterNot { it.isInterface || Modifier.isAbstract(it.modifiers) } + + /** Every non-sealed class in the hierarchy rooted at [root], depth-first. */ + private fun sealedLeaves(root: KClass<*>): List> = + if (root.isSealed) root.sealedSubclasses.flatMap(::sealedLeaves) else listOf(root) + + /** + * Reads the `EVENT_TYPE` constant, which a `const val` puts in a static field + * on its declaring class. It is not always on the event itself: + * `EmissionEvent.BaseProduced` takes its type from the `EmissionEvent.Produced` + * interface it implements, so supertypes are searched too. + */ + private fun Class<*>.eventTypeConstant(): EventType? = + declaredFields.firstOrNull { it.name == "EVENT_TYPE" }?.get(null) as? EventType + ?: (interfaces.asList() + listOfNotNull(superclass)).firstNotNullOfOrNull { it.eventTypeConstant() } +} diff --git a/ampere-eval/src/jvmTest/kotlin/link/socket/ampere/eval/trace/TraceRecorderTest.kt b/ampere-eval/src/jvmTest/kotlin/link/socket/ampere/eval/trace/TraceRecorderTest.kt index 67ed57c2..eeec43ed 100644 --- a/ampere-eval/src/jvmTest/kotlin/link/socket/ampere/eval/trace/TraceRecorderTest.kt +++ b/ampere-eval/src/jvmTest/kotlin/link/socket/ampere/eval/trace/TraceRecorderTest.kt @@ -17,10 +17,15 @@ import kotlinx.serialization.json.jsonPrimitive import link.socket.ampere.agents.domain.Urgency import link.socket.ampere.agents.domain.event.AssetAccessEvent import link.socket.ampere.agents.domain.event.Event +import link.socket.ampere.agents.domain.event.EventRegistry import link.socket.ampere.agents.domain.event.EventSource +import link.socket.ampere.agents.domain.event.ProbeEvent import link.socket.ampere.agents.events.bus.EventSerialBus import link.socket.ampere.data.DEFAULT_JSON import link.socket.ampere.eval.db.EvalDatabase +import link.socket.ampere.probe.ProbeId +import link.socket.ampere.probe.UndeterminedCause +import link.socket.ampere.probe.Verdict /** AMPR-183 task 1.4 validation + record -> persist -> load -> replay round-trip. */ @OptIn(ExperimentalCoroutinesApi::class) @@ -173,6 +178,72 @@ class TraceRecorderTest { ) } + /** + * AMPR-321 task 2: the recorder needs no change to capture a new event — it + * subscribes to [EventRegistry.allEventTypes], so registering `VerdictReached` + * is the whole wiring. What matters is that the verdict survives the trip + * through the trace: `subjectId` says what was judged, the [Verdict] subtype + * says how, and neither is recoverable from `BenchEvent.ProbeGraded`. + */ + @Test + fun `a probe verdict survives recording and decoding from a trace`() = runTest { + val handle = recorder.start(runId = "run-6", arcId = "arc-6") + + bus.publish( + ProbeEvent.VerdictReached( + eventId = "e1", + eventSource = source, + timestamp = Instant.fromEpochMilliseconds(1), + probeId = ProbeId("ampere.sequence"), + subjectId = "plan-7", + verdict = Verdict.Violated(reason = "cycle: T3 -> T7 -> T3"), + detail = mapOf("edge" to "T3 -> T7"), + ), + ) + + val trace = handle.stop().getOrThrow() + + assertEquals(1, trace.size) + val decoded = DEFAULT_JSON.decodeFromJsonElement( + Event.serializer(), + trace.events.single().payload, + ) as ProbeEvent.VerdictReached + + assertEquals("plan-7", decoded.subjectId) + assertEquals(Verdict.Violated(reason = "cycle: T3 -> T7 -> T3"), decoded.verdict) + assertEquals(ProbeId("ampere.sequence"), decoded.probeId) + assertEquals(mapOf("edge" to "T3 -> T7"), decoded.detail) + } + + @Test + fun `an undetermined verdict keeps its cause through a trace`() = runTest { + val handle = recorder.start(runId = "run-7", arcId = "arc-7") + + bus.publish( + ProbeEvent.VerdictReached( + eventId = "e1", + eventSource = source, + timestamp = Instant.fromEpochMilliseconds(1), + probeId = ProbeId("ampere.sequence"), + subjectId = "plan-8", + verdict = Verdict.Undetermined( + reason = "T7 publishes no schedule", + cause = UndeterminedCause.EVIDENCE_ABSENT, + ), + ), + ) + + val trace = handle.stop().getOrThrow() + val decoded = DEFAULT_JSON.decodeFromJsonElement( + Event.serializer(), + trace.events.single().payload, + ) as ProbeEvent.VerdictReached + + // An Undetermined must never flatten into a soft pass on the way out. + val verdict = decoded.verdict as Verdict.Undetermined + assertEquals(UndeterminedCause.EVIDENCE_ABSENT, verdict.cause) + } + private fun kotlinx.serialization.json.JsonElement.eventId(): String = DEFAULT_JSON.decodeFromJsonElement(Event.serializer(), this).eventId } diff --git a/docs/ampere/events.md b/docs/ampere/events.md index 3aabd7b1..4a427e85 100644 --- a/docs/ampere/events.md +++ b/docs/ampere/events.md @@ -94,3 +94,55 @@ on separate event types cannot rely on cross-event ordering at the handler level. A consumer that needs both signals together should subscribe to `EscalationConsidered` (which carries enough payload to act on its own) rather than correlating the two events. + +## Probe verdicts + +`ProbeEvent.VerdictReached` carries one Probe's judgement of one subject onto the +bus, so a decision is visible in the trace rather than only returned to whoever +asked. It is Ampere-owned and primitives-only: the subject itself never crosses +the boundary, because a consumer's Probe may judge a type Ampere cannot name. + +| Field | Meaning | +| --- | --- | +| `probeId` | The Probe that reached the verdict. | +| `subjectId` | Caller-supplied identity of what was judged — a Probe's subject type is unconstrained, so the SPI cannot ask a subject for its own id. | +| `verdict` | `Holds`, `Warn`, `Violated`, or `Undetermined`, each carrying its `reason`. | +| `detail` | Free-form key/value for the Oscilloscope. Keep it small; it is stored in every trace that captures the event. | + +Publishing is opt-in. A `ProbeSuite` constructed with an `eventBus` publishes one +event per report, in probe order, after every Probe in the suite has run: + +```kotlin +val suite = ProbeSuite( + probes = listOf(SequenceProbe()), + eventBus = bus, +) + +// Returns the reports as before, and puts each verdict on the bus. +val reports = suite.evaluate(subjectId = "plan-7", subject = workGraph) +``` + +Leave `eventBus` null — the default — and evaluation stays pure. Bench fixtures +and unit tests need no bus. + +### Rendering a verdict in the Oscilloscope + +A verdict event renders with its `subjectId`, not just its `probeId`. "Task T3 +depends on T7, which is scheduled after it" is only legible if the row names T3; +a stream of `ampere.sequence` rows with no subject is a stream of unattributable +judgements. + +The four verdicts render as **four** states, never as pass/fail with decoration: + +| Verdict | Reads as | Rendering | +| --- | --- | --- | +| `Holds` | decided, good | Neutral. A clean pass needs no explanation and `reason` is often null. | +| `Warn` | decided, bad, not disqualifying | Signal Amber. Shows `reason`. | +| `Violated` | decided, disqualifying | Distinct from `Warn`, and never collapsed into it. Shows `reason`. | +| `Undetermined` | **not decided** | Visually distinct from `Holds`. Shows `reason` *and* `cause`, because "no published spec" and "the page needed a JS engine" lead to different next actions. | + +`Undetermined` is the one that gets rendering wrong most easily. It is not a soft +pass and must never share a treatment with `Holds`: the Probe convicts but does +not acquit, and a viewer who reads an `Undetermined` row as "fine" has been told +the opposite of what happened. Routing on a verdict — re-plan, escalate to a +human — stays on the consumer side; the event is the signal, not the action. From 1a358d7761a6617fd26f629290fb915fe17e74fc Mon Sep 17 00:00:00 2001 From: Miley Chandonnet Date: Wed, 2 Sep 2026 20:37:56 -0500 Subject: [PATCH 2/2] AMPR-321 #735: update Probe and EventSerialBus concepts for verdict events MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The merge brought in `docs/concepts/probe.md` (AMPR-323), and this branch touches `tracked_sources` for it and for `EventSerialBus`, so both are updated in this diff rather than a follow-up. `Probe` gains the bus on `ProbeSuite` and three invariants the code now enforces: a verdict event carries the subject's *id* and never the subject, publishing is opt-in and never partial, and a verdict is recomputed rather than replayed. Its "a Probe is not a trace grader" invariant is narrowed — `VerdictReached` puts the verdict *in* the trace even though it was never computable *from* one. `EventSerialBus` records that registration means an entry in `EventRegistry.allEventTypes`, names the tripwire that now guards it, and qualifies the `AgentEventApi` anti-pattern: a publisher that is genuinely not agent-owned takes an explicit `eventSource`, and taking the bus without taking a source is the real mistake. I wrote this commit; Miley reviewed it. Co-Authored-By: Claude Opus 5 (1M context) --- docs/concepts/event-serial-bus.md | 8 +++++--- docs/concepts/probe.md | 16 +++++++++++++--- 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/docs/concepts/event-serial-bus.md b/docs/concepts/event-serial-bus.md index 667dce22..9f37f777 100644 --- a/docs/concepts/event-serial-bus.md +++ b/docs/concepts/event-serial-bus.md @@ -11,7 +11,7 @@ tracked_sources: - ampere-core/src/commonMain/kotlin/link/socket/ampere/agents/domain/event/** - ampere-core/src/commonMain/sqldelight/link/socket/ampere/db/events/** related: [PropelLoop, AgentSurface, CognitionTrace, MemoryProvenance, LinkLayer] -last_verified: 2026-07-28 +last_verified: 2026-09-02 --- # EventSerialBus @@ -61,12 +61,14 @@ properties for free: - `agents/domain/event/Event.kt` and the `event/` package — the sealed `Event` hierarchy. - `agents/domain/event/CognitivePhaseEvent.kt` — phase transition events emitted by `PhaseSparkManager` when a bus is wired. - `agents/domain/event/LinkEvent.kt` — Link lifecycle (granted, revoked, resolved, resolution failed); see [LinkLayer](link-layer.md). +- `agents/domain/event/ProbeEvent.kt` — `VerdictReached`, one Probe's judgement of one identified subject; see [Probe](probe.md). +- `agents/domain/event/EventRegistry.kt` — the hand-maintained list of every event type, and the only thing `subscribeToAll`, the relay, and `TraceRecorder` enumerate. - `ampere-core/src/commonMain/sqldelight/link/socket/ampere/db/events/EventStore.sq` — persistence schema (with `run_id` indexes for trace queries). ## Invariants - **Direct agent-to-agent method calls are forbidden for coordination.** If agent A needs to influence agent B, A publishes; B subscribes. The only direct calls allowed are within an agent's own services or into stateless helpers. (Read: tests around `AgentReasoning` injecting fakes are fine; an agent calling another agent's `handleX(...)` is not.) -- **Every event type has a serializer, a registration in the event hierarchy, and a CLI display handler.** New event types must satisfy all three before merge — see the "Agent System Rules" in `AGENTS.md`. +- **Every event type has a serializer, a registration in the event hierarchy, and a CLI display handler.** New event types must satisfy all three before merge — see the "Agent System Rules" in `AGENTS.md`. Registration means an entry in `EventRegistry.allEventTypes`: an event missing from that list reaches no subscriber that did not name its type and appears in no recorded trace. Fourteen declared events had drifted out of it (AMPR-321); `EventRegistryCompletenessTest` now walks the sealed hierarchy in both directions so the next omission fails there. - **Handler exceptions never propagate to the publisher.** The bus swallows and logs handler failures. Publishers cannot rely on subscriber success; if a downstream effect is required, it gets its own event. - **The bus does not persist; loggers and stores do.** A change that makes `EventSerialBus.publish` write to a database directly violates the layering — persistence belongs to `EventStore` invoked by an event-aware logger or projector. - **`run_id` is propagated through the event chain.** Events emitted within an Arc run carry the originating `run_id` so trace projection can find them. Lossy event handlers that strip `run_id` break time-travel. @@ -92,7 +94,7 @@ properties for free: - **"Just call the other agent's method directly, the event is annoying."** This is how AMPERE became opaque the first time. The cost of an event is one serialized struct; the cost of bypassing one is invisibility. - **Catching exceptions inside a handler and silently dropping them.** The bus already swallows handler errors and logs them. Adding a second swallow inside the handler hides real failures from the logger. - **Using `runBlocking` inside a handler.** Handlers run on the bus's `CoroutineScope`. Blocking that scope blocks the next dispatch loop. Suspend functions only. -- **Emitting events outside an agent's `AgentEventApi`.** Direct `bus.publish` calls in domain code skip the source-tagging the api adds, which means the trace can't attribute the event to an agent. +- **Emitting events outside an agent's `AgentEventApi`.** Direct `bus.publish` calls in domain code skip the source-tagging the api adds, which means the trace can't attribute the event to an agent. A publisher that is genuinely not agent-owned — `ProbeSuite`, which a consumer may drive with no agent in sight — takes an explicit `eventSource` instead, so attribution is carried rather than lost. Taking the bus without taking a source is the actual anti-pattern. - **Persisting state in the bus.** The bus is a router. Anything that needs persistence belongs in a store one layer up. - **Unsubscribing by event type from a shared consumer.** It reads like "stop listening" and behaves like "nobody listens." Use the `Subscription` handle unless you are certain you are the only subscriber on that type, and say so in a comment if you are. - **Exposing `subscribe` across the FFI boundary.** It calls `runBlockingCompat`, so a Swift call from the main thread blocks the UI and can deadlock on Kotlin/Native. Swift gets a Flow or a callback facade, never the bus. diff --git a/docs/concepts/probe.md b/docs/concepts/probe.md index 365b5679..88908982 100644 --- a/docs/concepts/probe.md +++ b/docs/concepts/probe.md @@ -16,7 +16,10 @@ manifest, a recalled fact — that returns a four-valued `Verdict`: `Holds`, `Warn` (decided, bad, not disqualifying), `Violated` (decided, disqualifying), or `Undetermined` (not decided; carries an `UndeterminedCause`). A `ProbeSuite` runs an ordered list over one subject and yields `ProbeReport`s; -a `ProbeRegistry` lists Probes for discovery (Oscilloscope), not dispatch. +a `ProbeRegistry` lists Probes for discovery (Oscilloscope), not dispatch. A +suite handed an `EventSerialBus` also publishes one +`ProbeEvent.VerdictReached` per report, so the verdict is legible in the trace +and not only to whoever called `evaluate`. `SequenceProbe : Probe` and `FreshnessProbe : Probe` are the shipped Probes. `Observed` is the one-field interface (`observedAt: Instant`) that lets it run over a canon @@ -44,7 +47,8 @@ type would leave foreign subjects with no base to extend. - `probe/SequenceProbe.kt` — dangling `dependsOn` and cycles over a `CanonWorkGraph`. - `probe/FreshnessProbe.kt` — per-Probe `maxAge`, injected `now`. - `probe/AmpereProbes.kt` — `registerAmpereProbes(freshnessMaxAge, now)`, the one-call wiring for a listing. -- `commonTest/.../probe/` — `ProbeSuiteTest`, `ProbeSerializationTest`, `SequenceProbeTest`, `FreshnessProbeTest`. +- `agents/domain/event/ProbeEvent.kt` — `VerdictReached`, the verdict on the bus. It lives in the event package, not here, because `Event` is sealed and Kotlin requires sealed subtypes to share module and package with the base type. +- `commonTest/.../probe/` — `ProbeSuiteTest`, `ProbeSuiteEventTest`, `ProbeSerializationTest`, `SequenceProbeTest`, `FreshnessProbeTest`. ## Where `observedAt` binds @@ -64,13 +68,17 @@ and is never re-stamped on receipt, cache hit, or Plan. - **`Undetermined` never renders as a soft pass.** A consumer that maps it to "ok" has silently accepted absent evidence. - **`observedAt` is bound once, at the source.** Re-stamping on cache hit or Plan would make every cached fact look fresh forever. - **`S` stays unconstrained.** A Probe must be able to check a subject type Ampere never imports. -- **A Probe is not a trace grader.** Grading a recorded run belongs to the eval harness (`EvalCase`, `Meter`s). A freshness verdict cannot be computed from a `MemoryEvent.KnowledgeRecalled` trace alone — it carries no per-fact timestamps — and that is an observability gap, not a reason to move the Probe. +- **A Probe is not a trace grader.** Grading a recorded run belongs to the eval harness (`EvalCase`, `Meter`s). A freshness verdict cannot be computed from a `MemoryEvent.KnowledgeRecalled` trace alone — it carries no per-fact timestamps — and that is an observability gap, not a reason to move the Probe. `VerdictReached` narrows the gap from the other side: the verdict a Probe reached is now *in* the trace, even though it was never computable *from* it. +- **A verdict event carries the subject's id, never the subject.** `S` is unconstrained, so a payload holding the subject would either bind the event to Ampere's types or force a foreign type across the boundary. `subjectId`, `probeId`, `Verdict`, and a small `detail` map are the whole payload. +- **Publishing is opt-in and never partial.** A `ProbeSuite` with no `eventBus` is pure — Bench fixtures and unit tests depend on that. A suite with one publishes after every Probe has run, so a subscriber never sees half a verdict set from a suite that threw halfway through. +- **A verdict is recomputed, never replayed.** `PlaybackRelay` replays a recorded run's LLM calls; it does not replay verdicts. A Probe re-evaluated against a replayed subject must reach its verdict again, or a stale judgement outlives the code that formed it. ## Common operations - **Check a canon entity's freshness** — `FreshnessProbe(maxAge = 24.hours, now = Clock.System::now).evaluate(entity.provenance)`. - **Check a consumer-side binding** — implement `Observed` on the binding, copying the source timestamp at bind time; the same Probe instance applies. - **Run several Probes over one subject** — `ProbeSuite(listOf(freshness, ...)).evaluate(subjectId, subject)`. +- **Make verdicts visible in the trace** — `ProbeSuite(probes, eventBus = bus)`. `eventSource`, `now`, and `idGenerator` are constructor parameters too, so a test can pin exactly what a published event carries. - **Expose Probes for listing** — `ProbeRegistry().registerAmpereProbes(freshnessMaxAge = 24.hours)` registers both shipped Probes; add consumer Probes with `register` afterwards. `all()` feeds the Oscilloscope listing. ## Anti-patterns @@ -78,3 +86,5 @@ and is never re-stamped on receipt, cache hit, or Plan. - **A `Boolean` verdict.** Two independent recons each needed a third value, and different thirds. - **Reading `WebPage.fetchedAt` from Ampere.** It is a Socket type; the isolation check forbids it and the binding already carries the timestamp. - **Per-fact tolerances.** Out of scope by design (AMPR-323); a fact does not know how stale is too stale for a given consumer. +- **Routing on a verdict inside Ampere.** Re-plan, escalation to a human, retry — all consumer-side (Socket decision D21). The event is the signal, not the action. +- **Reaching for a subject-typed verdict event.** The pull is real — a `VerdictReached` would carry more — and it is exactly what makes the event unusable by a consumer whose subject Ampere cannot name.