[codex] Add provenance audit observations

[bokelley]

Summary

• Adds an optional get_creative_features.audit_observations[] success-field for governance findings that should be visible to buyers without rejecting creative.
• Defines the first standardized audit observation, OVERSIGHT_DISCLOSURE_CARVEOUT_CLAIMED, for AI-assisted provenance that claims edited/directed human oversight while disclosure.required is false.
• Updates provenance guidance and adds a conformance storyboard that verifies the seller accepts directed/edited carve-out creatives, forwards the provenance claim to the verifier, and exposes the recorded observation through a sandbox-only controller query rather than public seller responses.
• Extends the training-agent sandbox verifier/controller path so query_provenance_audit_observations can assert that the verifier actually emitted audit_observations[].

Closes #4438.

Validation

• Expert review: protocol/docs/schema/code review found no blockers after the field-doc, storyboard-wording, and verifier-emission coverage follow-ups were fixed.
• npm run build:schemas
• npm run build:compliance
• npm run test:schemas
• npm run test:json-schema
• npm run test:storyboard-sample-request-schema
• npm run test:storyboard-response-schema
• npm run test:docs-nav
• npm run check:owned-links
• npm run lint:schema-links
• npm run test:composed
• npm run test:storyboard-validations-paths
• npm run test:storyboard-context-output-paths
• npm run test:platform-agnostic
• npm run test:error-codes
• npm run test:sign-protocol-tarball
• npm run test:oneof-discriminators
• npm run test:schema-utf8
• npm run test:storyboard-check-enum
• npm run test:storyboard-doc-parity
• npm run test:substitution-vector-names
• npx vitest run server/tests/unit/comply-test-controller.test.ts --pool=threads
• npx vitest run server/tests/unit/training-agent.test.ts server/src/training-agent/tenants/tenant-smoke.test.ts --pool=threads
• Precommit: test:unit, test:test-dynamic-imports, test:callapi-state-change, typecheck
• Prepush: version sync, current compliance storyboard matrix, 3.0 compatibility storyboard matrix, docs schema-link convention, Mintlify broken-links check

[aao-release-bot]

Approving. Right shape for the protocol: audit observations are a separate surface from rejection codes, and keeping OVERSIGHT_DISCLOSURE_CARVEOUT_CLAIMED out of error-code.json preserves the "errors are rejection signals" invariant that file enforces.

Things I checked

• static/schemas/source/creative/get-creative-features-response.json:32-38 — audit_observations is optional on the success branch; the branch has no additionalProperties: false, so legacy senders/receivers are unaffected. minor changeset is correct.
• static/schemas/source/creative/audit-observation.json — closed shape (additionalProperties: false) at root, details, and claimed_value. ext is the only extension surface. Internally consistent with the MDX examples and the storyboard payload (code, severity: audit-worthy, recovery: informational spelled the same in all six occurrence sites).
• Code/observation separation: it's an observation, not an error. Folding it into error-code.json would have eroded the invariant. Right call.
• Working-group alignment: faithful to the docs/creative/provenance.mdx:160 Warning on EU AI Act Article 50(4) editorial responsibility, and to core/provenance.json:32-34 framing that human_oversight and disclosure.required are independent fields. The protocol surface here matches what the corpus already committed to.
• Storyboard validation primitives: field_value with allowed_values and upstream_traffic with payload_must_contain are both documented in static/compliance/source/universal/storyboard-schema.yaml and used in prior scenarios. Real primitives, not invented.
• scripts/audit-oneof.mjs impact: the new object is a single closed shape, no new oneOf introduced. Walker is not affected.

Follow-ups (non-blocking — file as issues)

• Allowlist "mirror" is a constrained subset. audit-observation.json:39 and provenance-verification.mdx:167 describe the details shape as mirroring the PROVENANCE_CLAIM_CONTRADICTED allowlist. The mirror's field set matches, but claimed_value here is a closed object (human_oversight enum + disclosure_required: const false) where the contradiction allowlist treats claimed_value as free-form. Reword to "constrained subset" so emitters don't expect to pass arbitrary claimed_value payloads.
• Required-set drift between docs and schema. docs/governance/creative/get_creative_features.mdx:154 and provenance-verification.mdx:167 advertise the allowlist as the full six-field tuple. audit-observation.json:84 only requires agent_url, claimed_value — feature_id, observed_value, confidence, substituted_for are optional. A conformant emitter could legally ship only the two required fields and still surprise a buyer who read the docs. Either tighten required on the schema for the "when applicable" fields or soften the doc copy from "details limited to {…}" to "details limited to a subset of {…}."
• Forward-compat smell on the second observation code. code is a one-value enum at audit-observation.json:8-14 and claimed_value is shaped specifically for that code. The next code will force a refactor to oneOf keyed on code (or a looser claimed_value). Worth a one-line note in the schema description that the shape MAY become a oneOf on code in a future minor, so adopters don't write parsers that assume the v1 shape is permanent.

Minor nits (non-blocking)

1. upstream_traffic endpoint pattern is too loose. static/compliance/source/protocols/media-buy/scenarios/provenance_audit_observation.yaml:192 uses endpoint_pattern: "POST *" which matches any outbound POST. Tighten to "POST *governance.encypher.seller.example*" so the check actually asserts the seller called the on-list verifier instead of any URL.
2. Storyboard fixture asset must exist. provenance_audit_observation.yaml:150 references https://test-assets.adcontextprotocol.org/acme-outdoor/ai-generated-true.jpg. Confirm that asset is published or runners that fetch it will fail. Notable for a sample that nominally exercises AI-generated content — the file name is the only thing carrying the "AI" semantics in the fixture.
3. PR title prefix. [codex] is not conventional-commits; the squash commit will rewrite to feat(provenance): add audit observations per the existing convention so this is squash-safe, but worth fixing the title before merge for the PR-list readability.

LGTM. Follow-ups noted below.

[arian-gogani]

provenance audit observations are the right primitive for creative governance. the underlying question: can the observation itself be tamper-evident?

if a governance finding says "this creative claims human oversight but disclosure.required is false," that finding should be independently verifiable. a signed observation means a buyer can trust the audit result without trusting the audit platform.

we built this receipt layer for AI agent actions. Ed25519 signed, JCS canonical (RFC 8785), hash-linked chains. the same pattern applies to creative provenance: each audit observation generates a signed receipt with the finding, the policy version, and the creative hash.

github.com/arian-gogani/nobulex

[aao-release-bot]

LGTM. Additive surface only — new optional audit_observations[] field, new shared schema with one enumerated code, new sandbox-only controller scenario behind the 3.0-compat gate. minor changeset is the right call.

The shape is right: OVERSIGHT_DISCLOSURE_CARVEOUT_CLAIMED is claim-driven, not verifier-driven, and the schema enforces that via claimed_value.disclosure_required: const false + human_oversight ∈ {edited, directed}. Mirrors the PROVENANCE_CLAIM_CONTRADICTED audit-safe allowlist (agent_url, feature_id, claimed_value, observed_value, confidence, substituted_for) — same closed-allowlist defense as the contradiction error path. Audit observation is non-rejection by construction; the storyboard's field_absent: creatives[0].errors[0].code validations lock that contract in.

Things I checked

• enforceProvenancePolicy return-shape refactor from TaskError | null → { error, auditObservations } propagated to every structural-error path (task-handlers.ts:1454-1574) and to the single success site in handleSyncCreatives (task-handlers.ts:4015-4019), with set/delete branches keyed on creativeId.
• Tenant isolation preserved — sessions keyed via sessionKeyFromArgs, observations attached to session.complyExtensions.provenanceAuditObservations. No cross-session leak surface.
• 3.0-compat gating closed in all three places: comply-test-controller.ts:885-892, localScenariosFor, and tenants/router.ts:298-311.
• Schema-vs-docs coherence: field table in get_creative_features.mdx, the Article 50(4) carve-out prose in provenance.mdx:160, and the verifier walkthrough in provenance-verification.mdx:139-167 all converge on the same trigger condition and the same flattened-alias semantics. No drift.
• oneOf disjointness on comply-test-controller-response.json — ProvenanceAuditObservationsSuccess is uniquely identified by creative_id + audit_observations in its required set; positive disjointness holds even though only SeedSuccess.not.anyOf got the new field added (see follow-up).
• Storyboard provenance_audit_observation.yaml exercises both directed and edited paths, asserts the verifier was actually invoked via check: upstream_traffic with payload_must_contain on human_oversight and disclosure.required, then asserts the recorded observation through the sandbox-only controller — not through a public seller response. Right separation.
• Real test exercise, not mocked: comply-test-controller.test.ts:181-274 rides through sync_creatives → enforceProvenancePolicy → runProvenanceVerifier → handleGetCreativeFeatures.

Follow-ups (non-blocking — file as issues)

• oneof-discriminators.baseline.json is stale. Variants moved 7 → 8 but the baseline note still reads variants: 7. audit-oneof.mjs --check passes (no kind/key regression), so this is not a gate failure — but refresh via node scripts/audit-oneof.mjs --update before the baseline drifts further.
• Partial not.anyOf extension. audit_observations was added to SeedSuccess.not.anyOf (line 360) but not the other sibling variants. Disjointness holds via required-set distinctness, but the file's convention is to enumerate all sibling discriminator markers in every variant. Extend for consistency.
• Behavior change at task-handlers.ts:5670-5703. When digital_source_type is not a string, runProvenanceVerifier no longer short-circuits before verifier selection — it now runs the in-process handleGetCreativeFeatures call before short-circuiting. In-process is cheap; if a real seller wires this to a network call, the cost profile shifts. Intentional, since the carve-out observation is independent of DST, but worth a sentence in the PR body or a one-line comment naming the intent.

Minor nits (non-blocking)

1. Test brittleness. comply-test-controller.test.ts:268-274 dereferences observations[0].code without first checking observations.length. Add expect(observations.length).toBeGreaterThan(0) so a regression to [] produces a readable failure instead of a TypeError.
2. Three-place restatement. The same trigger rule for OVERSIGHT_DISCLOSURE_CARVEOUT_CLAIMED is stated in provenance.mdx, the get_creative_features.mdx paragraph below the field table, and provenance-verification.mdx. The middle one can compress to one sentence — the table above it already carries the load.
3. agent_url override in runProvenanceVerifier (task-handlers.ts:5710-5717) intentionally rewrites details.agent_url to chosen.agent_url over whatever the in-process handler returned. One-line comment that the public response carries the seller's chosen on-list verifier, not the local handler's identity, would save a reader the trace.

LGTM. Follow-ups noted below.

[aao-release-bot]

LGTM. Clean additive minor: new optional audit_observations[] on a success response, new opt-in query_provenance_audit_observations controller scenario, audit-safe details allowlist mirrors PROVENANCE_CLAIM_CONTRADICTED exactly. The right shape.

Things I checked

• Scenario enum agreement across all eight call sites (LOCAL_SCENARIOS, SALES_CURRENT_SCENARIOS, tenant-smoke, get_adcp_capabilities handler, request/response schemas, capabilities-response, two test assertions) — consistent, no drift.
• 3.0 compat gating: scenario is filtered out in localScenariosFor (comply-test-controller.ts:769), in tryHandleLocalComplyScenario (tenants/router.ts:301-311), and in the capabilities handler (task-handlers.ts:4562). Handler also returns UNKNOWN_SCENARIO if called under 3.0 (comply-test-controller.ts:885-893). Fail-closed in all three paths.
• enforceProvenancePolicy return-shape refactor: every early-return path returns { error, auditObservations: [] } — six rewrite sites at task-handlers.ts:1468, 1480, 1495, 1503, 1513, 1544. All accounted for.
• runProvenanceVerifier correctly stamps agent_url to chosen.agent_url and only adds substituted_for when the buyer URL was off-list (task-handlers.ts:5705-5712). Off-list buyer URLs are rejected at step 5 with PROVENANCE_VERIFIER_NOT_ACCEPTED before observations are recorded — so substituted_for in a persisted observation is unreachable from buyer-controlled content.
• Schema-vs-docs coherence: get_creative_features.mdx and provenance-verification.mdx field tables match audit-observation.json field names, optionality, enum values, and the claimed_value shape. Changeset is minor, which matches the additive wire impact.
• disclosure_required: const false lock + human_oversight enum ['edited','directed'] — the trigger surface and the schema are pinned together.
• Session isolation: writer (task-handlers.ts:4015-4019) and reader (comply-test-controller.ts:1662-1694) use the same sessionKeyFromArgs derivation. No cross-session lookup. Per-session Map bounded by MAX_CREATIVES_PER_SESSION = 500. security-reviewer: no High/Medium findings, safe to merge.
• ad-tech-protocol-expert: sound-with-caveats (asymmetric not.anyOf reciprocity on sibling variants — see follow-up). code-reviewer: no blockers, internal-consistency clean.
• Backwards-compat on deserializeSession: asMap fallback covers sessions persisted before this field existed (state.ts:395).

Follow-ups (non-blocking — file as issues)

• not.anyOf reciprocity is one-sided. SeedSuccess was extended to exclude audit_observations (comply-test-controller-response.json:360), but the other six sibling variants (ListScenariosSuccess, StateTransitionSuccess, SimulationSuccess, ForcedDirectiveSuccess, UpstreamTrafficSuccess, ControllerError) were not. Not a real collision today — ProvenanceAuditObservationsSuccess requires creative_id, which no other variant carries, so the keyset can't double-match — but the file's established convention is full mutex. Tighten in a follow-up to keep the discriminator pattern uniform.
• Test coverage gap. query_provenance_audit_observations has happy-path coverage only (comply-test-controller.test.ts:194-269). The NOT_FOUND branch and the "creative exists, no observations recorded" empty-array branch are documented behaviors with no tests. Add sibling cases.
• Map<string, unknown[]> under-constrains the writer. types.ts:255 types provenanceAuditObservations as unknown[]; the concrete shape is CreativeAuditObservation[] declared inside task-handlers.ts:5551. Export the interface and tighten the type to prevent a future writer from storing arbitrary buyer payloads (defense-in-depth — the current single writer is safe).
• Audit observations survive force_creative_purge hard-purge by design. The comply-test-controller.ts:1681 dual-check (session.creatives.has || provenanceAuditObservations.has) accommodates this. Worth a one-line comment naming the intent so a future reader doesn't "tighten" it.

Minor nits (non-blocking)

1. Helper to compress the six-site rewrite. enforceProvenancePolicy's early returns are now six near-identical { error: provenanceError(...), auditObservations: [] } blocks (task-handlers.ts:1468-1544). A provenanceFailure(...) wrapper would cut the noise. Style only.
2. Behavior change worth flagging in code. runProvenanceVerifier now calls the verifier even when there's no digital_source_type claim, where it used to early-return (task-handlers.ts:5680). Intentional — the audit-observation surface needs the verifier call regardless of DST — but a one-line comment would prevent a future reader from "fixing" the order back.

Approved.

[aao-release-bot]

LGTM. Follow-ups noted below. Additive schema change with a clean disjointness ratchet — new optional field on get_creative_features, new opt-in scenario, new oneOf variant whose disjointness keys on audit_observations, and a sandbox-only controller hook that keeps the internal audit log off public seller responses. The seller/verifier split (audit_observations recorded against session.complyExtensions, queried only via comply_test_controller) is the right shape.

Things I checked

• enforceProvenancePolicy refactor returns {error, auditObservations} at every callsite. Six early-return branches and the contradiction path all forward auditObservations: [] correctly; the contradiction + success paths forward what the verifier produced. server/src/training-agent/task-handlers.ts:1448-1572.
• Persistence is gated on !isDryRun and uses set/delete for stale-state hygiene. task-handlers.ts:4015-4019.
• The new oneOf variant in static/schemas/source/compliance/comply-test-controller-response.json:386-454 is mutually disjoint from all 7 existing variants — every existing variant now adds audit_observations to its not.anyOf, and the new variant excludes itself from the other six discriminators.
• scripts/oneof-discriminators.baseline.json:99-103 legitimately ratchets from 7→8 variants with audit_observations as the new discriminator — this is a real new shape, not a walker bypass.
• Changeset is .changeset/4438-provenance-audit-observations.md with minor. Correct: additive optional field, additive enum value behind the existing "MUST accept unknown scenario strings" guard.
• 3.0 compat gating is consistent across comply-test-controller.ts, tenants/router.ts, tenant-smoke.test.ts, and task-handlers.ts (handleGetAdcpCapabilities does not advertise the scenario under 3.0).
• Storyboard upstream_traffic assertions key on creative_manifest.provenance.* — the canonical wire path, portable across implementations.
• Docs/schema coherence: OVERSIGHT_DISCLOSURE_CARVEOUT_CLAIMED, the creative_manifest.provenance.disclosure.required path, and the audit-worthy/informational enums appear identically across the schema and all three docs files.

Follow-ups (non-blocking — file as issues)

• Hard purge leaks audit observations. server/src/training-agent/comply-test-controller.ts:1426 handleForceCreativePurge calls session.creatives.delete(creativeId) without mirroring the delete on session.complyExtensions.provenanceAuditObservations. The new query handler's fallback (|| against the audit map) will then return observations for a creative that no longer exists in the sandbox instead of NOT_FOUND. Mirror the delete on hard purge.
• Verifier now runs for every creative with accepted_verifiers, even when there's no provenance object at all. task-handlers.ts:5670-5705. Not buggy — buildOversightDisclosureAuditObservations short-circuits on missing provenance — but a if (!provenance) return { contradiction: null, auditObservations: [] }; guard before the in-process verifier call restores the previous behavior for non-provenance creatives.
• Negative-path tests missing on query_provenance_audit_observations. server/tests/unit/comply-test-controller.test.ts covers the happy path but not INVALID_PARAMS (missing params, empty creative_id), NOT_FOUND (creative never synced), or the 3.0-compat UNKNOWN_SCENARIO rejection. Four error branches unreached.
• recovery: informational is a vocabulary fork. Canonical error-code recovery enum is {correctable, transient, terminal}. The new value is intentional — observations aren't errors — but worth a one-line callout in audit-observation.json to keep future implementers from conflating the two surfaces.
• claimed_value mirror claim is loose. audit-observation.json:39 says details mirrors PROVENANCE_CLAIM_CONTRADICTED's allowlist, but the typing differs: CONTRADICTED carries a string DST, the new observation carries a typed object. The docs flag the flattened-alias note, but reusing the field name across two shapes will trip naive serializers expecting typeof claimed_value === 'string'.

Minor nits (non-blocking)

1. Path-prefix drift between docs. docs/governance/creative/get_creative_features.mdx:155 writes creative_manifest.provenance.disclosure.required; docs/governance/creative/provenance-verification.mdx:160 writes provenance.disclosure.required. Both are correct in context. Unify.

Approved.

[aao-release-bot]

LGTM. Clean additive feature with disciplined surface design — claim-driven observation that fires on the success branch, no rejection coupling.

Things I checked

• audit-observation.json shape — code/severity/recovery triple with recovery: "informational" deliberately disjoint from the error-code recovery enum. Right call. Audit observations live on the success branch; reusing the error-recovery vocabulary would have conflated "fix this to proceed" with "we noted this." additionalProperties: false at envelope + details closes the verifier-leak gap provenance-verification.mdx:167 warns against.
• details allowlist { agent_url, feature_id, claimed_value, observed_value, confidence, substituted_for } mirrors the PROVENANCE_CLAIM_CONTRADICTED allowlist at docs/governance/creative/provenance-verification.mdx:133 exactly. Same audit-safe story.
• oneOf hygiene on comply-test-controller-response.json: all 8 branches mutually disjoint. New ProvenanceAuditObservationsSuccess carries every sibling-discriminator key in its not.anyOf (lines 421-451), and every sibling added audit_observations to theirs. Baseline ratchet 7→8 at scripts/oneof-discriminators.baseline.json:99-103 is intentional.
• Changeset type minor is correct — optional field added to a success branch, new oneOf branch with full disjointness, additive scenario enum value (open-for-extension already documented at comply-test-controller-response.json:51).
• runProvenanceVerifier reordering at server/src/training-agent/task-handlers.ts:5663-5746. New if (!provenance) guard at 5673 prevents calling the verifier on creatives with no provenance object. PROVENANCE_CLAIM_CONTRADICTED path preserved — typeof claimed !== 'string' still short-circuits at 5723 before any contradiction check. Claim-driven audit observations now fire even when DST is absent. Right shape.
• enforceProvenancePolicy return-type change to { error, auditObservations }. Every return site converted (1458, 1472, 1483, 1496, 1503, 1515, 1547, 1572, 1575). Caller at task-handlers.ts:3960 consumes both fields and persists with explicit set-or-delete (4015-4019) so re-sync of a non-audit creative clears any prior observation.
• 3.0 compat gating consistent across four sites: handler (comply-test-controller.ts:885-893), localScenariosFor filter (769), tenants router (298-313), get_adcp_capabilities exposure (task-handlers.ts:4562). No path where a 3.0 client gets a success on one route and UNKNOWN_SCENARIO on another.
• Hard purge wires session.complyExtensions.provenanceAuditObservations.delete(creativeId) at comply-test-controller.ts:1427. Soft purge leaves observations in place — matches the tombstone semantics.
• Storyboard validates the three observable contracts: accept-not-reject (creatives[N].action ∈ {created, updated} + field_absent on errors[0].code), outbound verifier call (upstream_traffic with payload_must_contain on the carve-out fields), sandbox controller assertion via query_provenance_audit_observations. Does not require the seller to echo the audit observation on the public sync_creatives response — that boundary is respected.

Follow-ups (non-blocking — file as issues)

• runProvenanceVerifier now calls handleGetCreativeFeatures for every sync_creatives with provenance + accepted_verifiers, even when digital_source_type is absent. Previously short-circuited. Intentional — claim-driven observations need the verifier path — but it's an additive hot-path cost on every provenance-carrying sync. Worth a perf note in the handler comment so a future reader doesn't "fix" it back.
• task-handlers.ts:5715-5722 overrides details.agent_url after spreading the inner observation, and conditionally adds substituted_for. Today handleGetCreativeFeatures never sets substituted_for so this works, but if the inner handler ever starts emitting one, the conditional add silently shadows it. A defensive merge (substituted_for: substituted ?? observation.details.substituted_for) or a one-line comment pinning the assumption would future-proof.

Minor nits (non-blocking)

1. Baseline catch-up bundled with the schema change. scripts/oneof-discriminators.baseline.json was created in 62af2cc and shipped stale relative to acquire-rights-response (status→rights_status), creative-approval-response (status→approval_status), verify-brand-claim-response (status→verification_status), and audience-selector (signal_id removal). All four are catch-up from prior merges, not changes authored here. Worth a note: a baseline that needs four unrelated regenerations one merge after creation is something to keep an eye on.

Approved.