Cap a failed observation's error_message and reserve the openarmature_ namespace

CHANGELOG.md

@@ -21,6 +21,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The
 - **Adaptive call-level retry: per-attempt request override** (proposal 0095, llm-provider §7.1, spec v0.91.0). The LLM-completion call-level retry loop gains an opt-in per-attempt request override. A new `LlmRetryConfig` (the llm-provider-scoped superset of the generic `RetryConfig`, exported from `openarmature.llm`) carries a `per_attempt_override`: a schedule of `RuntimeConfig` partials applied to retries. Attempt 0 uses the caller's base `config` unchanged; retry `i` merges `per_attempt_override[i]` onto the base (the override's non-None fields replace; a None or unspecified field inherits the base, per the §6 null-skip semantics), and the last entry carries forward when the schedule is shorter than the retry count. The canonical use is an escalating temperature schedule that breaks the "temperature 0 replays the same output" determinism trap on a retried structured-output call. `complete()` never mutates the caller's `config` (each attempt config is a fresh copy), and a plain `RetryConfig` preserves the existing byte-identical replay. The per-attempt OTel span carries a new `openarmature.llm.retry_reason` attribute (`transient`) on retries, absent on the base attempt. This is the first half of proposal 0095; the structured-output reask half follows. Spec v0.91.0 is beyond the current v0.88.0 pin, so the behavior ships ahead of the pin (unit-tested); the conformance fixtures 061-066 ride the v0.17.0 pin bump.
 - **Adaptive call-level retry: structured-output reask** (proposal 0095, llm-provider §7.1, spec v0.91.0). The second half of 0095. `LlmRetryConfig` gains an opt-in `reask` builder (`Callable[[StructuredOutputInvalid], str]`). When present, a `structured_output_invalid` failure becomes retryable for that call (a call-level convenience, not a classifier change; without a builder it stays non-transient and raises on the first occurrence). On each such failure the loop appends two messages to a working transcript, the model's raw output as an `assistant` message and the builder's returned correction as a `user` message, so the retry is informed rather than a byte-identical replay. OA authors no prompt of its own (the caller owns every word beyond the model's output); the builder receives the raised `StructuredOutputInvalid` (its `raw_content` and `failure_description`). The transcript accumulates reask pairs across reask retries and consumes the `max_attempts` budget; a transient retry interleaved in a reask loop re-sends the accumulated transcript unchanged. `complete()` never mutates the caller's `messages` (each reask replaces the transcript with a fresh list rather than appending in place). The retry span's `openarmature.llm.retry_reason` is `reask` on a reask retry, `transient` otherwise. A reask always appends the model output as a fresh `assistant` message (never continues a trailing one): §3 requires the last message before a call to be `user`/`tool`, so the transcript never ends in `assistant`. Ships ahead of the pin (unit-tested); fixtures 062-066 ride the pin bump.
 - **Langfuse observer: credentials-in construction with tracer-provider isolation** (proposals 0114 + 0116 + 0117 + 0118, observability §6 / §8.9 / §8.4, spec v0.108.0 / v0.110.0 / v0.111.0 / v0.112.0). The Langfuse observer gains a second construction mode alongside today's caller-supplied client: `LangfuseObserver.from_credentials(public_key=..., secret_key=..., host=...)` (over the lower-level `LangfuseSDKAdapter.from_credentials(...)`) builds an OA-owned `Langfuse` client on a dedicated `TracerProvider` by default, so its observations no longer bind the global provider and leak onto the application's OTel backend. A Langfuse v4 client constructed with no `tracer_provider=` attaches its span processor to the globally-registered provider, so in any service that registers a global provider (the standard app-tracing setup) attaching the Langfuse observer silently exported every observation, prompts and completions included, to the app backend. Because the Langfuse SDK caches one client per `public_key`, a dedicated provider takes effect only when OA is the first constructor for that credential; OA reuses one isolated provider per credential and reads the actual binding back after construction. The invariant covers every payload OA harvests from the runtime -- the provider payload (`disable_provider_payload`), the Trace-level state input/output (`disable_state_payload` and the `trace_input_from_state` / `trace_output_from_state` hooks), and a failed Tool / Embedding / Retriever / LLM observation's `error_message` -- but not the dimensions the caller deliberately attaches (`correlation_id` / `session_id` / `userId` / trace name / caller metadata), which stay verbatim as cross-backend join keys. When any construction-determinable channel is live and OA establishes the client is bound to a provider it did not isolate, construction fails loud with a categorized `LangfuseProviderIsolationUnavailable` before any observation is emitted, rather than leaking payloads to a shared backend; where OA cannot establish the binding at all (a future SDK), it suppresses every channel and logs a warning. A failed observation's `error_message` is harvested exception text, so `disable_provider_payload` governs it for every failure category on all four provider observations: with payloads off it is not rendered, and the error category still rides as the status message where the event carries one. A Tool failure has no category, so its status message is null rather than falling back to the exception string. `error_type` is a classification token rather than harvested content and is never gated, which matters most for a Tool failure where it is the only remaining discriminator; it is optional, so it is emitted only where the failure event supplies one. A single `accept_shared_provider=True` opt-out turns the whole thing into a warn-and-proceed onto the shared provider. With no channel live (the default privacy posture), an un-isolatable client neither raises nor warns. The existing caller-supplied path (mode a) is unchanged and never mutated: a caller who builds their own client stays responsible for isolating its `tracer_provider`, and OA documents the remedy rather than reaching into the supplied client. The `secret_key` is accepted as a `pydantic.SecretStr`, masked in OA's own reprs and logs with the plaintext read only at the SDK call (`public_key` and `host` stay plain strings), and a blank credential is rejected at the boundary rather than falling through to the SDK's ambient `LANGFUSE_*` environment fallback. A `sample_rate` passed for the client is applied to the isolated provider, since the SDK only honors it on a provider it builds itself. `accept_shared_provider` binds the provider the application already registered rather than letting the SDK construct and globally register one of its own, which would capture OTel's single-assignment global slot. The new `LangfuseProviderIsolationUnavailable` derives from an `ObservabilityError` base, a fourth hierarchy alongside the graph-engine, llm-provider, and checkpoint ones. The behavior shipped ahead of the pin (unit-tested) and the pin has since advanced to v0.112.0. Fixture 159 now runs, and fixtures 098 / 137 / 138 are un-deferred and reconciled to the post-0118 shape. Fixtures 157 / 158 stay deferred: they need the conformance-adapter provider-faithful Langfuse fake and the `langfuse_client` construction directive, which are not yet built, and the source behavior is covered meanwhile by the unit suite.
+- **Failed-observation `error_message` byte cap, and the `openarmature_` reserved namespace** (proposal 0119, observability §5.5.5 / §8.7 / §3.4, spec v0.116.0). Closes the two error-channel edges 0118 left open. **The cap:** a failed observation's `error_message` is now subject to the §5.5.5 per-value byte cap, on all four mapped provider observations (Generation, Embedding, Tool, Retriever). It takes the contract's *direct-application* arm rather than its *inheritance* arm: the OTel surface defines no `error_message` span attribute, so the value arrives untruncated and the observer that writes it applies its own `payload_byte_cap`, rather than inheriting a cap from an upstream OTel truncation that never happened. Re-applying a second cap to an already-truncated value would move the marker and misreport its byte total, which is why the two arms source the cap differently. A failed Tool observation renders the same harvested string twice, once in `metadata.error_message` and once as the observation's `statusMessage`, and **both** copies are capped: §5.5.5 governs payload-classified values rather than payload-classified fields, so capping one surface while the other still carried the whole exception would defeat the cap. The remaining `statusMessage` writes take the error *category*, a classification token, and stay uncapped. Under the default posture (`disable_provider_payload=True`) the field is absent entirely per 0118, so the cap is observable only where payloads are enabled, and the omission arm is unchanged: a withheld message still leaves `statusMessage` null on a Tool failure rather than substituting the message. **The reserved keys:** `openarmature_` joins `openarmature.` and `gen_ai.` as a reserved caller-metadata *namespace* prefix, and four exact names (`error_type`, `error_message`, `token_budget`, `token_budget_exceeded`) join the reserved set, which grows from 29 to 33. With `error_message` absent under the default posture, an unreserved caller key of that name would otherwise land unopposed in the very field 0118 requires to be absent, reintroducing through the metadata channel the leak the gate closes. Pre-1.0 behavioral change: a caller passing `invocation_metadata` with a key beginning `openarmature_`, or with any of those four names, is now rejected at the `invoke()` boundary with `ValueError` where it previously passed. Spec v0.116.0 is beyond the current v0.112.0 pin, so the behavior ships ahead of the pin (unit-tested); the `conformance.toml` entry and the fixtures ride the pin bump. No fixture drives the Tool arm today: the conformance adapter exposes no tool-calling node, so that arm is unit-tested only and the gap is reported to spec. **The maintenance check 0119 asks for lands with it:** a test that statically scans the Langfuse mapping for every top-level metadata key it writes and fails when one is neither reserved by name nor covered by a reserved namespace. It discovers the metadata bags from what is passed to a `metadata=` argument rather than hardcoding one name, because the observer builds five of them and a scan of the obvious one would miss four; 0119 asks for this specifically, since a sweep that looked at only a subset is what missed nine keys before. The check found three keys on the failure-isolation marker span (`error_category`, `failure_isolation_event_name`, `failure_isolation_node`) that a caller key still overwrites, because that handler merges caller metadata last. Those are held rather than reserved unilaterally: the marker is a graph-mechanism span that no mapping table covers, so whether the reserved set should reach an unmapped span is a spec question, and it is raised as one. **Truncation is also now surrogate-safe:** the cap runs on the failure path, where the value is harvested exception text and is the likeliest string in the observer to carry a lone surrogate (an `OSError` naming a surrogateescape-decoded path). Encoding one raises, an observer that raises is only warned about rather than logged, and the cap runs before the observation is created, so an unguarded encode would have deleted the very observation reporting the failure. A malformed message now degrades that one field and still respects the cap.
 
 ### Changed
 

docs/agent/non-obvious-shapes.md

@@ -119,7 +119,7 @@ Different classes, same OTel-Logs export path. If both are attached against the
 
 The trace-side sibling of the `LoggerProvider` log-bridge gotcha above. When OA constructs the Langfuse client for you (`LangfuseObserver.from_credentials(...)` / `LangfuseSDKAdapter.from_credentials(...)`), it binds the client to a dedicated `TracerProvider` so OA's observations do not leak onto a provider shared with the application's OTel backend. But the Langfuse v4 SDK caches ONE client per `public_key` process-wide: if any client for that key was constructed first (the app called `Langfuse()`, used `langfuse.openai` / `@observe`, or an earlier OA call), the SDK returns the cached client and OA's dedicated provider is silently discarded. So isolation only holds when OA is the FIRST constructor for that credential.
 
-Rather than leak silently, OA detects the binding and fails closed. When a harvested-payload channel is live and OA establishes the client landed on a provider it did not isolate, `from_credentials` raises `LangfuseProviderIsolationUnavailable` at construction, surfacing an init-ordering bug you would otherwise never see. The guarded channels are the provider payload (`disable_provider_payload`) and the Trace-level state input/output (`disable_state_payload` and the `trace_input_from_state` / `trace_output_from_state` hooks). A failed observation's `error_message` is harvested exception text, so `disable_provider_payload` governs it too, on all four mapped provider observations (LLM, Embedding, Tool, Retriever) and for every failure category: with payloads off it is simply not rendered. The error category still rides as the status message where the event carries one, so you can still see what kind of failure it was; a Tool failure has no category, so its status message is null rather than falling back to the exception string. `error_type` is a classification token, not harvested content, so it is never gated, which matters most for a Tool failure where it is the only discriminator left. A graph-mechanism span (a node span, a failure-isolation marker) never carries the exception message on any provider; that detail rides the OTel span instead (`openarmature.failure_isolation.message`), never Langfuse. Caller-attached dimensions (`correlation_id`, `session_id`, `userId`, trace name, your own metadata) are join keys by design and stay verbatim.
+Rather than leak silently, OA detects the binding and fails closed. When a harvested-payload channel is live and OA establishes the client landed on a provider it did not isolate, `from_credentials` raises `LangfuseProviderIsolationUnavailable` at construction, surfacing an init-ordering bug you would otherwise never see. The guarded channels are the provider payload (`disable_provider_payload`) and the Trace-level state input/output (`disable_state_payload` and the `trace_input_from_state` / `trace_output_from_state` hooks). A failed observation's `error_message` is harvested exception text, so `disable_provider_payload` governs it too, on all four mapped provider observations (LLM, Embedding, Tool, Retriever) and for every failure category: with payloads off it is simply not rendered. The error category still rides as the status message where the event carries one, so you can still see what kind of failure it was; a Tool failure has no category, so with payloads off its status message is null rather than falling back to the exception string. Where payloads are on, the message is capped at the observer's own `payload_byte_cap` (nothing upstream has truncated it, because the OTel surface defines no `error_message` attribute), and a Tool failure's status message carries that same capped string rather than the full exception. `error_type` is a classification token, not harvested content, so it is never gated, which matters most for a Tool failure where it is the only discriminator left. A graph-mechanism span (a node span, a failure-isolation marker) never carries the exception message on any provider; that detail rides the OTel span instead (`openarmature.failure_isolation.message`), never Langfuse. Caller-attached dimensions (`correlation_id`, `session_id`, `userId`, trace name, your own metadata) are join keys by design and stay verbatim.
 
 Remedies: construct OA's Langfuse client before any other client for that `public_key`; or pass `accept_shared_provider=True` to acknowledge a shared provider (OA warns and proceeds); or build your own client with an isolated `tracer_provider=` and pass it in via the caller-supplied path (`LangfuseObserver(client=LangfuseSDKAdapter(your_client))`), which OA never mutates. Under OA's default privacy posture (no payload channel live), an un-isolatable client is harmless and neither raises nor warns.
 

docs/concepts/observability.md

@@ -340,15 +340,29 @@ Every observability backend picks the entries up:
   work without any custom dashboard config.
 
 Validation runs at the `invoke()` boundary before any work begins.
-Two rules:
-
-- **Keys** MUST NOT start with `openarmature.` or `gen_ai.`
-  (reserved for spec-normative attribute namespaces; collisions
-  would silently overwrite OA-emitted state).
+Three rules:
+
+- **Keys** MUST NOT start with `openarmature.`, `gen_ai.`, or
+  `openarmature_` (reserved namespaces; collisions would silently
+  overwrite OA-emitted state). `openarmature_` is the underscore
+  form, for backends whose key syntax cannot carry a dot, and it is
+  a namespace rather than a list: any key under it is rejected, not
+  only the ones a mapping happens to write today.
+- **Keys** MUST NOT exactly match a reserved name. These are the
+  top-level metadata keys OA itself writes alongside yours, so a
+  caller key of the same name would overwrite one. The list grows
+  with the spec mapping and currently includes `correlation_id`,
+  `entry_node`, `spec_version`, `namespace`, `step`, `error_type`,
+  `error_message`, `token_budget`, and `token_budget_exceeded`. The
+  authoritative set is `_RESERVED_KEY_NAMES` in
+  `openarmature.observability.metadata`.
 - **Values** MUST be OTel-attribute-compatible scalars (`str`,
   `int`, `float`, `bool`) or homogeneous arrays of those types.
   `None`, nested objects, and mixed-type arrays are rejected.
 
+`userId` is deliberately not reserved: OA reads it to promote to
+Langfuse's first-class `trace.userId`, so you can keep passing it.
+
 Violations raise `ValueError` synchronously: no spans emitted, no
 work runs.
 
@@ -1255,9 +1269,19 @@ What remains is enough to triage the failure. `error_type` is a
 classification token (an exception class name or vendor code), never
 gated, and the error category still rides as the observation's status
 message wherever the event carries one. A **tool** failure has no
-category, so its status message is null rather than falling back to the
-exception string. The full exception text is unaffected on the OTel
-side.
+category, so when the message is withheld its status message is null
+rather than falling back to the exception string. The full exception
+text is unaffected on the OTel side.
+
+Where the flag does permit the message, it is capped at
+`payload_byte_cap` like any other payload value. The cap is applied
+directly by this observer rather than inherited: the OTel surface
+defines no `error_message` span attribute, so unlike
+`generation.input` / `output` the value arrives untruncated and has
+been capped by nobody upstream. A tool failure renders the same string
+twice, in `metadata.error_message` and as the status message, and both
+copies are capped, so a provider that echoes an HTML error page cannot
+render in full through either one.
 
 ### Prompt linkage
 

docs/patterns/caller-supplied-trace-identifiers.md

@@ -87,8 +87,10 @@ Validation runs synchronously, before any node body fires. Both
 `invoke(metadata=...)` and `set_invocation_metadata(...)` enforce
 the same rules:
 
-- Keys MUST NOT start with `openarmature.` or `gen_ai.` (reserved
-  namespaces per the spec).
+- Keys MUST NOT start with `openarmature.`, `gen_ai.`, or
+  `openarmature_` (reserved namespaces per the spec). The
+  underscore form covers backends whose key syntax cannot carry a
+  dot.
 - Keys MUST NOT collide with the spec's reserved per-trace metadata
   keys (`correlation_id`, `entry_node`, `spec_version`, etc.). The
   set is enforced at the `invoke()` and `set_invocation_metadata`