Fix caller-metadata handling on two Langfuse handlers

[chris-colinsky]

Two defects in how the Langfuse observer merges caller-supplied invocation metadata, plus a correction to reasoning that a spec ruling overturned. Both defects were found while acting on spec's ruling in the v0.17.0 coordination thread, not by a fresh sweep.

The failure-isolation marker let a caller key win

_handle_failure_isolated merged the caller set last, with a plain per-key assignment and no collision check. That made it the only handler where a colliding caller key won: the LLM handlers pick the set up through the shared typed-event metadata builder, and the embedding and rerank handlers apply it directly, all of them before writing their own keys.

It mattered because three of the marker's top-level keys (error_category, failure_isolation_event_name, failure_isolation_node) are not in the reserved set, so invoke() does not reject a caller key of the same name at the boundary and the collision reached the observer. A caller passing error_category silently replaced the marker's only failure discriminator, on the failure path, with nothing logged.

The caller set now merges first. A non-colliding caller key still comes through unchanged.

This is the lesser harm, not a settled precedence rule. OA-wins still drops the caller's value silently, which is the mirror of the bug rather than its opposite, and §3.4 rejects a reserved collision precisely because silent resolution in either direction loses information without telling anyone. The real fix is reservation, and it arrives with the span's mapping: spec ruled that §3.4's reserved set does not reach a span no §8.4.x table maps, and committed the forthcoming failure-isolation mapping to carry these three keys.

The OTel observer needs no equivalent change. Every attribute it writes is openarmature.-prefixed and caller keys land under openarmature.user.*, both covered by a reserved prefix, so a colliding caller key is rejected at the boundary and never reaches the merge. Langfuse metadata is flat, which is the whole reason the reserved name set exists alongside the prefixes. The precedence silence spec found is therefore reachable only on a flat-metadata backend.

The Tool observation carried no caller metadata at all

§8.4.2 maps the caller set to observation.metadata.<key> on every Observation. The unscoped wording is deliberate: the same table scopes its other rows explicitly where it means to, for example fan_out_item_count to the fan-out node Span only.

The tool handler picked the set up through neither route, so a caller filtering Langfuse by their own key (a tenant id, a request id) saw every observation from an invocation except the tool calls. The OTel observer's tool span has carried it all along, so the two bundled observers disagreed about the same event.

Fixed, merged before the OA keys to match the other handlers. Note that precedence is unobservable on this particular observation either way, because every key it writes is reserved (the openarmature_* pair by prefix, error_type / error_message by name), so a colliding caller key never reaches the merge. Merge-first is for consistency with the handlers where it is load-bearing. That was reachable before 0119, though: error_type was unreserved then, so a caller key of that name merged last would have replaced the only discriminator a failed Tool observation carries.

Corrected reasoning, unchanged behavior

Three sites argued that the failure-isolation marker sits outside §5.6 because §5.6's scope sentence enumerates the spans it reaches without including it. Spec has since ruled that list illustrative rather than exhaustive: the rule reaches every span emitted during the invocation, because a framework span missing openarmature.correlation_id is broken in the way that attribute exists to prevent.

The conclusion survives on the other ground, which spec confirms is the load-bearing one: openarmature.failure_isolated appears nowhere in the observability spec, so §5.6 cannot mandate attributes on a span the spec never defines. The comments now rest on that alone and say explicitly not to resurrect the enumeration argument, which has now been wrong in two directions across three passes. The practical consequence is recorded too: once the span is mapped, §5.6 reaches it and this stops being a consistency choice.

Retraction

PR #288, its CHANGELOG entry, and two coordination messages claimed the conformance adapter exposes no tool-calling node, so no fixture could reach §8.7's Tool arm. That is wrong and is retracted here.

The adapter has driven tool calls since 0092. calls_tool with a mock_tool block is a supported directive, mock_tool: {raises: {error_type, message}} produces a ToolCallFailedEvent carrying the fixture's own message, and fixture 098 case 2 already drives exactly that into a Langfuse Tool observation and asserts error_message in its metadata. It runs green. The real gap is narrower: fixture 160 carries no Tool case, and one is writable today against machinery that already exists. #288's description and the CHANGELOG are corrected.

Testing

Two new tests, both mutation-verified. Reverting the failure-isolation merge order turns its test red with the caller's value sitting in error_category, which is the original defect exactly; removing the tool handler's caller merge turns the other red.

One mutant survives deliberately: merging the caller set last on the tool handler changes nothing observable, for the reservation reason above. That is recorded in the test rather than left looking like a coverage gap.

[Copilot]

🟢 Approval recommended

The changes are targeted, consistent with existing handler patterns, and are covered by new regression tests that specifically exercise the reported failure modes.

Pull request overview

This PR fixes two Langfuse observer metadata-merging defects to align behavior across handlers and with the observability spec’s caller-metadata mapping, and it updates comments/docs to reflect a recent spec ruling about §5.6 scope semantics.

Changes:

• Fix Langfuse openarmature.failure_isolated marker metadata merge order so OA-emitted keys cannot be silently overwritten by colliding caller metadata.
• Ensure the Langfuse Tool observation includes caller-supplied invocation metadata (consistent with §8.4.2 and the other provider observations).
• Add focused regression tests and update related reasoning in comments and CHANGELOG.md.

File summaries

File	Description
tests/unit/test_observability_langfuse.py	Adds a unit test asserting Tool observations carry caller invocation metadata.
tests/unit/test_failure_isolation_middleware.py	Updates rationale comments and adds a regression test for collision precedence on the failure-isolation marker.
src/openarmature/observability/otel/observer.py	Corrects explanatory comments per updated spec ruling (no behavior change).
src/openarmature/observability/langfuse/observer.py	Fixes merge ordering for failure-isolation marker and adds caller-metadata merging to Tool observations.
CHANGELOG.md	Documents the two Langfuse fixes and corrects prior claims about Tool-arm conformance reachability.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The changelog retains an outdated claim that the now-settled reservation question remains unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Balanced