Skip to content

Harden the orphan-fallback conformance driver and the handler-enumeration guard #280

Description

@chris-colinsky

Blocks v0.17.0. Test-quality items from two adversarial reviews of #277. None changes shipped behaviour; each is a case where a green run means less than it appears.

1. _handler_event_types() silently drops handlers

tests/unit/test_observability_otel.py derives its coverage list by regexing the observer's source for def _handle_*(self, event: ...) followed by event=event,. That misses a multi-line signature, a differently-named parameter, and a call site passing the event positionally — so a future handler opting into orphan-path synthesis is silently uncovered, which is exactly the class the guard exists to close.

Prefer inspect.signature over a regex, and assert the parsed set is non-empty and matches an expected count, so a parse that silently narrows fails loudly.

2. test_dispatch_span_openers_tolerate_every_optional_field_being_absent never calls an opener

It drives _event_caller_metadata / _apply_caller_metadata / _subgraph_identity_at directly. Confirmed by mutation: reverting the openers' defensive read leaves it green — only the end-to-end test_failure_isolated_marker_survives_orphan_path_synthesis fails. Either drive a real opener or rename it to say it tests the helpers.

3. _stored_lineage does not guard chain_len == 0

branch_name_chain[: chain_len - 1] becomes [:-1] at chain_len == 0, which drops the last element instead of returning empty. Not currently reachable (a dispatch span always has a non-empty prefix), so it is a latent trap rather than a live defect.

4. Conformance driver nits (tests/conformance/test_observability.py)

  • _mock_bodies_by_request_content validates the pairing with a count over a list but returns a dict keyed by request content, so two wrappers declaring the same content collapse to one entry while the count assertion still passes — making orphan_llm_span_routed_to_correct_branch_by_name vacuous.
  • The mock _handler's AssertionError on unmatched content is raised inside fan-out instance middleware under error_policy: collect, which absorbs it, so a pairing failure surfaces as an unrelated missing-span message rather than the handler's diagnostic.
  • dispatch_spans_close_in_declaration_order reads end times from a dict whose insertion order is event-arrival order, which Resolve orphan provider span parents structurally, wire 152/153 #277 makes depend on which branch's wrapper-issued event drains first.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions