#481 chore: add diagnostic logging to broadcast/drain pipeline#486
#481 chore: add diagnostic logging to broadcast/drain pipeline#486hoblin wants to merge 1 commit into
Conversation
Wires dev-only logging on every server emit (PendingMessage broadcast_created, broadcast_removed, MessageBroadcaster#emit, DrainJob mailbox decisions) to log/broadcast.log, and a parallel TUI-side logger (--broadcast-debug flag, log/tui_broadcast.log) on every received WebSocket message. Pair the two streams to count creates/removes/messages per batch round and identify any per-PM mismatch, per the issue #481 investigation. Server-side BroadcastDiagnostics follows the Melete/Mneme module logger pattern — null logger outside development so call sites stay unconditional. TUI-side BroadcastLogger follows the PerformanceLogger flag-gated pattern.
|
Closing — over-engineered for the goal. The acceptance criterion is to identify the broadcast leak's root cause once; shipping a permanent module + TOML setting + CLI flag + TUI logger class for a one-shot investigation is dead weight in the gem. Replacing with inline |
Summary
Investigative logging for #481 (TUI pending tool responses leak: some entries persist until restart). Adds a paired server + TUI logging stream so a leak round can be replayed post-hoc.
BroadcastDiagnostics.logger(Melete/Mneme module-logger pattern) writes tolog/broadcast.login dev only:PendingMessage#broadcast_created/#broadcast_removed— every emit, with PM id, message_type, source_name, tool_use_id, session_id, view_modeMessageBroadcaster#emit— every Message broadcast, with id + tool_use_id + actionDrainJob#perform— whenstart_processing!refuses, log state +tool_round_complete?+ mailbox countsDrainJob#drain_mailbox— round membership (tool_response PM ids + tool_use_ids), promoted countTUI::BroadcastLogger(PerformanceLogger flag-gated pattern), opt-in via--broadcast-debug, writes tolog/tui_broadcast.log:action,type,id,pending_message_id,tool_use_idpending_message_removed: logs whetherMessageStore#remove_pendingactually matched a stored entry (the silent-miss case is the leak's smoking gun)[broadcast] log_pathsetting; existing users will needanima migrateto pick it up before passing--broadcast-debug.Why this approach
Research ruled out the original "decorator raise eats the broadcast" framing —
drain_mailboxruns eachpromote!in its own transaction, so a per-PM render failure can't take down siblings. Research also ruled out the comment's revised hypothesis that afrom_melete_goalunknown-tool error breaksSession#tool_round_complete?— unknown-tool PMs havemessage_type: \"tool_response\"and the correcttool_use_id, so the round-completion query counts them correctly. The remaining failure modes (broadcast lost in transit, TUI subscription race on reconnect,remove_pendingsilent miss because the create broadcast was missed) all need the paired server-emit / TUI-receipt log to diagnose, which is what this PR delivers.How to use
Reproduce a batch tool round (e.g. spawn multiple sub-agents in parallel), wait for the leak, then compare:
broadcast_createdserver lines → Nrecv pending_message_createdTUI lines?broadcast_removedserver lines → Nrecv pending_message_removed matched=trueTUI lines?MessageBroadcaster ... broadcastserver lines → matchingrecv action=create type=tool_responseTUI lines?Any mismatch identifies the leak point.
Test plan
bundle exec rspec spec/models/pending_message_spec.rb spec/jobs/drain_job_spec.rb spec/lib/events/subscribers/message_broadcaster_spec.rb spec/lib/tui/— 748 examples passbundle exec rspec spec/lib/anima/cli_spec.rb— updated to assert newbroadcast_debugkwarg, new spec for--broadcast-debugflagbundle exec standardrb— clean--broadcast-debug, trigger a multi-tool round, verify both log files populate; record findings on TUI pending tool responses leak: some entries persist until restart #481Breaking changes
~/.anima/tui.tomlwill need to runanima migrateto pick up the new[broadcast] log_pathsetting before passing--broadcast-debug. Users who never pass the flag are unaffected (the setting is only read when the flag is on).Closes part of #481 (the diagnostic-logging acceptance criterion). Root-cause identification + follow-up implementation issue are the remaining ACs and will be handled after the diagnostic logs let us reproduce.