docs(runs,cleanup): record the measured ceiling on the live-session removal guard (#732)

CHANGELOG.md

@@ -45,6 +45,8 @@ breaking changes may land in a minor release.
 
 ### Changed
 
+- Document the live-session removal guard's measured ceiling (#732): `delete`, `archive` and `clean` still remove a run directory when a listing omits a live session. Behavior unchanged; the psmux half is reported upstream (psmux/psmux#622), its retirement tracked in #754.
+
 - **psmux sessions now live in a per-project registry** (#537). bmad-loop points
   `PSMUX_DATA_DIR` at `<state root>/<project>/_mux`, so a prune in one project cannot address
   another's servers at all. A bare `psmux ls` no longer shows them — `bmad-loop mux` prints the
@@ -180,6 +182,7 @@ breaking changes may land in a minor release.
 
 ### Fixed
 
+- Tell a failed window listing apart from an empty session: liveness raises when the failure is unproven, metadata keeps its sentinel and warns (#525).
 - Anchor the TUI's paused-spec read and its `Request replan` write on the tree the run
   owns. Under isolation both resolved against the main checkout, so the review modals
   showed that copy of the spec and the replan reset it — reporting success while the run's

docs/FEATURES.md

@@ -288,7 +288,7 @@ Result` section. Every other spec keeps warn-and-continue, and the record says w
 - `bmad-loop stop <run-id>` — stop a live run. The default is a **hard stop**: stop now, abandoning the in-flight item and killing the agent session. `--graceful` instead requests a **graceful stop** — the engine finishes the in-flight item (a story through commit, a sweep bundle through commit, or an in-progress sweep triage — after which no bundles start), then finalizes cleanly and stops as a resumable `stopped` run, suppressing any pending auto-sweeps; `--cancel-graceful` withdraws a pending request. Both modes ride the same `stop-request.json` control file, which carries the mode: a hard stop lodges `mode: "hard"` **before** it signals, and that atomic, project-confined write is also what supersedes a pending graceful request. The engine honors a hard request at the next item boundary and mid-session, where each adapter's wait loop polls it twice per iteration — before and after the loop's own up-to-5s wait — so a quiet session normally lands the stop well inside the 10s grace window. That is the common case rather than a bound: an iteration blocked on a transport call, or waiting out `RESULT_GRACE_S` for an artifact, can exceed the window on either adapter before the next poll — an in-flight socket read or tmux call cannot be interrupted from the polling thread, so no placement of the check makes the interval unconditionally short. What the file does guarantee is reach: a hard stop lands on every platform and multiplexer backend, including one where an inter-process signal is never delivered at all (#319). A nested auto-sweep runs inside its parent but mints its own run dir, so it also polls the _owning_ run's channel: stopping the parent stops the child mid-session rather than leaving it to the force-kill backstop. The child's read of the parent channel is hard-only — a graceful stop already keeps a child sweep from starting, and lets one already in flight finish. Teardown is unbounded on top of that — the opencode HTTP adapter then asks the server to abort and to report usage, and a server that will not answer those leaves the stop to the force-kill backstop, exactly as it would have before #319. SIGTERM still goes out alongside it as the POSIX fast path, but it is no longer the mechanism; the engine stays the single writer of `stopped`, and the external force-kill + `run-stop fallback=True` past the grace window now marks a stop this tool had to finish from outside — a teardown that outran the window reaches it as readily as an engine that never read the request — where before #319 it marked every native-Windows stop. The TUI surfaces the same pair: `x` hard-stops, `S` requests a graceful stop.
 - `bmad-loop delete <run-id>` — delete a run directory and its out-of-tree control-plane dir (`--force` stops it first if live).
 - `bmad-loop archive <run-id>` — compress a run into `.bmad-loop/archive` and remove it, control-plane dir included (`--force` stops it first if live). The tarball holds the run dir, so it carries no `events/`. It is staged through an exclusively created temp under a fresh unpredictable name per attempt, so a planted name is never followed or reused, the failure cleanup is provably its own, and a temp stranded by a kill cannot deny later attempts; the tarball is `fsync`ed before the publish — the run dir is removed immediately after, so it is the only remaining copy. A published archive lands at mode `0600` rather than a umask-derived one (#591).
-- Removal refuses while a matching agent session is live that the project cannot prove is another one's, even when the engine is dead: for an untagged session the run dir is the last ownership proof `cleanup` can read, so removing it would leak the session ([#419](https://github.com/bmad-code-org/bmad-loop/issues/419)). A session tagged to another project carries its own proof and never blocks. Run `cleanup` first, having confirmed the session is this project's (`attach`): for an _untagged_ session `cleanup` proves ownership by that same run dir, so two projects sharing a run id can prune each other's. Or pass `--force`, which removes anyway and kills nothing. `clean` leaves such a run untouched and reports it as protected.
+- Removal refuses while a matching agent session is live that the project cannot prove is another one's, even when the engine is dead: for an untagged session the run dir is the last ownership proof `cleanup` can read, so removing it would leak the session ([#419](https://github.com/bmad-code-org/bmad-loop/issues/419)). A session tagged to another project carries its own proof and never blocks. Run `cleanup` first, having confirmed the session is this project's (`attach`): for an _untagged_ session `cleanup` proves ownership by that same run dir, so two projects sharing a run id can prune each other's. Or pass `--force`, which removes anyway and kills nothing. `clean` leaves such a run untouched and reports it as protected. The check is an observation with a measured ceiling ([#732](https://github.com/bmad-code-org/bmad-loop/issues/732)): a listing that omits a live session reads as "no session", so removal proceeds. Two ways to meet it, both measured on psmux 3.3.8 — a `has-session` whose 500 ms connect does not land reaps a live session's registry entry, and until the server's registry maintenance re-writes it `ls` omits that session while exiting 0 with nothing on stderr (that maintenance runs on a nominal 5 s check in the server's own loop, so the window has no hard bound; 1.7 s was one measured sample); and a process whose PATH lacks the multiplexer binary reads every session as absent for as long as that PATH lasts, however long the server keeps running. The first is a successful query with an incomplete answer and the second is a missing binary, and they do not even look alike — a reap takes one entry, so its listing may still name every other session, while a missing binary yields nothing at all — but both reach the guard as the same verdict, this run's name is not in the listing, which is what a genuinely dead session produces too. So it accepts both rather than wedging every removal path behind `--force`. If a removal does strand a session, clear it with `kill-session` on `bmad-loop-<run-id>` — but neither failure leaves it reachable straight away: restore the multiplexer to PATH first, and after a reap wait for the server's next registry tick to republish the entry, or the target will not resolve. On psmux the sessions live in the per-project registry, so run it with the export `bmad-loop mux` prints. Confirm the session is this project's before killing, since the name carries no project — and note that `attach` is no longer the way to do it: it resolves the run directory first, which the removal just deleted. Attach in the multiplexer directly instead, and for an _untagged_ session accept that the run dir it lost was its only ownership proof, so what is left is the session's own contents and what you know of the run.
 - `bmad-loop cleanup` — remove leftover tmux artifacts for finished/stopped runs. `--json` emits the sessions and ctl windows removed (or, with `--dry-run`, that would be) as a stable machine-readable document per the [contract below](#machine-readable-output---json).
 - `bmad-loop clean` — reclaim disk from concluded runs per `[cleanup]`: tear down worktrees a mid-flight stop orphaned, trim heavy `worktrees/` from runs kept for history, archive/delete past the retention window, and sweep orphaned run control-plane dirs from the out-of-tree state root (`--dry-run`, `--keep`, `--retain N`, `--hard`). `--json` emits what was reclaimed (or would be) as a stable machine-readable document per the [contract below](#machine-readable-output---json), with `freed_bytes` a raw integer.
 - `bmad-loop tui` — the interactive dashboard (`--low-frame-rate` for slow/SSH links).

src/bmad_loop/adapters/multiplexer.py

@@ -216,17 +216,30 @@ def list_window_ids(self, session: str) -> list[str]:
         that diverges remains usable, but falls back to the ambiguous by-name
         lookup whenever several kinds share a run id.
 
-        Raises :class:`MultiplexerError` if the transport itself fails (timeout /
-        missing binary): an empty list means "no windows" and must not be
-        conflated with "couldn't ask" — this op backs the engine's liveness
-        probe (:meth:`window_alive`)."""
+        Raises :class:`MultiplexerError` whenever the listing could not be TAKEN
+        — a transport failure (timeout / missing binary) or a query that failed
+        without proving the session gone: an empty list means "no windows" and
+        must not be conflated with "couldn't ask", because this op backs the
+        engine's liveness probe (:meth:`window_alive`).
+
+        So ``[]`` carries a positive claim, not a shrug: the backend either
+        listed the session's windows and found none, or established that the
+        session no longer exists. A backend answering over a server therefore
+        owes callers a discrimination — a server that errors while its windows
+        are alive must not answer ``[]`` (#525). Which conditions PROVE absence
+        is the backend's own question: the exit code alone does not decide it,
+        and neither does a confirming :meth:`has_session`, whose False is
+        weaker than it looks (see its note)."""
 
     @abstractmethod
     def list_windows(self, session: str, fields: list[str]) -> list[tuple[str, ...]]:
         """One tuple per window in ``session``, each holding the requested
         backend fields in order. Best-effort: returns ``[]`` on a transport
-        failure (unlike :meth:`list_window_ids`, this is metadata, not a liveness
-        probe, so a sentinel is safe).
+        failure OR a failed query (unlike :meth:`list_window_ids`, this is
+        metadata, not a liveness probe, so a sentinel is safe — the answer
+        degrades toward doing nothing, never toward claiming a death or a
+        kill). A backend SHOULD still say on stderr when the failure did not
+        prove the session gone, so an every-call failure is not silent (#525).
 
         A ``window_id`` column carries the same id form :meth:`current_window_id`
         AND :meth:`list_window_ids` return; core compares all three directly. The
@@ -241,10 +254,13 @@ def list_windows(self, session: str, fields: list[str]) -> list[tuple[str, ...]]
     def window_alive(self, session: str, window_id: str) -> bool:
         """True iff ``window_id`` is still a window of ``session``.
 
-        May raise :class:`MultiplexerError` when liveness is unknowable (a
-        transport timeout / missing binary) — callers must treat that as "don't
-        know", not "dead", and must not tear down a possibly-working session on
-        it."""
+        May raise :class:`MultiplexerError` when liveness is unknowable — a
+        transport timeout / missing binary, or any other failure to take the
+        listing this membership test reads (see :meth:`list_window_ids`).
+        Callers must treat that as "don't know", not "dead", and must not tear
+        down a possibly-working session on it. Reachable in ordinary operation,
+        not only under a hung binary: a live server that refuses or drops the
+        query answers here (#525)."""
 
     @abstractmethod
     def kill_window(self, target: str) -> None:

src/bmad_loop/adapters/psmux_backend.py

@@ -555,6 +555,17 @@ def list_window_ids(self, session: str) -> list[str]:
         # psmux's list-windows emits bare `@N` lines; qualify them identically
         # to new_window or window_alive's membership check (native_id in
         # list_window_ids) would read every window as dead.
+        #
+        # No `_SESSION_GONE_STDERR` override, and that is a measurement rather
+        # than an omission (#525). psmux 3.3.8 words a vanished session as
+        # `psmux: no server running on session '<name>'` — the base's fragment
+        # matches it — and its client-side variant carries `can't find session`
+        # too. The failures that must NOT read as gone are worded well clear of
+        # both: a live session whose key was rejected answers `psmux: Invalid
+        # session key`, and one whose server is unreachable answers `psmux:
+        # connection timed out`. Both were rc 1 with the windows demonstrably
+        # alive — this backend is where the bug was actually reachable, because
+        # a per-session TCP server has failure modes tmux's socket does not.
         return [
             self._qualified_window_id(session, window_id)
             for window_id in super().list_window_ids(session)
@@ -594,6 +605,16 @@ def list_windows(self, session: str, fields: list[str]) -> list[tuple[str, ...]]
         id_columns = {i for i, field in enumerate(fields) if field == "window_id"}
         if not opt_columns and not id_columns:
             return rows
+        if not rows:
+            # No rows to fill, so the option listing below has nothing to fill
+            # them WITH — a pure short-circuit, byte-identical output. It is
+            # load-bearing anyway (#525): the base answers [] both for a session
+            # it proved gone and for a spawn that never landed, silently in each
+            # case, and pressing on would spend a second probe and then warn that
+            # the option listing failed — about a session that is legitimately
+            # gone, or on a box with no multiplexer at all. The base's honest
+            # silences must not be re-broken by the wrapper that reads them.
+            return rows
         # The #221 degrade: an empty or `:`-bearing session cannot be routed
         # with `-t`, and an unrouted read would answer from whichever server
         # the fallback picks — fill "" without issuing reads at all.
@@ -901,7 +922,12 @@ def _scoped_options(self, session: str) -> dict[str, str] | None:
         surprising {} is possible and is not proof that no keys are set."""
         try:
             proc = self._run(["show-options", "-q", "-t", session], check=False)
-        except (subprocess.SubprocessError, OSError):
+        except (subprocess.SubprocessError, OSError, UnicodeError):
+            # UnicodeError as in the base's listings (#525): this is the SECOND
+            # probe of a two-probe read, so a strict-codec leaf that decoded the
+            # window listing cleanly can still fault here — and the caller above
+            # is a best-effort metadata op that must degrade to "unset", never
+            # raise a decode error out of it.
             return None
         if proc.returncode != 0:
             return None
@@ -951,10 +977,12 @@ def _sweep_orphan_keys(self, session: str) -> None:
                 # A session being swept just minted a window, so an empty live
                 # list is a failed probe, not an empty session — treating it as
                 # truth would sweep every key, live windows included. Warned for
-                # the same reason the listing failure above is, and this is the
-                # branch that actually fires: list_window_ids RAISES on a
-                # transport fault (caught below) and answers [] only on rc != 0,
-                # so silence here is a server failing every launch with no signal.
+                # the same reason the listing failure above is. Since #525 the
+                # only way to reach here is a listing that PROVED the session
+                # gone — every other failure raises and lands in the arm below —
+                # which under a just-minted window means the server died between
+                # the mint and the sweep. Its keys died with it, so the warning
+                # is the whole remaining duty.
                 print(
                     f"warning: orphan-key sweep on {session} could not list live "
                     "windows; orphaned keys unswept until the next launch",
@@ -978,10 +1006,11 @@ def kill_window(self, target: str) -> None:
         # (the project tag scopes the prune retry; the return key keeps both
         # return legs armed, see _parked_trailer). Scope resolves before the kill because a name
         # token cannot be resolved once the window is dead. An empty liveness
-        # listing is ambiguous — a failed probe, or a session that died with
-        # its last window — so it degrades toward retaining the keys; the
-        # launch-time orphan sweep reclaims them once the window is provably
-        # gone. Discovery is generic by the seam's marker — the backend must
+        # listing means the session is gone (#525 narrowed it to that; anything
+        # unproven raises into the arm below), and its keys went with the
+        # server — but this path cannot tell that from the pre-#525 reading, so
+        # it still degrades toward retaining them; the launch-time orphan sweep
+        # reclaims whatever survived once the window is provably gone. Discovery is generic by the seam's marker — the backend must
         # not know which option names callers use. Best-effort throughout:
         # cleanup failure warns (the sweep precedent) but never blocks or
         # fails the kill.