Skip to content

fix(api): stop the reaper claiming "no response from agent" when the agent replied (#3097) - #3172

Open
bdunncompany wants to merge 1 commit into
LanternOps:mainfrom
bdunncompany:fix/3097-reaper-terminal-guard
Open

fix(api): stop the reaper claiming "no response from agent" when the agent replied (#3097)#3172
bdunncompany wants to merge 1 commit into
LanternOps:mainfrom
bdunncompany:fix/3097-reaper-terminal-guard

Conversation

@bdunncompany

Copy link
Copy Markdown
Collaborator

Step 1 of the sequencing you set on #3097 — the reaper guard, on its own, ahead of the extraction. Leaves #3097 open for the extraction PR.

What it fixes

Results submitted over the HTTP path never reach script_executions, so the row stays pending, ages past the cutoff, and lands in reapStaleScriptExecutions — which stamped it timeout with "Server-side timeout: no response from agent".

That's false whenever a terminal device_commands row exists. The agent did respond; the result was simply never mirrored. 89 executions on one live instance read timeout while their command had completed successfully with output.

The reaper now reads the related command before deciding, and when that row is terminal it records the outcome the command actually reached — mapped the same way handleScriptResult maps it, so a reaped row agrees with what the WS path would have written. Non-terminal command, or no command row at all: original wording stands, because there it's true.

It does not persist stdout/stderr. That's the extraction's job. This only stops the reaper asserting something it cannot know.

The command lookup already existed a few lines below, used only for batchId. It moved above the update and serves both purposes now, so no extra round-trip.

Two things to look at in review

The batch counter carried the same false assumption — it incremented devicesFailed unconditionally, so a recovered success was still counted as a batch failure. It now increments devicesCompleted when the command completed. That's slightly beyond a literal "add a guard", and I'd rather you saw it flagged than buried: leaving it would have kept the same wrong claim one level up.

reapStaleScriptExecutions is now exported so it can be tested directly.

Tests

Four cases, and the two I care most about are the negative ones — a guard that swallowed real agent silence would be worse than the bug it fixes:

  • command completed → recovers as completed, message no longer claims silence
  • command failed → records failed, not timeout
  • command non-terminal (sent) → still timeout, still "no response from agent"
  • no command row at all → same

Confirmed the two recovery cases fail against the previous behaviour before the guard went in.

Local gate

Node 22.23.2, all green: tsc --noEmit clean; vitest run in apps/api 1271 files / 20146 tests passed, 0 failed; eslint clean on both changed files.

Extraction next, once this lands — services/ home for the handler block and registry, canonical byte-length commandResultSchema exported from schemas.ts and imported by both transports, the two dynamic imports left as-is, and a body that can honestly say the handlers are byte-identical with the schema unification as the one intentional behaviour change.

…agent replied (LanternOps#3097)

Script results submitted over the HTTP path never reach `script_executions` —
only `agentWs` registers `script: handleScriptResult` — so the row stays pending,
ages past the cutoff, and lands in `reapStaleScriptExecutions`, which stamped it
`timeout` with "Server-side timeout: no response from agent".

That claim is false whenever a terminal `device_commands` row exists: the agent
did respond, the result simply was never mirrored onto the execution. On one live
instance 89 executions read `timeout` while their command had completed
successfully with output captured in `device_commands.result`.

The reaper now reads the related command BEFORE deciding. When that row is
terminal it records the outcome the command actually reached — mapped the same
way `handleScriptResult` maps it, so a reaped row agrees with what the WS path
would have written — with an error message saying the result was delivered but
never recorded. When the command is non-terminal, or there is no command row at
all, the original wording stands, because there the claim is true.

This does NOT persist stdout/stderr. Mirroring the result belongs with the shared
handler-registry work; this only stops the reaper asserting something it cannot
know, which is where the false `timeout` labels come from.

The command lookup already existed a few lines further down, used only to find
`batchId`. It moved above the update and now serves both purposes, so this adds
no extra round-trip.

Two consequences worth flagging in review:

- The batch counter followed the same false assumption: it incremented
  `devicesFailed` unconditionally. A recovered success now increments
  `devicesCompleted` instead. Slightly beyond a literal reading of "add a guard",
  but leaving it would have kept the same wrong claim one level up.
- `reapStaleScriptExecutions` is exported so it can be tested directly.

Four tests: command completed (recovers as completed, message no longer claims
silence), command failed (records failed, not timeout), command non-terminal
(genuine silence still reported as timeout), and no command row (same). The last
two matter most — a guard that swallowed real agent silence would be worse than
the bug. Confirmed the two recovery cases FAIL against the previous behaviour.

Local gate, green on node 22.23.2: `tsc --noEmit` clean; `vitest run` in apps/api
1271 files / 20146 tests passed, 0 failed; eslint clean on both changed files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant