Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions cmd/spinloop/dashboard_render.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ const (
dashHealthy dashHealthTier = iota
dashAttention
dashUnhealthy
dashNotServing
dashUnknown
)

Expand All @@ -128,11 +129,14 @@ const (
// outcome is unhealthy; then a running engine the daemon has explicitly
// reported not ready is attention — the case this tier exists for, a cloud
// node whose process is up but still loading weights; then an answer that
// carries no state at all is unknown; anything else, including a running
// engine the daemon reports no readiness for at all (an older daemon, or a
// runner with no known health check), is healthy, so this degrades to the
// pre-readiness behaviour rather than showing a tier the daemon cannot
// actually back.
// carries no state at all is unknown; then a node that answered with nothing
// serving is not serving, a faded dot rather than the green of a node that
// is up and serving — idle, the daemon with nothing started; stopped, a
// daemon engine that was stopped; undeployed, a remote environment with no
// instance at all; anything else, including a running engine the daemon
// reports no readiness for at all (an older daemon, or a runner with no
// known health check), is healthy, so this degrades to the pre-readiness
// behaviour rather than showing a tier the daemon cannot actually back.
func dashHealthTierFor(r fleet.NodeResult, a dashAction) dashHealthTier {
switch {
case a.verb != "":
Expand All @@ -145,6 +149,8 @@ func dashHealthTierFor(r fleet.NodeResult, a dashAction) dashHealthTier {
return dashAttention
case r.Metrics.State == "":
return dashUnknown
case r.Metrics.State == "idle" || r.Metrics.State == "stopped" || r.Metrics.State == "undeployed":
return dashNotServing
default:
return dashHealthy
}
Expand All @@ -154,13 +160,18 @@ func dashHealthTierFor(r fleet.NodeResult, a dashAction) dashHealthTier {
// panel's name line, in the same raw-ANSI style renderBar already uses for
// the resource bars inside the tile — the tile body is one plain string
// wrapped in a single lipgloss style at the border, so per-character colour
// here has to be ANSI, not lipgloss.Color.
// here has to be ANSI, not lipgloss.Color. Not serving shares unknown's
// faded shade and keeps the dot: the two greys are told apart by their mark,
// a filled dot for a known undeployed node against the ? for one that has
// not answered yet.
func dashHealthGlyph(tier dashHealthTier) string {
switch tier {
case dashHealthy:
return "\033[92m●\033[0m"
case dashAttention:
return "\033[33m●\033[0m"
case dashNotServing:
return "\033[90m●\033[0m"
case dashUnknown:
return "\033[90m?\033[0m"
default:
Expand Down
28 changes: 25 additions & 3 deletions cmd/spinloop/fleet_dashboard_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -330,12 +330,27 @@ func TestDashTileStoppedByteStable(t *testing.T) {
Metrics: metrics.Stats{State: "idle"},
}
want := dashTileExpected([]string{
dashHealthGlyph(dashHealthy) + " idle idle",
dashHealthGlyph(dashNotServing) + " idle idle",
"", "", "", "", "", "", "", "", "", "", "",
})
if got := dashTile("idle", r, false, dashAction{}); got != want {
t.Errorf("stopped tile mismatch:\n%q\nwant:\n%q", got, want)
}
// A remote environment with no instance at all reports undeployed and
// keeps its deployment: the serving line rides on the same shape, and
// the dot is the faded one, not the green of a serving node.
u := fleet.NodeResult{
Name: "dev-1", Outcome: fleet.OutcomeOK,
Metrics: metrics.Stats{State: "undeployed", Runner: "llamacpp", ModelID: "unsloth/Qwen3.8-27B-GGUF"},
}
wantUndeployed := dashTileExpected([]string{
dashHealthGlyph(dashNotServing) + " dev-1 undeployed",
"llamacpp unsloth/Qwen3.8-27B-GGUF",
"", "", "", "", "", "", "", "", "", "",
})
if got := dashTile("dev-1", u, false, dashAction{}); got != wantUndeployed {
t.Errorf("undeployed tile mismatch:\ngot:\n%q\nwant:\n%q", got, wantUndeployed)
}
}

// A node with an action in flight and no report yet shows the verb and the
Expand Down Expand Up @@ -441,8 +456,12 @@ func TestDashHealthTierFor(t *testing.T) {
Metrics: metrics.Stats{State: "running", Ready: "not-ready"}}, dashAction{}, dashAttention},
{"running with no readiness signal degrades to healthy", fleet.NodeResult{Outcome: fleet.OutcomeOK,
Metrics: metrics.Stats{State: "running"}}, dashAction{}, dashHealthy},
{"idle is healthy", fleet.NodeResult{Outcome: fleet.OutcomeOK,
Metrics: metrics.Stats{State: "idle"}}, dashAction{}, dashHealthy},
{"idle is not serving", fleet.NodeResult{Outcome: fleet.OutcomeOK,
Metrics: metrics.Stats{State: "idle"}}, dashAction{}, dashNotServing},
{"stopped is not serving", fleet.NodeResult{Outcome: fleet.OutcomeOK,
Metrics: metrics.Stats{State: "stopped"}}, dashAction{}, dashNotServing},
{"undeployed is not serving", fleet.NodeResult{Outcome: fleet.OutcomeOK,
Metrics: metrics.Stats{State: "undeployed"}}, dashAction{}, dashNotServing},
{"crashed is unhealthy", fleet.NodeResult{Outcome: fleet.OutcomeOK,
Metrics: metrics.Stats{State: "crashed"}}, dashAction{}, dashUnhealthy},
{"unreachable is unhealthy", fleet.NodeResult{Outcome: fleet.OutcomeUnreachable}, dashAction{}, dashUnhealthy},
Expand All @@ -458,6 +477,9 @@ func TestDashHealthTierFor(t *testing.T) {
{"action in flight over a crashed report is attention regardless",
fleet.NodeResult{Outcome: fleet.OutcomeOK, Metrics: metrics.Stats{State: "crashed"}},
dashAction{verb: "stop"}, dashAttention},
{"start in flight over an idle report is attention, never not serving",
fleet.NodeResult{Outcome: fleet.OutcomeOK, Metrics: metrics.Stats{State: "idle"}},
dashAction{verb: "start"}, dashAttention},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-09-02
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
## Context

`fleet dashboard` renders each node as a fixed tile whose name line is
prefixed by a status dot: `dashHealthGlyph(dashHealthTierFor(result, action))`
in `cmd/spinloop/dashboard_render.go`. Today the tier switch has four
outcomes — attention (action in flight), unknown (no refresh yet, or a
report with no state), unhealthy (failed outcome or `crashed`), and
everything else — and the not-serving reports (`idle`, `stopped`,
`undeployed`) fall into "everything else", drawing the green dot of a
serving node. See proposal.md for why that is wrong. The tile body already
prints the state word on its state line, so the text is never in doubt;
only the dot lies.

Constraints: the glyph is raw ANSI in the same style as the resource bars
(the tile body is one plain string under a single lipgloss border style),
and the tile's content lines must stay byte-identical to what the `fleet
metrics` bar format prints for the node — the dot is tile-only, prepended
in `dashTileContent`, and is the one thing the byte-stable tile tests pin
beside the shared lines.

## Goals / Non-Goals

**Goals:**
- A settled report of `idle`, `stopped`, or `undeployed` with no action in
flight draws a faded grey dot instead of green.
- The new grey stays distinguishable from the `unknown` grey, which marks
"no answer yet" with a `?`.
- Every precedence rule the tier switch already enforces (action in flight
beats everything; failed outcome beats state; empty state reads unknown)
is preserved unchanged.

**Non-Goals:**
- No change to any tile body line, to `fleet metrics`/`fleet status`
output, or to the detail view (none of them draw the dot).
- No new state vocabulary: `idle`, `stopped`, and `undeployed` are exactly
the daemon states and the remote control-plane states that mean "not
serving", so the tier keys on the state strings it already sees.
- No colour for states like a remote environment reporting `starting`
outside of a dashboard-initiated start — that path is covered by the
existing action-in-flight rule when the dashboard drove the start, and
untouched otherwise.

## Decisions

**1. A new tier, not a reuse of `dashUnknown`.**
Add `dashNotServing` to the `dashHealthTier` enum with its own case in
`dashHealthGlyph`. Reusing `dashUnknown` would render an undeployed node
with the `?` mark, erasing the difference between "the dashboard hasn't
heard from this node" and "it has, and the node is up with nothing
deployed" — the exact distinction the spec's "different marks" scenario
locks in.

**2. The glyph is `\033[90m●\033[0m`: the existing faded shade, the dot
kept.**
SGR 90 (bright black) is the file's established "faded" code —
`dashUnknown` already uses it for its `?` — so the new tier introduces no
new colour and works on 16-colour terminals like the rest of the palette
(92/33/31/90). The mark tells the two greys apart: filled dot = known
state, `?` = no answer. Alternative considered: a darker 256-colour grey
matching the unselected border (`240`) — rejected, it needs the 256-colour
profile and would make the tile the only part of the view that breaks on a
16-colour terminal.

**3. The new case sits in `dashHealthTierFor` after the empty-state case,
immediately before `default`.**
The switch's order is its precedence. The action-in-flight, no-outcome,
failed-outcome/`crashed`, not-ready, and empty-state cases all come first
and are untouched, so a start in flight over an `idle` report still reads
attention and a failed refresh still reads unhealthy regardless of state.
`idle`, `stopped`, and `undeployed` are non-empty states, so the new case
overlaps nothing; placing it just before `default` is the only position
that changes any behaviour, and it changes exactly the states named in the
spec.

**4. All three off states are in scope: `idle`, `stopped`, `undeployed`.**
A daemon that stops its engine reports `stopped`; a remote environment
reports `stopped` while its instance is stopped and `undeployed` when it has
no instance at all (the control-plane states pass through
`statusFromRemote` unchanged). All are "node up, serving nothing" — the
same fact the dot exists to convey — and keying the tier on `idle` alone
would leave off states wearing different dots on one grid. The proposal
records this as a scoping decision.

**5. Tile body untouched; tests move with the glyph.**
Only `dashHealthTierFor` and `dashHealthGlyph` change, so the
content-line invariant (tile lines = `fleet metrics` bar lines) is intact
by construction. In `fleet_dashboard_test.go`, the byte-stable idle tile
(`TestDashTileStoppedByteStable`) and the tier table's `idle is healthy`
entry change their expected glyph to the new tier, the byte-stable tiles
gain an `undeployed` case carrying its serving line, and the table gains
`stopped`/`undeployed` and idle-with-start-in-flight rows.

## Risks / Trade-offs

- [The byte-stable tile tests pin the glyph bytes, so a wrong escape
sequence ships as a test failure, not a surprise] → they are the check;
the ASCII-profile tests render the new dot as plain `●` like the others.
- [`unknown` and `not serving` now share a colour] → the spec requires
different marks (`?` vs filled dot), and the tile's state line repeats
the word, so a misread needs both the mark and the text to be missed.
- [A remote control plane reporting an off-state string not covered
(e.g. a future state name) would fall through to green] → same
limitation the current switch already has for any unexpected state; the
tier is deliberately keyed on the states the daemon and control plane
actually report today.

## Migration Plan

No migration: a visual change to one dot in one view, no data, no API, no
config. Rollback is reverting the change; nothing written to disk is
affected.
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
## Why

In `fleet dashboard`, a node with no engine deployed — a daemon reporting
`idle`, or a remote environment reporting `stopped` (its instance is
stopped) or `undeployed` (it has no instance at all) — shows the same green
status dot as a node that is running and ready. Green reads as "serving", so a grid of undeployed nodes looks
healthy at a glance when it is actually empty. The dot should carry the
node's serving status, not just its absence of failure.

## What Changes

- Add a fifth dashboard health tier for nodes that are up and answering but
not serving anything: engine state `idle`, `stopped`, or `undeployed`,
with no start or stop in flight on the node. Its status dot is grey — the same faded shade
the `unknown` tier uses, but a filled dot rather than the `?`, so "known
to be undeployed" stays distinct from "no answer yet".
- `dashHealthTierFor` routes settled `idle`/`stopped`/`undeployed` reports
to the new tier instead of falling through to `healthy`. Everything already ranked
above it is untouched: an action in flight still reads attention (a node
that is starting must never read undeployed), a failed outcome still
reads unhealthy, and a report with no state still reads unknown.
- The health-indicator requirement in the `fleet-client` spec moves from
four tiers to five, with scenarios for an undeployed node and for
undeployed-with-start-in-flight.
- The off states are covered as a set: `idle` (the daemon has started
nothing), `stopped` (a daemon engine that was stopped), and `undeployed`
(a remote environment with no instance at all) are the same fact — the
node is up but serving nothing — and off states wearing different dots
would read as inconsistent on the same grid.

## Capabilities

### New Capabilities

(none)

### Modified Capabilities

- `fleet-client`: the "Dashboard panels show a health indicator" requirement
gains a fifth tier — a node whose last refresh reports it not serving
(`idle`, `stopped`, or `undeployed`) and has no action in flight reads
grey, not green.

## Impact

- `cmd/spinloop/dashboard_render.go` — `dashHealthTier`,
`dashHealthTierFor`, `dashHealthGlyph`, and their doc comments. No other
surface draws the glyph: `fleet metrics`, `fleet status`, and the detail
view are unaffected.
- `cmd/spinloop/fleet_dashboard_test.go` — the byte-stable idle tile and
the tier table expect the old green for `idle`; both move to the new
tier, and `stopped`/`undeployed` cases join the table and the
byte-stable tiles.
- No API, config, or dependency changes; no behaviour change outside the
dashboard tile's status dot.
Loading