remove_stale_outputs runs after all detection, so its own deletions are never routed to consumers

[typeless]

Summary

remove_stale_outputs deletes generated files whose producing command no longer exists, but it runs after every detection pass and its deletions never enter changed_files. A consumer that reaches the deleted file by a route not visible in its own command text is therefore not notified during that build.

Pass order in build_single_variant (src/cli/cmd_build.cpp):

find_changed_files_with_implicit -> reconcile_input_set -> expand_implicit_deps
  -> detect_new_commands -> [--rerun] -> remove_stale_outputs -> "Nothing to do" check

remove_stale_outputs returns void, and nothing re-runs after it.

Repro

mkdir -p v/{a,b} && cd v && touch Tupfile.ini
printf ': |> echo hi > %%o |> gen.txt\n' > a/Tupfile
printf ': | ../a/gen.txt |> ls ../a > %%o |> listing.txt\n' > b/Tupfile   # order-only
putup configure && putup
cat b/listing.txt          # Tupfile gen.txt

printf '' > a/Tupfile      # drop a's rule; gen.txt becomes stale
putup                      # reports success
ls a/gen.txt               # gone, deleted as stale
cat b/listing.txt          # STILL "Tupfile gen.txt"  <- names a file that no longer exists

gen.txt is order-only for b, so it never appears in b's rendered text and never affects its identity. At detection time it still hashes clean, so it is not "changed"; it is still in the graph, so reconcile_input_set does not treat it as removed; then it is deleted.

Severity: one build, not permanent

The next invocation does heal it:

putup                      # Build completed: 1 commands
cat b/listing.txt          # Tupfile

So the failure is "one build reports success with a stale output", not indefinite staleness. Worth stating explicitly because it bounds the impact — but a build that exits 0 having left an output inconsistent with the tree is still wrong, and in CI the healing run may never happen.

Fix

Have remove_stale_outputs return the paths it deleted and feed them into changed_files before the up-to-date check. The deleted path is still a graph node at that point, so collect_affected_commands routes it in one hop through the edges that already exist.

More durably: this is the #166 asymmetry again — detection walks the old index, routing walks the current graph, and putup's own deletions belong inside that reconciliation rather than after it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HDzr9JV1ofTZKCL1H2qDPg

[typeless]

Root cause: #189 — the index has no single definition of a command's recorded state, so each category needs its record/compare/route legs wired by hand. This issue is one of the categories where a leg is missing.

[typeless]

Two corrections to this issue, both reproduced on current main (a47fe41f5).

1. It is not bounded to one build. Out-of-tree it never heals.

p1/Tupfile: : |> echo x > %o |> ../g/a.gen
p2/Tupfile: : |> echo y > %o |> ../g/b.gen
m/Tupfile:  : | ../g/*.gen |> ls ../build/g | grep -c "gen$" > %o |> count.txt

putup configure -B build && putup -B build     # count.txt: 2
printf '' > p2/Tupfile                         # b.gen's producer removed

rebuild 1: count=2   files: a.gen
rebuild 2: count=2   files: a.gen
rebuild 3: count=2   files: a.gen              # correct answer is 1

b.gen is deleted, and the consumer that reached it only through an order-only glob is never re-run. In-tree the same project heals one build late — the still-on-disk file re-enters the graph through the filesystem half of the glob and stat-fails on the next run. Out-of-tree it cannot: the file lands under build/, so at deletion time it stats clean (never entering changed, which is reconcile_input_set's only source of removals), and it leaves both the graph and the about-to-be-saved index in the same run. Nothing ever stats it again.

2. The fix proposed here cannot work for this variant.

Routing deleted paths through collect_affected_commands assumes the path still has a graph node. At deletion time it does not — the producing rule is gone, so there is no node to walk from. The routing has to go through the old index's edges, which is the path reconcile_input_set already uses via the command join.

Concretely: compute the doomed-output set before reconcile runs (or have remove_stale_outputs return it) and feed those paths into reconcile's removal loop alongside changed, so they route through old-index Normal/OrderOnly edges to the joined commands.

Note the underlying asymmetry this exposes: reconcile_input_set detects inputs removed by the user (stat fails) but is blind to inputs removed by putup itself. That is the same shape as #152, and it is why this is worth fixing at the routing layer rather than by reordering the passes.

Found by an adversarial review of the merged PRs in this campaign.

[typeless]

Bisect result: no first-bad commit — broken on arrival at 9b7fe4edb

9b7fe4edb  2025-12-16  Fix: delete stale outputs when source files are removed
parent     51a3c8ec9   not expressible (stale outputs are never deleted at all)
9b7fe4edb              BAD 4/4

This is not a regression. The routing gap shipped in the very commit that introduced
stale-output deletion — there has never been a version of putup where this worked.

The parent is not "correct", it is inapplicable: at 51a3c8ec9 the stale output simply
survives, so there is no deletion to route. Verified directly:

listing after build1:        gen.txt Tupfile
a/gen.txt still present?     YES          <- no deletion happens at all
listing after build2:        gen.txt Tupfile

At 9b7fe4edb the deletion starts happening and b/listing.txt immediately goes stale for
one build, which is the behaviour reported here. So remove_stale_outputs was introduced
already sitting after every detection pass, already returning void — the "detection walks
the old index, routing walks the current graph" asymmetry was present from the feature's
first line rather than introduced later.

Reproducer used

Exit 0 = the listing is refreshed in the same build, 1 = it still names the deleted file,
125 = the premise doesn't hold (nothing was deleted, so there is nothing to route).

#!/bin/bash
P="$1"
D=$(mktemp -d) || exit 125
trap 'rm -rf "$D"' EXIT
cd "$D" || exit 125

mkdir -p a b
touch Tupfile.ini
printf ': |> echo hi > %%o |> gen.txt\n' > a/Tupfile
printf ': | ../a/gen.txt |> ls ../a > %%o |> listing.txt\n' > b/Tupfile

"$P" configure >/dev/null 2>&1 || exit 125
"$P" >/dev/null 2>&1 || exit 125
grep -q 'gen.txt' b/listing.txt 2>/dev/null || exit 125

printf '' > a/Tupfile
"$P" >/dev/null 2>&1 || exit 125
[ -e a/gen.txt ] && exit 125               # premise: the stale output was deleted

grep -q 'gen.txt' b/listing.txt && exit 1  # listing still names a deleted file => bug
exit 0

Confirmed BAD at every probe from 9b7fe4edb through fc14f89de.

Method

Historical commits were built from the committed bootstrap-linux.sh (driver objects only,
-flto stripped, ccache + parallel xargs). Only optimisation and link flags differ from
a normal build. Boundary verdicts were re-measured 4-5x.