Skip to content

fix(index): re-derive task IDs when the derivation rules change - #56

Merged
fohara merged 4 commits into
mainfrom
fix/54-id-derivation-drift
Aug 12, 2026
Merged

fix(index): re-derive task IDs when the derivation rules change#56
fohara merged 4 commits into
mainfrom
fix/54-id-derivation-drift

Conversation

@fohara

@fohara fohara commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Fixes #54.

A derived value cached behind a hash of its input, where the derivation is the
other input and nothing was watching it.

A task's ID comes from its title and is not written to the Markdown unless the
author pins it with @id:. 0.3.0 changed the derivation — underscores became
separators instead of vanishing — so every unpinned ID moved while every
content hash stayed byte-identical. Incremental indexing keys off those hashes,
so a file nobody had edited since the upgrade was never re-parsed and kept
serving IDs derived under rules no longer in force.

The failure was silent in four directions at once. lash show read the stored
record and printed the old ID. lash lint derived a new one and rejected the
reference. check-index compared hashes, found them equal, and said in sync.
lash index said "Unchanged: 1". Assuming lint was wrong is the only
conclusion the output supports.

The index records what derived it

ID_DERIVATION_VERSION names the current rules and is stamped into the
metadata table. On mismatch — or absence, the same thing — the hash diff is
ignored and every file is re-parsed. An upgrade repairs itself on the next
lash index.

The stamp is written only after a run that can vouch for the whole project:
unscoped, no parse errors. A scoped run re-derives part of it and a failed
parse leaves that file's old rows in place; stamping after either claims a
freshness the index does not have, and the next run skips the repair.

The repair captures what it moved

Correcting the stored IDs is half of it. A @depends-on written against an old
ID is text in a file and stops resolving the moment the stored IDs move — all
of them together, which is why --force made the rebuild look like the cause
of the damage rather than the fix.

The re-derive is the only moment both spellings exist, so the mapping is taken
there, into a new id_migrations table (schema v9).

Matching old rows to new tasks without using the ID — the thing under suspicion
— is by title plus structural position (depth, order_index), none of which
the rules touch. That is exact only because the file's hash is unchanged, which
is checked first. Any key claimed twice on either side is dropped rather than
guessed: an ambiguous pairing becomes a rename written into someone's Markdown,
whereas a missed rename surfaces as an unresolved reference the author fixes.

check-index re-derives instead of trusting hashes

It now parses each file whose hash already matches and compares the IDs — the
expensive path for the otherwise-cheap case, and the only one that catches
this, because an unchanged file is precisely the file whose IDs never get
re-derived. Reports IssueKind::StaleTaskIds; opt out with
VerifierConfig::with_task_id_check(false).

lint names the cause

An E_LINK_NOT_FOUND whose target the index still recognises, or that matches
a pending rename, is not a typo. The note goes in the diagnostic's help for
JSON and -v, and once more after the summary — help is hidden at normal
verbosity, and this is the difference between "lint is wrong" and "here is what
happened".

lash migrate-ids

Consumes the recorded renames. Previews by default, writes with --write,
--forget discards them for a project repairing by hand. Touches whole
references on @depends-on: lines and nothing else: prose mentioning an old ID
is someone's notes, and the unqualified old-id form is left alone because a
bare token can name a file as readily as a task. That gap is printed rather
than left implicit.

--force no longer destroys the mapping

--force wipes the database, and the database is the only place a task's
previous ID survives. It was the workaround the issue documented, and it used
to correct the stored IDs while leaving every reference dangling with nothing
left to explain them.

Pending renames are now read out before the wipe — and undetected drift drawn
out first, with one incremental pass — then restored afterwards. Best-effort
throughout: a database too broken to read is exactly why someone reached for
--force, so a failed salvage does not block the rebuild.

Line endings

apply_rewrites splits lines with their terminators attached and puts them
back untouched. Splitting on content and rejoining with \n would have
rewritten every line ending in a CRLF file as a side effect of changing one
reference, and would have added a trailing newline to a file that had none.

End to end, from the reported state

$ lash index
1 task ID changed: this index was built under older ID rules.
  index#founder-add-releasesmirrortoken-secret-t → index#founder-add-releases-mirror-token-secret
...
$ lash lint
1 of these reference(s) point at task IDs that a derivation change moved, not at tasks that are missing.
  Run `lash migrate-ids --write` to update these references.
$ lash migrate-ids --write
Rewrote 1 reference(s):
  lash.index.md:18  index#founder-add-...-t → index#founder-add-releases-mirror-token-secret
$ lash lint && lash check-index
✓ All files passed linting
✓ Index is in sync

Tests

  • 24 integration tests in id_derivation_drift_test.rs covering all four
    surfaces and the full recovery, including that a genuine typo is not
    dressed up as a derivation change and that an edited file is not guessed at
  • 7 indexer unit tests (forced re-derive, exact mapping, persistence, the
    version stamp, scoped runs)
  • 4 verifier, 5 repository, 3 migration, 3 connection, 15 migrate-ids
    unit tests (reference matching, line endings, line-number agreement)

Full workspace suite passes (61 suites); clippy and fmt clean. The agent-prompt
snapshot is updated for the new subcommand.

fohara added 4 commits August 11, 2026 15:49
The index now records the ID-derivation version it was built under and
re-derives every file on a mismatch, since a rules change moves every
unpinned ID while leaving content hashes identical. check-index compares
stored IDs against freshly derived ones, lint names the cause, and the
new lash migrate-ids rewrites references left dangling.

Fixes #54
Rejoining split lines with \n rewrote every line ending in a CRLF file
as a side effect of changing one reference. Discovery now goes through
FileWalker so the rewritten set matches what the project indexes.
--force wipes the database, which is the only place a task's previous ID
survives. Pending renames are now read out (and undetected drift drawn
out) before the wipe, and restored after.
…drift

# Conflicts:
#	CHANGELOG.md
#	devlog.md
@fohara
fohara merged commit d2076c4 into main Aug 12, 2026
21 checks passed
@fohara
fohara deleted the fix/54-id-derivation-drift branch August 12, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale task IDs survive lash index, so lash show and lash lint disagree while check-index reports in sync

1 participant