feat: reuse chunk vectors across re-ingest, and clear the ones we missed

[matthewjhunter]

Closes the memstore-shaped half of task 3137.

The bug fix has to be read first

clearVectors (the recipe reconciler) and ResetEmbeddings (memstore admin reset-embeddings) both promised every stored vector and cleared only the fact side. Document chunks were given vectors in #208 and neither path was updated.

So reset-embeddings reported a count that excluded document vectors and left them in place, and a recipe change would have rebuilt facts under the new recipe while documents stayed in the old one -- two vector spaces in one store, and a document ranking that degrades without announcing itself. That is exactly the failure the embedder fingerprint exists to prevent, reintroduced underneath it.

This commit lands first because vector reuse is only sound while that guarantee holds.

Reuse

Re-ingest replaces a document's whole chunk set -- the corpus design is explicit that chunks are replaced, not merged -- so every chunk of a changed file was re-embedded even when its own bytes never moved. On a repo sync that is most of the embedding work in the run, and all of it is waste: the same text under the same header renders the same input to the same model.

The key is everything that reaches the embedder and nothing else:

• Included: content, heading path, symbol scope, declaration kind, document path. ChunkEmbedText renders the heading and scope into a header ahead of the body, so identical text under a moved heading is a different input and must be embedded again. There is a test for that, because matching on content alone is the obvious implementation and it would silently keep a vector for text the model never saw in that form.
• Excluded: ordinal, byte offsets, line offsets. A chunk that shifted down the file because something above it grew is the same input, and re-embedding it is the waste being removed.

The document path is in the key even though reusableChunkVectors already scopes its query to one document. The path is the header's source field, so a key without it would permit cross-document reuse the moment someone widened that query, and the borrowed vector would carry another file's name in its header. Both are pinned by tests.

Fields are length-prefixed rather than delimited, so no field value can impersonate a boundary between two others.

What 3137 asked for that was already here

The task's headline safeguard -- an embedding-model fingerprint that refuses rather than returning silently degraded rankings -- already exists and is stronger than the pattern it cites: reconcileEmbedder makes a model or dimension change a hard error at store-open with a remediation hint, and clears vectors automatically on a recipe-only change. Its torn-write and atomic-rename safeguards are Postgres transactions here, and its cache-version constant is schemaVersion. File-level skipping of unchanged files already exists in the manifest sync.

🤖 Generated with Claude Code