feat(cli): add migrate-model command for legacy index migration#232
Open
moabualruz wants to merge 8 commits into
Open
feat(cli): add migrate-model command for legacy index migration#232moabualruz wants to merge 8 commits into
moabualruz wants to merge 8 commits into
Conversation
Add failing tests that define the expected behavior for embedding model tagging on chunks and the multi_model configuration option. Tests cover: - EmbedModel filter returns only matching chunks - EmbedModel filter excludes chunks with empty tags (strict) - No filter returns all chunks (zero behavioral change) - MultiModel defaults to false - EmbedModelTag() helper returns provider/model format - EmbedModelTag() guards against empty provider or model
…ict filtering - Add EmbedModel string field to store.Chunk for provider/model tagging - Add EmbedModel filter to SearchOptions; when set, GOBStore.Search returns only chunks with exact match (empty tags excluded) - Add MultiModel bool to StoreConfig (defaults to false) - Add Config.EmbedModelTag() helper that returns "provider/model" with empty guards for missing provider or model
- Add embedModelTag field and SetEmbedModelTag method to Indexer - Pass embedModelTag through createStoreChunks to stamp on every chunk - Wire in cli/watch.go: call SetEmbedModelTag when multi_model is true - Update existing indexer tests for new createStoreChunks signature
- Add embedModelTag field and SetEmbedModelFilter method to Searcher - Pass EmbedModel in SearchOptions for vector-only search - Filter allChunks by model tag in hybridSearch before text search - Wire in cli/search.go and mcp/server.go when multi_model is true
When multi_model is enabled, watch and search commands now check for chunks with empty EmbedModel before proceeding. If untagged chunks are found, the command exits with a message directing the user to run 'grepai migrate-model <provider/model>'. Shared helper countUntaggedChunks in cli/multi_model.go.
New command `grepai migrate-model <provider/model>` stamps all chunks with empty EmbedModel fields with the given tag. Metadata-only operation; no embedding API calls. Validates provider/model format contains "/". Currently supports only the GOB backend.
This was referenced Apr 23, 2026
Author
|
Design discussion for the multi-model feature (prerequisite for this PR) opened in #233. This command is part of the migration story described there. |
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
grepai migrate-model <provider/model>— a metadata-only command that stamps all chunks with an emptyEmbedModelfield with the given provider/model tag.Depends on #200 (multi-model index support). Design discussion: #233.
What it does
provider/modelformat (must contain/, both parts non-empty)store.multi_model: trueon an existing index so legacy chunks become searchable under the new per-model filteringFiles changed
cli/migrate_model.godocs/src/content/docs/backends/embedders.mdCHANGELOG.mdRelated
Split from #199 (closed) as requested by maintainer. Part 3 of 3:
Design and open questions tracked in #233.