feat(store): hierarchical database path resolution with per-project support - #257
feat(store): hierarchical database path resolution with per-project support#257Acharnite wants to merge 3 commits into
Conversation
Add 6-level database path resolution chain that eliminates the need for wrapper scripts or hardcoded paths: 1. --db CLI flag (highest priority) 2. ICM_DB environment variable (new) 3. Global config [store].path (was parsed but unused — now wired up) 4. Project-local .icm/config.toml [store].path (auto-detected via git root) 5. Project-local .icm/memories.db (auto-detected via git root, if file exists) 6. Default platform data directory (unchanged fallback) Also adds: - detect_project_root() helper using git rev-parse --show-toplevel - icm init --per-project now creates .icm/config.toml with project-local DB - icm config shows full resolution chain with source tracing - Updated config/default.toml to document resolution order
Change icm init --per-project to write [store] path = ".icm/memories.db" instead of "memories.db", so the database lives inside .icm/ alongside config.toml. Update README with --per-project docs.
Add comprehensive docs for the 6-level database path resolution chain: - resolve_db_path() algorithm and priority flow - detect_project_root() and git-aware auto-detection - Per-project .icm/ setup - icm config display with source tracing Also update the user guide with per-project setup instructions and add noise-tool filtering to the opencode plugin.
|
Thanks for this, @Acharnite! Could you retarget this PR to |
pszymkowiak
left a comment
There was a problem hiding this comment.
Thanks for this — per-project memory (#321) is a genuinely useful feature and the resolution hierarchy (--db → \$ICM_DB → global config → project .icm/config.toml → project .icm/memories.db → default) is a sensible design. A few things need addressing before it can land:
1. Blocking — the branch is stale (pre-#301 multi-backend refactor)
develop has since moved open_store to the runtime-selected backend enum:
// develop
fn open_store(db: Option<PathBuf>, embedding_dims: usize) -> Result<Store> // not SqliteStoreThis PR reintroduces -> Result<SqliteStore> and the old open_store(db: PathBuf, …) signature, so it won't compile / will conflict. It also predates open_store_readonly (#263/#319) and open_maintenance (#313), which now resolve the same DB path — they'd need the new resolve_db_path wired in too, or per-project resolution silently won't apply to --read-only serve, doctor, and repair. Please rebase onto current develop and re-thread the resolution through all the openers.
2. Security — untrusted project .icm/config.toml can redirect where ICM writes
resolve_db_path auto-reads store.path from a project-local .icm/config.toml and honors absolute paths (if Path::new(path_str).is_absolute()). That means cloning/opening any repo that ships a .icm/config.toml can point ICM's SQLite DB at an arbitrary location on my machine — ICM then creates and writes a DB file there (potential overwrite / data loss / writing outside the project). Please either:
- restrict project-config paths to within the project root (reject absolute paths and
..escapes), and/or - gate project-config auto-discovery behind an explicit opt-in.
The.icm/memories.db-if-exists branch (4b) is fine since it's fixed under the project.
3. Reuse existing git detection
The codebase already detects the repo via project_from_path (git rev-parse --git-common-dir) for project tagging. detect_project_root adds a second git rev-parse --show-toplevel spawn on every resolution. Consider consolidating (and note this runs on latency-sensitive hook paths — walking up for .git/.icm in pure Rust avoids a subprocess and the hard git dependency).
Nits
- Tests: please add coverage for the resolution order (each tier), the absolute-path rejection, and the no-git fallback.
config/default.tomldoc comment referencessrc/main.rs— the crate path iscrates/icm-cli/src/main.rs.
Happy to re-review once it's rebased and the path-injection is bounded — the feature itself is welcome.
Summary
Replace the simple
--db-or-default path logic with a 6-level hierarchical database path resolution chain. This eliminates the need for wrapper scripts or hardcoded paths when working across multiple projects.Resolution chain (highest priority first)
--dbCLI flagicm --db /tmp/test.db store ...ICM_DBenv varICM_DB=/tmp/icm.db icm recall ...[store].path~/.config/icm/config.toml.icm/config.toml[store].path.icm/memories.db~/.local/share/icm/memories.dbKey changes
crates/icm-cli/src/main.rs(+165/−19):resolve_db_path(cli_db, cfg) → PathBufwith the 6-level chaindetect_project_root() → Option<PathBuf>usinggit rev-parse --show-toplevelicm init --per-projectnow creates.icm/config.tomlwith[store] path = "memories.db"icm configshows the full resolution chain with source tracing (resolved path, config path, env var, project root, .icm/ state)open_store()simplified fromopen_store(db: Option<PathBuf>)toopen_store(db: PathBuf)— resolution happens before the store is openedconfig/default.toml(+11):icm init --per-projectcreates project-local configOpenCode plugin improvements (
plugins/opencode-icm.ts, +17/−10)NOISE_TOOLSset to skip extraction for Edit/Write/Question/todowrite calls (reduces noise)EXTRACT_EVERYfrom 3 to 10 (reduces extraction frequency)cwdparameter to all ICM subprocess calls so the plugin respects per-project databasesrecall-projectto use named flags (--project,--limit) instead of positional argsDocumentation
docs/architecture.md: NewDatabase Path Resolutionsection with the full algorithm, priority flow diagram, andicm configoutput exampledocs/guide.md: Per-project setup instructions and resolution priority tableREADME.md: Already updated with--per-projectsetup,ICM_DBenv var mention, and per-project storage sectionBackward compatibility
Fully backward compatible. The existing
--dbflag still works at the same priority. The default platform path (level 6) is unchanged. Existing setups see no behavioral difference.Closes the gap where users had to set
--dbon every invocation or write wrapper scripts to use per-project databases.