Skip to content

feat(store): hierarchical database path resolution with per-project support - #257

Open
Acharnite wants to merge 3 commits into
rtk-ai:developfrom
Acharnite:feat/hierarchical-db-path
Open

feat(store): hierarchical database path resolution with per-project support#257
Acharnite wants to merge 3 commits into
rtk-ai:developfrom
Acharnite:feat/hierarchical-db-path

Conversation

@Acharnite

Copy link
Copy Markdown

Summary

Replace the simple --db-or-default path logic with a 6-level hierarchical database path resolution chain. This eliminates the need for wrapper scripts or hardcoded paths when working across multiple projects.

Resolution chain (highest priority first)

Level Source Example
1 --db CLI flag icm --db /tmp/test.db store ...
2 ICM_DB env var ICM_DB=/tmp/icm.db icm recall ...
3 Global config [store].path ~/.config/icm/config.toml
4 Project-local .icm/config.toml [store].path Git root auto-detection
5 Project-local .icm/memories.db Auto-detected at git root (if file exists)
6 Platform default data directory ~/.local/share/icm/memories.db

Key changes

crates/icm-cli/src/main.rs (+165/−19):

  • New resolve_db_path(cli_db, cfg) → PathBuf with the 6-level chain
  • New detect_project_root() → Option<PathBuf> using git rev-parse --show-toplevel
  • icm init --per-project now creates .icm/config.toml with [store] path = "memories.db"
  • icm config shows the full resolution chain with source tracing (resolved path, config path, env var, project root, .icm/ state)
  • open_store() simplified from open_store(db: Option<PathBuf>) to open_store(db: PathBuf) — resolution happens before the store is opened

config/default.toml (+11):

  • Documents the full 6-level resolution order in the config header
  • Explains icm init --per-project creates project-local config

OpenCode plugin improvements (plugins/opencode-icm.ts, +17/−10)

  • Added NOISE_TOOLS set to skip extraction for Edit/Write/Question/todowrite calls (reduces noise)
  • Increased EXTRACT_EVERY from 3 to 10 (reduces extraction frequency)
  • Added cwd parameter to all ICM subprocess calls so the plugin respects per-project databases
  • Fixed recall-project to use named flags (--project, --limit) instead of positional args

Documentation

  • docs/architecture.md: New Database Path Resolution section with the full algorithm, priority flow diagram, and icm config output example
  • docs/guide.md: Per-project setup instructions and resolution priority table
  • README.md: Already updated with --per-project setup, ICM_DB env var mention, and per-project storage section

Backward compatibility

Fully backward compatible. The existing --db flag still works at the same priority. The default platform path (level 6) is unchanged. Existing setups see no behavioral difference.

Closes the gap where users had to set --db on every invocation or write wrapper scripts to use per-project databases.

Acharnite added 3 commits May 26, 2026 16:40
Add 6-level database path resolution chain that eliminates the need for
wrapper scripts or hardcoded paths:

1. --db CLI flag (highest priority)
2. ICM_DB environment variable (new)
3. Global config [store].path (was parsed but unused — now wired up)
4. Project-local .icm/config.toml [store].path (auto-detected via git root)
5. Project-local .icm/memories.db (auto-detected via git root, if file exists)
6. Default platform data directory (unchanged fallback)

Also adds:
- detect_project_root() helper using git rev-parse --show-toplevel
- icm init --per-project now creates .icm/config.toml with project-local DB
- icm config shows full resolution chain with source tracing
- Updated config/default.toml to document resolution order
Change icm init --per-project to write [store] path = ".icm/memories.db"
instead of "memories.db", so the database lives inside .icm/ alongside
config.toml. Update README with --per-project docs.
Add comprehensive docs for the 6-level database path resolution chain:
- resolve_db_path() algorithm and priority flow
- detect_project_root() and git-aware auto-detection
- Per-project .icm/ setup
- icm config display with source tracing

Also update the user guide with per-project setup instructions
and add noise-tool filtering to the opencode plugin.
@pszymkowiak

Copy link
Copy Markdown
Contributor

Thanks for this, @Acharnite! Could you retarget this PR to develop (our integration branch — PRs to main fail the target-branch check) and rebase on the latest develop? The per-project .icm/config.toml + git-root resolution is a nice complement to the worktree project-detection fix we just merged (#235). Happy to review once it's rebased and CI is green.

@Acharnite
Acharnite changed the base branch from main to develop June 21, 2026 22:31

@pszymkowiak pszymkowiak left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this — per-project memory (#321) is a genuinely useful feature and the resolution hierarchy (--db\$ICM_DB → global config → project .icm/config.toml → project .icm/memories.db → default) is a sensible design. A few things need addressing before it can land:

1. Blocking — the branch is stale (pre-#301 multi-backend refactor)

develop has since moved open_store to the runtime-selected backend enum:

// develop
fn open_store(db: Option<PathBuf>, embedding_dims: usize) -> Result<Store>   // not SqliteStore

This PR reintroduces -> Result<SqliteStore> and the old open_store(db: PathBuf, …) signature, so it won't compile / will conflict. It also predates open_store_readonly (#263/#319) and open_maintenance (#313), which now resolve the same DB path — they'd need the new resolve_db_path wired in too, or per-project resolution silently won't apply to --read-only serve, doctor, and repair. Please rebase onto current develop and re-thread the resolution through all the openers.

2. Security — untrusted project .icm/config.toml can redirect where ICM writes

resolve_db_path auto-reads store.path from a project-local .icm/config.toml and honors absolute paths (if Path::new(path_str).is_absolute()). That means cloning/opening any repo that ships a .icm/config.toml can point ICM's SQLite DB at an arbitrary location on my machine — ICM then creates and writes a DB file there (potential overwrite / data loss / writing outside the project). Please either:

  • restrict project-config paths to within the project root (reject absolute paths and .. escapes), and/or
  • gate project-config auto-discovery behind an explicit opt-in.
    The .icm/memories.db-if-exists branch (4b) is fine since it's fixed under the project.

3. Reuse existing git detection

The codebase already detects the repo via project_from_path (git rev-parse --git-common-dir) for project tagging. detect_project_root adds a second git rev-parse --show-toplevel spawn on every resolution. Consider consolidating (and note this runs on latency-sensitive hook paths — walking up for .git/.icm in pure Rust avoids a subprocess and the hard git dependency).

Nits

  • Tests: please add coverage for the resolution order (each tier), the absolute-path rejection, and the no-git fallback.
  • config/default.toml doc comment references src/main.rs — the crate path is crates/icm-cli/src/main.rs.

Happy to re-review once it's rebased and the path-injection is bounded — the feature itself is welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants