Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 41 additions & 22 deletions .agents/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,43 +8,62 @@ paths:

# CodeGraph System Instructions

This project is backed by a custom **CodeGraph MCP server**. CodeGraph maintains a local Tree-sitter knowledge graph encompassing every symbol, edge, boundary, and file within this workspace. Reads operate at sub-millisecond speeds and deliver accurate structural insights that traditional text-based tools (like grep) cannot match.
This project is backed by a **CodeGraph MCP server** — a local tree-sitter
semantic graph of every symbol and call chain in the workspace. Reads are
sub-millisecond and return structural information grep cannot match.

---

## 🚨 CRITICAL CONSTRAINTS (Read First)

- **NEVER use generic text-search, grep, or file-reading tools** if a symbol, reference, or definition can be located using CodeGraph.
- **DO NOT double-check or re-verify** CodeGraph results with native file reads. Treat the knowledge graph as the absolute, single source of truth for codebase architecture.
- **Handle uninitialized states immediately:** If any CodeGraph tool returns a `"not initialized"` or missing index error, **STOP execution immediately** and instruct the user to run `codegraph init -i` in their terminal. Do not attempt to parse or scan the codebase manually to compensate.
- **Minimize token overhead:** Prefer targeted structural queries over dumping entire file contents into the context window.
- **NEVER use generic text-search, grep, or file-reading tools** when a
symbol, reference, or definition can be located with CodeGraph.
- **Do NOT re-verify** CodeGraph results with file reads — the graph is the
single source of truth for codebase structure.
- **Handle unbound sessions:** query tools refuse until the session is bound.
Call `codegraph_init {"path": ...}` (non-blocking, does NOT index), then
`codegraph_index {}` to build/refresh the index.
- **Minimize token overhead:** prefer targeted structural queries and
`id`-based lookups over dumping file contents into context.

---

## 🛠️ Tool Selection Guide

Always prefer `codegraph` tools for **structural** questions — tracing call hierarchies, mapping dependencies, determining definitions, and verifying signatures. Use standard filesystem tools *only* for literal text queries or applying actual code edits.
Prefer codegraph for **structural** questions. Use filesystem tools only for
literal text queries or applying edits.

| Intent / Question | Recommended MCP Tool |
| :--- | :--- |
| *"Where is symbol X defined?"* | `codegraph_search` |
| *"What callers invoke function Y?"* | `codegraph_callers` |
| *"What methods or functions does Y call?"* | `codegraph_callees` |
| *"What components or files will break if I modify Z?"* | `codegraph_impact` |
| *"Show me Y's exact signature and internal block"* | `codegraph_node` |
| *"Give me focused, aggregated context for this task"* | `codegraph_context` |
| *"What files exist under a specific path/ directory?"* | `codegraph_files` |
| *"Is the local knowledge graph healthy and active?"* | `codegraph_status` |
| *"Where is symbol X defined?"* | `codegraph_search_symbol` (contains/prefix/suffix/exact) |
| *"Show me this symbol by id / exact name"* | `codegraph_symbol` |
| *"What calls function Y?"* | `codegraph_callers` |
| *"What does Y call directly?"* | `codegraph_callees` |
| *"What breaks if I modify Z?"* | `codegraph_impact` |
| *"Show me Y's call chain"* | `codegraph_flow` |
| *"Find flows containing a pattern (loop + call)"* | `codegraph_search_flow` |
| *"Give me focused, aggregated context for a task"* | `codegraph_context` |
| *"Who calls the library function foo?"* | `codegraph_references` |
| *"What fields/methods does class C have?"* | `codegraph_class` |
| *"Which symbols are annotated @X?"* | `codegraph_search_by_annotation` |
| *"What files exist under path/?"* | `codegraph_files` |
| *"Is the index healthy?"* | `codegraph_status` |
| *"What does this MR change in the graph?"* | `codegraph_diff` |
| *"Simulate a flow's behavior with mocks"* | `codegraph_sandbox` / `codegraph_diff_simulate` |

---

## 💡 Rules of Thumb & Workflows
## 💡 Rules of Thumb

### 1. Unified Context Gathering
Do not chain manual searches and individual node inspections yourself. **`codegraph_context` is designed to perform aggregate lookups in a single call.** Execute it first when onboarding onto a new task or analyzing a localized bug.
1. **`codegraph_context` first** — it aggregates search + callers + callees
in one call; don't chain searches manually.
2. **Query impact before editing** — `codegraph_impact` pinpoints downstream
effects so you only touch relevant files.
3. **Trust the results** — AST-derived. If a lookup yields nothing, the
symbol is not in the active workspace index.
4. **Duplicate names** → the tool returns `ambiguous: true` with matches;
retry with the numeric `id` alone.

### 2. Defensive Token Preservation
Before updating an API route, a UI component, or a system utility, query `codegraph_impact` to pinpoint downstream effects. This ensures you only request and modify files strictly relevant to the current objective, preventing context window saturation.

### 3. Strict AST Reliance
Because CodeGraph parses the Abstract Syntax Tree (AST), its structural insights are guaranteed. If a symbol look-up yields no results, assume the symbol does not exist in the current active workspace index.
The full, always-current guide (timeout/resume protocol, response formats,
sandbox contracts) ships inside the binary as the server instructions — see
`docs/codegraph.md` in the CodeGraph repository.
12 changes: 12 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,19 @@
RUSTFLAGS: -D warnings

jobs:
sonarqube:
name: SonarQube
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarQube Scan
uses: SonarSource/sonarqube-scan-action@7006c4492b2e0ee0f816d36501671557c97f5995 # v8.1.0
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

clippy:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
name: clippy
runs-on: ubuntu-latest
steps:
Expand Down
21 changes: 21 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ members = [
"crates/codegraph-bench",
"crates/codegraph-installer",
"crates/codegraph",
"crates/codesmell",
]

[workspace.package]
Expand Down
200 changes: 39 additions & 161 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,26 @@ runs the MCP server. All reading/interacting goes through MCP tools.

Global flag `--path <dir>` overrides the workspace root.

## CodeSmell — convention linter for LLM agents

The workspace also ships [`codesmell`](docs/codesmell.md), a team-convention
linter built on CodeGraph facts. LLM agents read the conventions pack before
writing code and fix every reported violation afterwards — each violation
carries a `fix_hint`, so the agent repairs code instead of guessing:

```bash
codesmell init # write .codesmell/policy.toml
codesmell guide # conventions pack for the LLM / team
codesmell check # lint the repo (style, architecture, testing)
git diff | codesmell check --diff - # change-aware validation
```

Policies cover function size/parameter/nesting limits, naming conventions
(`*Service`, `Async` suffix), layer boundaries (`controller !-> repository`,
severity `blocking`), and required unit tests for changed business logic
(severity `required`), with per-area `[[override]]` scopes. See
[docs/codesmell.md](docs/codesmell.md) for the full policy schema.

## Supported languages

14 languages with full tree-sitter extraction + marker/chain walkers:
Expand Down Expand Up @@ -172,7 +192,7 @@ report, plus the session tools `codegraph_init` / `codegraph_deinit` /
| `codegraph_sandbox` | Compile a function group to machine code and run it against Rhai mocks |
| `codegraph_diff` | Draft report of what an MR/patch would change in the graph |

Read the [server instructions](crates/codegraph-mcp/src/server-instructions.md) that ship with the binary — they tell your agent when to reach for which tool.
Read the [server instructions](docs/codegraph.md) that ship with the binary — they tell your agent when to reach for which tool.

### `codegraph_search_flow` pattern examples

Expand Down Expand Up @@ -209,6 +229,7 @@ crates/
codegraph-mcp/ MCP server on the rmcp SDK (stdio + Streamable HTTP) + 27-tool dispatch, session-driven
codegraph-bench/ Benchmarks (criterion search benches, storage benches, codspeed)
codegraph/ CLI lifecycle (init/deinit/embed/serve --mcp) + watcher (notify + debounced full re-index)
codesmell/ Team-convention linter over in-memory CodeGraph facts (codesmell check / guide)
```

Pipeline:
Expand Down Expand Up @@ -237,175 +258,31 @@ A `.codegraph/` directory is created next to your project:
```
.codegraph/
db.sqlite SQLite (WAL mode, single file — entities + radix streams); db.lmdb/ directory when the LMDB backend is selected
config.toml Language toggles, walker filters, storage backend, embedding settings
config.toml Languages, effect rules, storage backend, sandbox, embedding settings
.gitignore Pre-filled so the index is never committed
version Codegraph version that created the directory
```

### config.toml example

```toml
# Language toggles (all 14 enabled by default)
[languages]
rust = true
go = true
python = true
typescript = true
javascript = true
java = true
c = true
cpp = true
csharp = true
ruby = true
php = true
scala = true
swift = true
lua = true

# Walker filters (same syntax as .gitignore)
[walker]
include = ["**/*"]
exclude = [
".git/**",
".codegraph/**",
"target/**",
"node_modules/**",
"*.min.js",
"*.lock"
]

# Storage backend — "sqlite" (default) | "lmdb" | "redis" | "memory" | "postgres" | "mysql"
[storage]
type = "sqlite"
# DSN override. Defaults: sqlite → sqlite://<root>/.codegraph/db.sqlite,
# lmdb → lmdb://<root>/.codegraph/db.lmdb (directory). Redis REQUIRES a dsn.
# dsn = "redis://localhost:6379"
# Postgres/MySQL use `dsns` (shard list) + `repo_id` — see below.

# Semantic search (vector KNN) — OFF by default. See "Semantic search" below.
[embedding]
# backend = "fastembed"
# model = "bge-small-en-v1.5"
# cache_dir = "~/.cache/codegraph/embeddings"
```

### Storage backends

The `[storage]` section selects where the index lives:
`config.toml` controls everything runtime-adjustable — highlights:

| `type` | Notes |
| Section | What it selects |
|---|---|
| `sqlite` | Default. Single-file `db.sqlite` (WAL) inside `.codegraph/`. |
| `lmdb` | Memory-mapped KV (`db.lmdb/` directory inside `.codegraph/`). Same local-first workflow, mmap-friendly for large indexes. Enabled by default in the `codegraph` binary. |
| `redis` | Requires an explicit `dsn` (e.g. `redis://localhost:6379`) — there is no sensible local default. |
| `memory` | Ephemeral in-process index; nothing is persisted. |
| `postgres` / `mysql` | Multi-tenant, sharded — see below. |

`dsn` (when set) overrides the derived default for any backend.

### Postgres / MySQL (multi-tenant, sharded)
| `[languages]` | `headers = "auto" \| "c" \| "cpp"` — how `.h` files route between the C/C++ grammars |
| `[[effect_rules]]` | Call-name → `EffectType` classification, evaluated before the built-in defaults (first match wins) |
| `[storage]` | Backend: `sqlite` (default), `lmdb`, `redis`, `memory`, `postgres`/`mysql` (multi-tenant, sharded by `repo_id`) |
| `[sandbox]` | Behavior-sandbox defaults: `mock_dirs`, `loop_cap`, `branch_policy` |
| `[embedding]` | Opt-in semantic search (`fastembed`/`hashing`, model, cache dir, sqlite-vss, execution provider) |

CodeGraph can store the index in PostgreSQL or MySQL instead of the local
SQLite file. Every table is partitioned by a leading `repo_id` (a `u64`
partition key), so each project root (`.codegraph/`) maps to its own
partition — re-indexing or deleting one repo never touches another. Sharding
is `repo_id % N` across the configured DSN list.
Ignore handling: `.gitignore` + `.codegraphignore` are honored by the walker;
files ≥ 4 MiB or non-UTF-8 are skipped.

Build with the `rdbms` feature (it is **on by default** for the `codegraph`
binary):

```bash
cargo build --features rdbms # default for `codegraph`
cargo build -p codegraph-mcp --features rdbms
```

`.codegraph/config.toml`:

```toml
[storage]
type = "postgres"
# type = "mysql"
# Shard DSNs — shard = repo_id % len(dsns). One entry = single shard.
dsns = [
"postgres://user:pass@db1:5432/codegraph",
"postgres://user:pass@db2:5432/codegraph",
]
# repo_id is generated automatically by `codegraph init` (self-heal) and
# written here. Do not edit it by hand.
# repo_id = 14028493579208694412
```

**Schema is applied manually** — the binary does not run migrations. Run the
SQL files from `sql/<engine>/` in order (currently `001-initial-schema.sql`
and `002-add-repos-registry.sql`) against every shard server before indexing:

```bash
psql "$DSN" -f sql/postgres/001-initial-schema.sql
psql "$DSN" -f sql/postgres/002-add-repos-registry.sql
# mysql:
# mysql "$DB" < sql/mysql/001-initial-schema.sql
# mysql "$DB" < sql/mysql/002-add-repos-registry.sql
```

Then `codegraph init` (CLI) or `codegraph_init` (MCP tool) generates the
`repo_id` and stores the index on the right shard automatically. See
`sql/README.md` for the full multi-tenant + sharding design.

### Semantic search (optional, opt-in)

Vector similarity search over symbol embeddings is **off by default** — no
embedding model runs unless you enable it in config. The release binary
already bundles the fastembed (ONNX sentence-transformer) backend, so
enabling it is config-only — no rebuild required:

1. Enable it in `.codegraph/config.toml`:

```toml
[embedding]
backend = "fastembed" # "hashing"/unset = off
model = "bge-small-en-v1.5" # 384-dim, default
cache_dir = "~/.cache/codegraph/embeddings" # global model cache (default)
# SQLite-only: point at a sqlite-vss (vector0/vss0) extension directory to
# run KNN through HNSW ANN inside the database:
# vss_extension = "~/.cache/codegraph/embeddings/vss"
# execution_provider = "coreml" # macOS hardware acceleration
```

2. Optionally pre-download the model so indexing works offline:

```sh
codegraph embed --model bge-small-en-v1.5
```

The `codegraph embed` subcommand is compiled in when the binary is built
with `--features fastembed`.

With embeddings enabled, `codegraph_search_symbol` gains the `match` modes
`"semantic"` (vector KNN — find symbols by similar/approximate names) and
`"hybrid"` (substring + semantic merged via Reciprocal Rank Fusion). Vectors
are persisted with the index, so restarts reuse them without re-embedding.

Notes:
- If the model fails to load (no network, missing ONNX runtime), opening the
index **errors out** — there is no silent fallback to a lexical baseline.
- On macOS you can build with `--features fastembed,apple-accel` to run
embeddings on the Apple Neural Engine / GPU via the CoreML execution
provider. That feature is macOS-only and fails to build elsewhere.

### C vs C++ headers (`.h`)

By default, `.h` files are resolved automatically:
- **C++ project** (`.cpp`/`.hpp` present, no `.c`) → parsed as C++
- **C project** (`.c` present, no C++ sources) → parsed as C
- **Mixed C/C++** → each `.h` inspected for C++ syntax (`namespace`, `class`, `template`, …)

Override in `.codegraph/config.toml`:
```toml
[languages]
headers = "auto" # "auto" (default), "c", or "cpp"
```
The full reference — every field, DSN derivation, sharding, the built-in
effect-rule table, embedding setup, and the file watcher — lives in
[docs/configuration.md](docs/configuration.md). Sandbox & mocking:
[docs/sandbox.md](docs/sandbox.md). MCP server & clients:
[docs/mcp.md](docs/mcp.md). All documentation is consolidated under
[docs/](docs/PLAN.md).

After changing this setting, run `codegraph init` (or call `codegraph_index` over MCP) to re-index headers.

## Why Rust?

Expand Down Expand Up @@ -457,6 +334,7 @@ cargo test -p codegraph-mcp
cargo test -p codegraph-sboxes # sandbox JIT: control flow + end-to-end traces
cargo test -p codegraph-bench # pipeline integration
cargo test -p codegraph-installer
cargo test -p codesmell # convention linter: policy, rules, fixtures
```

Feature flags on `codegraph-extract`:
Expand Down
Loading