feat: distill learned rules from human feedback#35
Conversation
Add migration 0003 with repo_learned_rules table (one row per repo, upserted on re-distillation). Add learned-rules.ts with functions for getting/storing rules and fetching feedback for distillation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Distills accumulated human feedback into concise team preference rules via LLM. Uses generateText() for dual-API support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add distillThreshold config (env DISTILL_THRESHOLD, default 5) - Inject learned rules into system prompt after CANON.md section - Fetch learned rules in pr-handler and pass to PromptContext - Trigger fire-and-forget distillation in feedback-handler when accumulated feedback reaches threshold Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Extend feedback-handler tests with distillation trigger tests - Extend prompt-builder tests for learned rules section - Extend pr-handler tests for learnedRules passing - Add distillThreshold to all test makeConfig() functions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| category: string; | ||
| severity: string; | ||
| finding: string; | ||
| existing_pattern: string; |
There was a problem hiding this comment.
🟡 Naming Convention · medium · confidence: 95%
DB row interface uses snake_case property names, while existing TypeScript interfaces in the repo use camelCase and map snake_case at query-read time.
Existing pattern: Use camelCase in TS interfaces/types even when SQL columns are snake_case (e.g., map row.file_path -> path, row.content_hash -> contentHash).
Reference: src/intelligence/embeddings.ts
| existing_pattern: string; | |
| export interface FeedbackRow { | |
| category: string; | |
| severity: string; | |
| finding: string; | |
| existingPattern: string; | |
| replyBody: string; | |
| replyAuthor: string; | |
| } | |
| // In query: | |
| SELECT | |
| f.category, | |
| f.severity, | |
| f.finding, | |
| f.existing_pattern AS "existingPattern", | |
| rf.reply_body AS "replyBody", | |
| rf.reply_author AS "replyAuthor" |
| `Category: ${row.category}`, | ||
| `Severity: ${row.severity}`, | ||
| `Canon's finding: ${row.finding}`, | ||
| `Existing pattern referenced: ${row.existing_pattern}`, |
There was a problem hiding this comment.
🟡 Pattern Violation · medium · confidence: 93%
Uses snake_case fields from FeedbackRow directly in app-layer code, deviating from the repo’s camelCase object-property pattern.
Existing pattern: Application-layer object properties are camelCase; SQL snake_case is translated at DB boundary.
Reference: src/intelligence/embeddings.ts
| `Existing pattern referenced: ${row.existing_pattern}`, | |
| `Existing pattern referenced: ${row.existingPattern}`, | |
| `Developer reply (${row.replyAuthor}): ${row.replyBody}`, |
Map snake_case SQL columns to camelCase at the query boundary using column aliases, following existing repo conventions (e.g. embeddings.ts). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the feedback distillation pipeline, DISTILL_THRESHOLD config, new files (learned-rules.ts, distiller.ts), and updated test counts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/canon review |
| @@ -1,5 +1,11 @@ | |||
| import type { Context } from "probot"; | |||
| import { storeFeedback } from "../db/review-store"; | |||
| import { getFeedbackCount, getStoredFeedbackCount, getFeedbackForDistillation, storeLearnedRules } from "../db/learned-rules"; | |||
There was a problem hiding this comment.
⚪ Style Inconsistency · low · confidence: 95%
Import list exceeds the repository’s established multiline import formatting pattern used for long named imports.
Existing pattern: Long named imports are split across multiple lines with one symbol per line for readability.
Reference: src/github/init-handler.ts
| import { getFeedbackCount, getStoredFeedbackCount, getFeedbackForDistillation, storeLearnedRules } from "../db/learned-rules"; | |
| import { | |
| getFeedbackCount, | |
| getStoredFeedbackCount, | |
| getFeedbackForDistillation, | |
| storeLearnedRules, | |
| } from "../db/learned-rules"; |
Summary
Changes
repo_learned_rulestable (one row per repo, upserted)src/db/learned-rules.ts: DB access functions (get/store rules, fetch feedback for distillation, count helpers)src/intelligence/distiller.ts: Sends feedback to LLM, returns max 10 bullet-point rulessrc/config/index.ts: NewdistillThresholdconfig (envDISTILL_THRESHOLD, default 5)src/review/prompt-builder.ts: NewlearnedRulesfield inPromptContext, injected after CANON.md sectionsrc/github/pr-handler.ts: Fetches learned rules from DB and passes to prompt contextsrc/github/feedback-handler.ts: Triggers distillation after feedback storage when threshold is reachedTest plan
npm run lint— type-check passesnpm test— all 330 tests pass (20 new tests added)🤖 Generated with Claude Code