Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
93ca85c
docs(concept): add reviewer redesign concept spec
sebastianwessel Jul 7, 2026
c10cee2
docs(concept): decide harness backbone — Vercel AI SDK, no own harness
sebastianwessel Jul 8, 2026
fca5d50
docs(concept): rework src structure for AI SDK decision; clarify conc…
sebastianwessel Jul 8, 2026
d8107a0
specs: graduate the structure & cleanup refactor plan (first concept→…
sebastianwessel Jul 8, 2026
5847cbb
specs+docs: add engineering contract and current-behavior baseline specs
sebastianwessel Jul 8, 2026
91ba95e
specs: add documentation standard; correct self-approved status to draft
sebastianwessel Jul 8, 2026
e36ae32
specs: migrate ADRs into domain specs; add decision log; nest by domain
sebastianwessel Jul 8, 2026
9adfc53
specs: split large foundation specs into per-responsibility nested files
sebastianwessel Jul 8, 2026
bf2bdea
specs: add readiness review report (needs_human_review)
sebastianwessel Jul 8, 2026
9953f1b
specs: apply EggAI decisions; record human approval
sebastianwessel Jul 8, 2026
c5d9ff7
docs: complete website-only migration (github-setup → website)
sebastianwessel Jul 8, 2026
0f66f52
researches: migrate agent-evaluation research from PR #149
sebastianwessel Jul 8, 2026
623d915
concept: add issue-tracker triage vs. concept (proposed verdicts)
sebastianwessel Jul 8, 2026
6af2cce
concept(triage): split #71.5 verdict — reject registry, adopt Marketp…
sebastianwessel Jul 8, 2026
0293076
concept: add eval-operations analysis (tracking, dataset growth, quic…
sebastianwessel Jul 8, 2026
7d3c5d7
concept(eval): add benchmark-landscape verdict + verified spike-port …
sebastianwessel Jul 8, 2026
8cb545b
Merge remote-tracking branch 'origin/main' into concept/reviewer-rede…
sebastianwessel Jul 8, 2026
8f62581
docs(concept,specs): align with merged main — A/B tooling shipped, ad…
sebastianwessel Jul 8, 2026
8d6ad1b
concept: deterministic PR destructuring — change-units, context packs…
sebastianwessel Jul 9, 2026
d8c8b36
concept+specs: rename forges->integrations; record divergent-duplicat…
sebastianwessel Jul 9, 2026
d263ef0
concept: propose configurable-agent as the port backend (challenges 08)
sebastianwessel Jul 10, 2026
bbab7ad
concept 11: add CA-R11 modular tool packs, accuracy-risk map, effort …
sebastianwessel Jul 10, 2026
e00d54d
concept 11: add head-to-head comparison vs the 08 AI-SDK-direct baseline
sebastianwessel Jul 10, 2026
aa80b6c
concept 11: future-fit analysis vs AI SDK v7 natives; add CA-R12 conf…
sebastianwessel Jul 10, 2026
ccad1e4
concept 11: context-handling analysis; add CA-R13 providerOptions pas…
sebastianwessel Jul 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .qualops/examples/github/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ permissions:

## Documentation

For detailed setup instructions, see [docs/github-setup.md](../../docs/github-setup.md).
For detailed setup instructions, see [the GitHub Action setup guide](https://github.com/eggai-tech/qualops/blob/main/website/src/content/docs/github-action/setup.md).

## Support

Expand Down
28 changes: 28 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# AGENTS.md

Guidance for AI coding agents working in this repository. This file is a pointer; the **binding engineering contract is [`CLAUDE.md`](CLAUDE.md)** — read it first and follow it exactly.

## Orientation

QualOps is an AI-powered pull-request reviewer (TypeScript, Node) shipped as a CLI and a GitHub Action.

- **Workflow:** `concept/ → specs/ → implementation → website/` (user docs). Implementation must follow the approved spec in [`specs/`](specs/README.md); never violate or unilaterally change a spec. User-facing docs live only in `website/` — there is no repo `docs/` folder.
- **Architecture & structure:** [`specs/architecture.md`](specs/architecture.md).
- **Current behavior the code implements:** [`specs/behavior/`](specs/behavior/).
- **In-flight direction (not yet built):** [`concept/`](concept/README.md).

## Non-negotiables (see CLAUDE.md for the full list)

1. Follow the relevant spec; if it's wrong or missing, fix the spec before coding.
2. No mocks, stubs, fakes, or placeholder implementations in production code — real implementations only.
3. One definition per concept: Zod schemas in `contracts/`, types inferred; validate at every boundary.
4. Layered imports only; every file has a clear home; no `utils/` dumping grounds; keep files small (≤ ~300 lines).
5. Proper `StructuredError` handling and redaction-safe logging; treat PR text and model output as untrusted.
6. Unit tests side-by-side (`*.test.ts`); integration/smoke in `tests/`; coverage ≥ 80%; happy **and** unhappy paths; test files excluded from the published package.
7. Keep the `website/` docs and `CHANGELOG.md` in sync with shipped behavior, in the same PR.

## Commands

```bash
npm run build npm test npm run test:integration npm run test:smoke npm run lint
```
84 changes: 84 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# CLAUDE.md — Engineering contract for QualOps

Rules for anyone (human or AI) changing this codebase. They are binding. When a rule and a request conflict, surface the conflict — don't silently break the rule.

## 0. The workflow: concept → spec → implementation → website (user docs)

- **`concept/`** — exploratory ideas, may conflict or be rejected. Not binding.
- **`specs/`** — approved, gap-free, aligned specifications. **The source of truth.**
- **implementation** — follows the relevant spec *precisely*. You may not violate a spec or change one on your own; specs are human-owned and human-reviewed. If a spec is wrong or a gap appears, stop and fix the spec first (or raise it), then implement.
- **`website/`** — user-facing documentation (Astro/Starlight); describes **only shipped behavior**, never planned behavior. No repo `docs/` folder.

Every change starts from a spec and ends with docs and tests in sync. See [`specs/README.md`](specs/README.md).

## 1. Architecture & file structure

- The code is layered: `contracts ← kernel ← platform ← llm ← domains/forges ← app`. Imports flow one way only. See [`specs/architecture.md`](specs/architecture.md) for the module map and the exclusivity rules (only `llm/backend` imports a model SDK; only `llm/boundary` parses model output; only `platform/env` reads `process.env`; only `platform/session-store` writes artifacts).
- **Every file has one clear home in the structure.** If you can't say which layer/module a file belongs to, the design is wrong — fix the design, don't dump it in a `utils/` bucket. New generic `utils/` folders are prohibited; shared helpers live in `kernel/`.
- **No very large files.** Target ≤ ~300 lines per file; treat > ~400 as a smell to split. A file should do one thing. Large files are almost always multiple responsibilities that belong in separate modules.
- Named exports only. **Functions by default**; classes only for genuine live state.
- **No new singletons.** Dependencies arrive explicitly via `RunContext` or parameters.

## 2. Types & validation

- **One definition per concept.** No parallel hand-written type for something a schema already describes.
- Shared shapes live in `contracts/` as **Zod schemas; TypeScript types are inferred** from them (`z.infer`), never written twice.
- **Validate at every boundary** — model output, config, tool I/O, forge payloads — with the shared schemas. Untrusted/model data is parsed and normalized at its boundary before any domain code sees it (model output: only in `llm/boundary`).

## 3. No fakes in shipped code

- **No mocks, stubs, fake implementations, placeholder returns, or `TODO`-stubbed functions in production code.** If something isn't implemented, it isn't merged. A function either does the real thing or does not exist.
- No hardcoded sample/demo data standing in for real logic. No "temporary" shortcuts that ship.
- Fakes belong **only** in tests (and only via the shared testing helpers).

## 4. Refactor discipline

- No spaghetti: keep concerns isolated, control flow flat, responsibilities single.
- **No duplication** — one home per piece of functionality (see the centralization map in `specs/architecture.md`). Before writing a helper, check `kernel/`; add it there if missing.
- Reduce complexity actively; leave code cleaner than you found it. Separate a *move* from an *edit* in distinct commits/PRs (see `specs/plans/refactor.md` §5 for the reviewable-refactor method).

## 5. Error handling & logging

- All failures normalize to the shared `StructuredError` (`{ code, category, recoverable, exitCode, details }`); no bare `throw`/`catch`-and-ignore, no swallowing.
- One exit point; telemetry is flushed before exit; the gate result drives the exit code.
- Logging is **redaction-safe by construction** — prompts, model output, tokens, and secrets must never reach a log or trace on the default path. Content capture is an explicit opt-in only.
- Log at the right level with structured fields; no `console.log`.

## 6. Security

- Treat all PR-derived text (titles, bodies, comments, diffs) and all model output as untrusted: sanitize before prompt assembly, never execute or eval it, always path-guard file access.
- Tool/shell execution goes through the sandbox with skip-pattern enforcement and secret redaction. Never enable a model backend's built-in tools; QualOps owns its tools.
- No secrets in code, logs, artifacts, or test fixtures.

## 7. Testing

- **Unit tests live side-by-side** with the code: `foo.ts` → `foo.test.ts` in the same folder.
- **Integration and smoke tests live in `tests/`** (they cross module or process boundaries or hit real providers).
- **Coverage ≥ 80%** (statements/lines/functions), enforced in CI. Coverage is a floor, not a goal.
- **Real tests, happy and unhappy paths.** Test observable behavior and error cases — not implementation detail, not trivial getters, no snapshot-only "tests." Every bug fix adds a test that would have caught it.
- **Test files must be excluded from the published package**: only `dist/` is published, and `*.test.ts` is excluded from the library build (`tsconfig.lib.json`). Verify the package tarball contains no test code.

## 8. Documentation

- User-facing docs live **only in the `website/`** (Astro/Starlight) — there is **no repo `docs/` folder**. The website tracks **shipped** behavior; if a change alters observable behavior, update the website **in the same PR**. Specs describe intent; the website describes reality; keep both true.
- The root **`README.md` stays short, crisp, and precise** (no fixed line limit): a one/two-paragraph intro, a runnable quick start (install, credential, zero-config run, Action snippet), and a **table of contents that links into the website**. Depth lives on the website, never inlined into the README.
- The website is **concise, current, and nested by reader journey** (overview → getting started → understanding → configuration → customizing → guides → reference → troubleshooting); one topic per page, one home per fact, no dead links, no "coming soon" stubs.
- Full standard and acceptance criteria: [`specs/documentation.md`](specs/documentation.md).
- Update `CHANGELOG.md` for any behavior-affecting change.

## 9. Dependencies

- Minimize them. A new runtime dependency needs justification (what it replaces, its license, its footprint). Prefer the small hand-rolled utilities already in `kernel/`. The model backbone is the Vercel AI SDK (`ai` + `@ai-sdk/*`); see [`concept/08-harness-decision.md`](concept/08-harness-decision.md).

## Commands

```bash
npm run build # tsc lib build (excludes tests) + alias + copy prompts/agents
npm test # unit tests (jest)
npm run test:integration # integration tests
npm run test:smoke # provider smoke tests (needs creds)
npm run test:evals # eval harness
npm run lint # eslint
```

Full engineering standards and the current-behavior/architecture specs are in [`specs/`](specs/README.md).
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ QualOps ships on **two tiers**:
- **`@beta`** — pre-release track for internal users. Consumed as `uses: eggai-tech/qualops@beta` or `npm install @eggai/qualops@beta`.
- **`@stable`** — production track for external clients. Consumed as `uses: eggai-tech/qualops@stable` or the default `npm install @eggai/qualops`.

`beta` and `stable` are **movable lightweight git tags**. They are moved automatically by CI on every release or promotion. **Never move them by hand.** If you find them out of sync, see [`docs/tdr/0001-release-process.md`](docs/tdr/0001-release-process.md).
`beta` and `stable` are **movable lightweight git tags**. They are moved automatically by CI on every release or promotion. **Never move them by hand.** If you find them out of sync, see [`specs/operations/release.md`](specs/operations/release.md).

### Cutting a beta release

Expand Down Expand Up @@ -152,7 +152,7 @@ Pre-release suffix: `-beta.N` (e.g., `0.3.0-beta.1`, `0.3.0-beta.2`). The `-rc.N
- **A release workflow fails partway.** Check the auto-generated failure issue — it lists which stage failed. If `publish-npm` succeeded but a later stage failed, re-running the workflow is safe (later stages are idempotent). If `publish-npm` itself failed, check npm for partial state before retrying.
- **`Create Release PR` left a `release/v*` branch behind after a failure.** Delete it manually with `git push origin --delete release/vX.Y.Z` and re-run.

The full design rationale lives in [`docs/tdr/0001-release-process.md`](docs/tdr/0001-release-process.md).
The full design rationale lives in [`specs/operations/release.md`](specs/operations/release.md).

## Code Style

Expand Down
Loading
Loading