constructorfabric · ktursunov · Jun 26, 2026 · Jun 26, 2026
@@ -1,12 +1,12 @@
 ---
-name: metric-e2e-test
-description: "Author and validate declarative YAML e2e tests for analytics metrics (src/ingestion/tests/e2e/specs/*.test.yaml). Use when asked to write/scaffold/validate an e2e test for a metric, seed bronze data for a test, add a fixture for a dashboard metric, or check a *.test.yaml. Covers schemas/, templates/, $ref+sibling composition, bronze records with duplicates, the batch endpoint POST /v1/metrics/queries, and expect rules (in / mongo-style find / equal subset / CEL assert)."
+name: metric-test
+description: "Author and validate declarative YAML tests for analytics metrics (src/ingestion/tests/e2e/metrics/*.test.yaml). Use when asked to write/scaffold/validate a test for a metric, seed bronze data for a test, add a fixture for a dashboard metric, or check a *.test.yaml. Covers schemas/, templates/, $ref+sibling composition, bronze records with duplicates, the batch endpoint POST /v1/metrics/queries, and expect rules (in / mongo-style find / equal subset / CEL assert)."
 disable-model-invocation: false
 user-invocable: true
 allowed-tools: Bash, Read, Write, Edit, Glob, Grep
 ---
 
-# Author a metric e2e test (declarative YAML)
+# Author a metric test (declarative YAML)
 
 This skill writes and validates `*.test.yaml` fixtures that drive the full
 `bronze → dbt silver → gold view → analytics-api` path and assert the result.
@@ -22,15 +22,15 @@ no need to load them every time:
 
 ## Commands
 
-- `/metric-e2e-test create <name> --metric <uuid> --tables <t1,t2>` — scaffold a new `<name>.test.yaml` (+ any missing `schemas/` and `templates/`).
-- `/metric-e2e-test validate <path>` — resolve refs, schema-validate records, lint `cases`/`expect` without running ClickHouse.
+- `/metric-test create <name> --metric <uuid> --tables <t1,t2>` — scaffold a new `<name>.test.yaml` (+ any missing `schemas/` and `templates/`).
+- `/metric-test validate <path>` — resolve refs, schema-validate records, lint `cases`/`expect` without running ClickHouse.
 
-(Plain prose like "write an e2e test for the emails-sent metric" triggers the same flow.)
+(Plain prose like "write a test for the emails-sent metric" triggers the same flow.)
 
 ## File layout
 
 ```
-src/ingestion/tests/e2e/specs/
+src/ingestion/tests/e2e/metrics/
   schemas/<db>.<table>.yaml      # one JSON schema per bronze table (all real columns)
   templates/<group>.yaml         # reusable records (people, m365_email, …)
   <name>.test.yaml               # ONE metric per file (discovered by the *.test.yaml suffix)
@@ -157,7 +157,7 @@ disagree with the analytics team (org_unit) — irrelevant to these assertions.)
 
 ### `assert` (CEL) bindings
 
-Assembled in `e2e_lib/expect_engine.py::evaluate_case` (the `bindings` dict),
+Assembled in `lib/expect_engine.py::evaluate_case` (the `bindings` dict),
 converted to CEL in `_eval_cel`:
 
 | Binding | Value | Present when |
@@ -178,7 +178,7 @@ For exact / `null`, use `equal` (Python `==`), not `assert`. CEL macros availabl
 
 `metric_date` bounds are **inclusive on both ends**: `metric_date ge '<lo>' and metric_date le '<hi>'`
 includes rows ON `<lo>` and ON `<hi>`. When a metric is date-windowed, prove the bounds
-with a dedicated spec (see `specs/collab_emails_read.test.yaml`):
+with a dedicated spec (see `metrics/collab_emails_read.test.yaml`):
 
 - **Boundary-value (BVA):** seed rows one-day-BEFORE the lower bound (must be excluded),
   AT the lower bound (included), AT the upper bound (included), and one-day-AFTER the upper
@@ -274,18 +274,18 @@ with a dedicated spec (see `specs/collab_emails_read.test.yaml`):
 
 ```bash
 cd src/ingestion/tests/e2e
-ls specs/*.test.yaml                       # list existing tests
-./e2e.sh test                              # run all tests (specs/ + meta/)
+ls metrics/*.test.yaml                       # list existing tests
+./e2e.sh test                              # run all tests (metrics/ + meta/)
 ./e2e.sh test -k <name>                    # run one test by name
 ./e2e.sh test -k <name> -v                 # verbose (per-step log)
 ./e2e.sh down                              # tear down the e2e compose stack + volumes (full reset)
 ```
 
-`<name>` is the file stem (e.g. `collab_emails_sent` for `specs/collab_emails_sent.test.yaml`).
+`<name>` is the file stem (e.g. `collab_emails_sent` for `metrics/collab_emails_sent.test.yaml`).
 
 Warm re-runs are fine. Isolation is **per-test**: each test first `TRUNCATE`s only the
 tables the PRIOR test recorded in a per-test ledger (`CHSeeder.TouchedLedger`;
-`specs/test_fixtures.py`, `e2e_lib/ch_seeder.py`), then seeds its own. The rig auto-records
+`metrics/test_fixtures.py`, `lib/ch_seeder.py`), then seeds its own. The rig auto-records
 the built **staging** and **silver** models too (not just bronze) — staging especially,
 because silver reads staging via the `union_by_tag` macro, so an un-truncated prior staging
 row would contaminate the silver rebuild. On TOP of that, a one-time session-start truncate
@@ -294,7 +294,7 @@ row would contaminate the silver rebuild. On TOP of that, a one-time session-sta
 makes WARM re-runs deterministic (CI starts fresh). `./e2e.sh down` is the e2e compose
 teardown (not a deploy), for when you want a fully clean ClickHouse.
 
-To create a new test, use `/metric-e2e-test create` or hand-author `specs/<name>.test.yaml`
+To create a new test, use `/metric-test create` or hand-author `metrics/<name>.test.yaml`
 as above. There is no `./e2e.sh new` — the old CSV-rig scaffolder was removed when the
 declarative `*.test.yaml` rig replaced the CSV rig.
 
@@ -321,7 +321,7 @@ connector that isn't there yet:
   reliably advance, so cargo relinked a stale object and the binary silently
   lacked new SeaORM migrations (symptoms: `query_ref`/catalog changes have no
   effect, a `find` matches 0 rows, `size(items)` off by your new key). FIXED in
-  `e2e_lib/analytics_api.py::build` — it now `touch`es the analytics-api crate
+  `lib/analytics_api.py::build` — it now `touch`es the analytics-api crate
   sources before `cargo build`, forcing a recompile every run (~1-2 min, only
   that crate). So a plain `./e2e.sh test` picks up new migrations now; you should
   NOT need `down -v` for this. If you still suspect a stale binary, confirm by

@@ -1,7 +1,7 @@
 name: E2E — Bronze to API
 
 # End-to-end transformation tests covering bronze → dbt → ClickHouse
-# migration views → analytics-api HTTP response → CSV assert.
+# migration views → analytics-api HTTP response → expect rules.
 #
 # Everything (Python, Rust, dbt, pytest) runs inside the runner image
 # defined at src/ingestion/tests/e2e/compose/Dockerfile.runner. The same
@@ -28,7 +28,7 @@ on:
 
 env:
   # NB: forbids developers from running `pytest --update-snapshots` in CI.
-  # The guard lives in src/ingestion/tests/e2e/fixtures/test_fixtures.py.
+  # The guard lives in src/ingestion/tests/e2e/metrics/test_fixtures.py.
   CI: "true"
 
 jobs:
@@ -55,14 +55,10 @@ jobs:
       - name: Build runner image
         run: ./e2e.sh build
 
-      # ─── Pre-pull data plane images in parallel with whatever else ──────
-      - name: Pre-pull ClickHouse + MariaDB
-        run: |
-          docker pull clickhouse/clickhouse-server:24.8.4.13 &
-          docker pull mariadb:11.4.4 &
-          wait
-
       # ─── Run the suite inside the runner ────────────────────────────────
+      # `./e2e.sh test` (docker compose run runner) starts the clickhouse +
+      # mariadb `depends_on` services — compose pulls their images on demand
+      # and waits for `service_healthy` — so no explicit image pre-pull is needed.
       # Serial on purpose: the session rig is not xdist-safe yet (see
       # conftest.py — per-worker analytics-api spawns race SeaORM migrations
       # in the shared MariaDB, non-primary workers don't wait for ClickHouse

@@ -491,7 +491,7 @@ metric ids for a genuinely new section):
   `source_tags: ["<slug>"]`) + a product-default `metric_threshold`. Model it on
   `m20260620_000002_seed_wiki_catalog.rs`. Register in `mod.rs`.
 
-Then prove the whole chain with a `/metric-e2e-test` fixture (see that skill) —
+Then prove the whole chain with a `/metric-test` fixture (see that skill) —
 seed bronze, query the bullet metric, assert your `metric_key`'s value. Adding a
 key to a shared section raises its `size(items)`, so bump any sibling test that
 asserts the old count.

@@ -237,7 +237,7 @@ so and skip this section.
 - [ ] The gold `insight.<section>_bullet_rows` view has a branch `FROM silver.class_<X> WHERE data_source = 'insight_<snake>'` emitting the connector's `metric_key`(s). (`data_source` literal == what the silver model SELECTs.)
 - [ ] EVERY bullet `query_ref` that should show the key was re-set in a NEW append-only SeaORM migration — the key appears in both the `sumIf(... metric_key='<k>') AS <k>_v` list AND the `ARRAY JOIN [('<k>', <k>_v), …]` unpivot. A section typically has several copies (IC `…0012`-style, Team, member-values `…0041`, dept-dist `…0045`); each must be updated for the surface it backs. Base the new SQL on the LATEST migration that set that id (grep the metric hex id), and register the migration in `migration/mod.rs`.
 - [ ] A `metric_catalog` row + product-default `metric_threshold` for `<section>_bullet_rows.<metric_key>` exists (new append-only migration, `source_tags: ["<slug>"]`, registered in `mod.rs`).
-- [ ] An e2e fixture (`/metric-e2e-test`) seeds bronze and asserts the metric_key's value end-to-end; any sibling test asserting the section's `size(items)` was bumped for the new key.
+- [ ] An e2e fixture (`/metric-test`) seeds bronze and asserts the metric_key's value end-to-end; any sibling test asserting the section's `size(items)` was bumped for the new key.
 
 ### Credentials Template
 - [ ] `credentials.yaml.example` lists all required fields

@@ -55,7 +55,7 @@ The DESIGN is decomposed into 8 features. The order is deliberate: a foundation
 - **Depends On**: None
 
 - **Scope**:
-  - Directory layout: `src/ingestion/tests/e2e/{compose,fixtures,e2e_lib,meta}`
+  - Directory layout: `src/ingestion/tests/e2e/{compose,fixtures,lib,meta}`
   - `compose/docker-compose.yml` with ClickHouse 24.x + MariaDB 11.x pinned to production parity, ports on `127.0.0.1` only, randomized credentials per run
   - `conftest.py` with `pytest_sessionstart` hook orchestrating compose-up + healthcheck wait
   - `migration-applier` logic (small enough to live inline, not as a separate feature): `clickhouse-client --multiquery` over `src/ingestion/scripts/migrations/*.sql` in lexical order
@@ -137,7 +137,7 @@ The DESIGN is decomposed into 8 features. The order is deliberate: a foundation
 
 - **Domain Model Entities**: `Fixture`, `SpecYaml`
 
-- **API**: (internal Python module — `e2e_lib.fixture_loader.load(path) -> Fixture`)
+- **API**: (internal Python module — `lib.fixture_loader.load(path) -> Fixture`)
 
 - **Data**: (read-only filesystem)
 

@@ -136,13 +136,13 @@ The test file is what the test asserts on. There is no "regenerate from current
 
 - [ ] `p1` - **ID**: `cpt-bronze-to-api-e2e-principle-record-composition`
 
-A bronze row (or a template) is a field map that may carry a `$ref: "<file>#/<json-pointer>"` to inherit from another record; sibling keys override the base (closest wins). Reusable people/source records live in `specs/templates/*.yaml`. A test spells out only the fields it exercises; everything else is inherited. This keeps a test small while the seeded row stays complete.
+A bronze row (or a template) is a field map that may carry a `$ref: "<file>#/<json-pointer>"` to inherit from another record; sibling keys override the base (closest wins). Reusable people/source records live in `metrics/templates/*.yaml`. A test spells out only the fields it exercises; everything else is inherited. This keeps a test small while the seeded row stays complete.
 
 #### The table schema is the source of truth for a row's shape
 
 - [ ] `p1` - **ID**: `cpt-bronze-to-api-e2e-principle-schema-is-truth`
 
-Per-table JSON schemas live in `specs/schemas/<db>.<table>.yaml` (the file stem is the full bronze table name, e.g. `bronze_m365.email_activity.yaml`) and are resolved by that name (the `bronze` key IS the table). After `$ref` resolution a row is padded with every missing schema column as `null` and validated; `additionalProperties:false` catches a misspelled column. Base templates carry the full column set (including the non-nullable `_airbyte_*` CDK columns, which transforms such as `insight.people`'s `argMax(..., _airbyte_extracted_at)` depend on).
+Per-table JSON schemas live in `metrics/schemas/<db>.<table>.yaml` (the file stem is the full bronze table name, e.g. `bronze_m365.email_activity.yaml`) and are resolved by that name (the `bronze` key IS the table). After `$ref` resolution a row is padded with every missing schema column as `null` and validated; `additionalProperties:false` catches a misspelled column. Base templates carry the full column set (including the non-nullable `_airbyte_*` CDK columns, which transforms such as `insight.people`'s `argMax(..., _airbyte_extracted_at)` depend on).
 
 ### 2.2 Constraints
 
@@ -322,7 +322,7 @@ A resolved bronze row must be a complete, well-typed table row. Centralizing sch
 
 ##### Responsibility scope
 
-Loads `specs/schemas/<db>.<table>.yaml`; pads a resolved record with missing schema properties as `null`; validates against the JSON schema (`additionalProperties:false`). Implements the post-step of `cpt-bronze-to-api-e2e-algo-yaml-resolve-refs`.
+Loads `metrics/schemas/<db>.<table>.yaml`; pads a resolved record with missing schema properties as `null`; validates against the JSON schema (`additionalProperties:false`). Implements the post-step of `cpt-bronze-to-api-e2e-algo-yaml-resolve-refs`.
 
 ##### Responsibility boundaries
 
@@ -345,7 +345,7 @@ Replaces `csv-asserter`. Dashboard metrics return many rows over a batch of quer
 
 Implements `cpt-bronze-to-api-e2e-algo-yaml-eval-expect`: selects a batch result by `id`; filters `result.items` to exactly one row by exact field equality; compares a subset of fields (`equal`, explicit `null`) or evaluates a CEL boolean (`assert`). Renders a precise failing-rule report.
 
-The CEL `assert` bindings are assembled in `e2e_lib/expect_engine.py::evaluate_case` (the `bindings` dict) and converted to CEL in `_eval_cel`:
+The CEL `assert` bindings are assembled in `lib/expect_engine.py::evaluate_case` (the `bindings` dict) and converted to CEL in `_eval_cel`:
 
 | Binding | Value | Present when |
 |---|---|---|

@@ -201,7 +201,7 @@ The system **MUST** discover every subfolder of `src/ingestion/tests/e2e/fixture
 
 **Touches**:
 
-- Files: `src/ingestion/tests/e2e/e2e_lib/fixture_loader.py`, `src/ingestion/tests/e2e/conftest.py`
+- Files: `src/ingestion/tests/e2e/lib/fixture_loader.py`, `src/ingestion/tests/e2e/conftest.py`
 - Components: `cpt-bronze-to-api-e2e-component-fixture-loader`
 
 ### Typed Bronze INSERT
@@ -217,7 +217,7 @@ The system **MUST** INSERT each CSV row into the bronze table with column types
 **Touches**:
 
 - API: ClickHouse HTTP 8123 / native 9000
-- Files: `src/ingestion/tests/e2e/e2e_lib/ch_seeder.py`
+- Files: `src/ingestion/tests/e2e/lib/ch_seeder.py`
 - Components: `cpt-bronze-to-api-e2e-component-ch-seeder`
 
 ### Scoped Dbt Build
@@ -234,7 +234,7 @@ The system **MUST** invoke `dbt build` with the selector from `spec.yaml` (e.g.
 
 **Touches**:
 
-- Files: `src/ingestion/tests/e2e/e2e_lib/dbt_runner.py`
+- Files: `src/ingestion/tests/e2e/lib/dbt_runner.py`
 - Components: `cpt-bronze-to-api-e2e-component-dbt-runner`
 
 ### API Roundtrip
@@ -252,7 +252,7 @@ The system **MUST** POST the request derived from `spec.yaml` to the analytics-a
 **Touches**:
 
 - API: `POST /v1/metrics/{id}/query`
-- Files: `src/ingestion/tests/e2e/e2e_lib/api_client.py`
+- Files: `src/ingestion/tests/e2e/lib/api_client.py`
 - Components: `cpt-bronze-to-api-e2e-component-api-client`
 
 ### Cell-Precise CSV Diff
@@ -267,7 +267,7 @@ On failure the system **MUST** render the first 20 mismatched cells as `(key, co
 
 **Touches**:
 
-- Files: `src/ingestion/tests/e2e/e2e_lib/csv_asserter.py`
+- Files: `src/ingestion/tests/e2e/lib/csv_asserter.py`
 - Components: `cpt-bronze-to-api-e2e-component-csv-asserter`
 
 ### Reference Test against insight.people