From d4ef87d1a9b18f2b8e77326c36adbfb68fc1e7e5 Mon Sep 17 00:00:00 2001 From: vyhuynh-product Date: Thu, 30 Jul 2026 11:36:57 +0700 Subject: [PATCH 01/17] chore(native.system): removed stop-and-wait for user approval after listing dimension proposal skill now proceed to breakdown immediately --- .../analyze-contribution/SKILL.md | 48 ++----------------- 1 file changed, 5 insertions(+), 43 deletions(-) diff --git a/native-skills/system/holistics-common/analyze-contribution/SKILL.md b/native-skills/system/holistics-common/analyze-contribution/SKILL.md index cdd2b7d..4688690 100644 --- a/native-skills/system/holistics-common/analyze-contribution/SKILL.md +++ b/native-skills/system/holistics-common/analyze-contribution/SKILL.md @@ -163,7 +163,7 @@ Run only Steps 1 and 2, then proceed directly to Phase 3 using `user_specified_d **If `user_specified_dimensions` is empty — standard path:** -Execute these five steps in the exact order below. Do not reorder, skip, or batch any step. +Execute these four steps in the exact order below. Do not reorder, skip, or batch any step. **Step 1 — Output the overview sentence as plain text:** @@ -175,11 +175,11 @@ Execute these five steps in the exact order below. Do not reorder, skip, or batc The table must include these columns: Comparison Value, Base Value, Delta Value, Pct Change. -**Step 3 — Output the transition sentence as plain text:** +**Step 3 — Post this update** > **There are multiple ways to explain what drove this change. In this analysis, I'll break it down by dimensions to identify which segments moved the most.** Here are my top [N] proposed dimensions: -**Step 4 — Output the dimension list as plain text.** This is a text output step, not a tool call. Write the numbered list directly in your response. Each item must include a short reason — **5–8 words max, business-focused, no technical jargon**: +**Step 4 — Output the dimension list as plain text, then proceed directly to Phase 3.** This is a text output step, not a tool call. Write the numbered list directly in your response. Each item must include a short, business-meaningful reason — **1 sentence max, focused on why this dimension is relevant to the metric change**: > **1. Dimension Name** — short business reason (e.g. "varies most by sales team", "differs by product line") > @@ -187,37 +187,7 @@ The table must include these columns: Comparison Value, Base Value, Delta Value, > > **3. Dimension Name** — short business reason -**Step 5 — Output the Stage 1 Summary Block as plain text, then STOP:** - -> --- -> Here's a summary of what we're going to analyze: -> -> - **Metric**: [metric name] -> - **Comparison period**: [comparison_period_literal] ([comparison_period_name]) -> - **Base period**: [base_period_literal] ([base_period_name]) -> - **Overall change**: [metric] [dropped/increased] [pct_change]%, from [comparison_value] to [base_value] (Δ [±overall_delta_value]) -> - **Dimension 1**: [dim1] -> - **Dimension 2**: [dim2] -> - **Dimension 3**: [dim3] -> -> Shall I proceed to break down **[metric]** by each of these dimensions to see where the [drop/increase] happened the most? -> --- - -**Do NOT call `ask_user` here.** Stop after outputting this block and wait for the user to reply in the chat. - ---- - -### Stage 2 Detection — check before Phase 3 - -Before running Phase 3, check the conversation history: - -- **If a Stage 1 Summary Block is present** (contains confirmed metric, periods, overall delta, and dimension list): - - Check the user's most recent reply: - - **Affirmative** (e.g. "yes", "proceed", "looks good") → load `metric`, `base_period_literal`, `comparison_period_literal`, `overall_delta_value`, and confirmed dimension list from the block. Do NOT re-run Phases 1 or 2. Proceed directly to Phase 3 with dimension #1. - - **Requests changes** (e.g. "remove X", "swap Y for Z", "add Y", "use X instead") → update the dimension list based on the feedback, output a new Stage 1 Summary Block with the updated list, then proceed directly to Phase 3. Do not stop for another confirmation. - - **Negative** (e.g. "no", "stop", "that's enough", "skip it") → stop. Output a single closing sentence: "Got it — stopping here. Let me know if you'd like to explore any dimension further." Do not proceed to Phase 3. - -- **If no Stage 1 Summary Block is present**: this is a fresh invocation — run from Phase 1. +After outputting the dimension list, proceed immediately to Phase 3. Do NOT stop or wait for user input. --- @@ -225,15 +195,7 @@ Before running Phase 3, check the conversation history: **Plan (once, before starting any dimension analysis):** -Post an update about the confirmed dimension list: -> # Analyzing these dimensions -> 1. [dim1] -> 2. [dim2] -> 3. [dim3] - -(Always show this update, even when the user accepted the proposed list without changes.) - -Then add new tasks to analyze the dimensions. +Add new tasks to analyze the dimensions. NOTE: If you are going to delegate, make sure to reference `/analyze-contribution` in the brief, so that sub-agent can follow the prompt structures correctly. From b3778a34778c3858fd20fbdbb523b5b8a3c1daa0 Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Tue, 11 Aug 2026 11:57:55 +0700 Subject: [PATCH 02/17] fix(native.system): echo confirmed data and delta changes in the opening lines --- .../holistics-common/analyze-changes/SKILL.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/native-skills/system/holistics-common/analyze-changes/SKILL.md b/native-skills/system/holistics-common/analyze-changes/SKILL.md index 27c9747..17b1a58 100644 --- a/native-skills/system/holistics-common/analyze-changes/SKILL.md +++ b/native-skills/system/holistics-common/analyze-changes/SKILL.md @@ -57,13 +57,23 @@ Brief: Parse the user's message for: - `metric` — the metric that changed (e.g., "revenue", "close rate", "daily signups") - `direction` — drop or increase -- `time_reference` — when the change occurred (e.g., "in 2026", "last quarter", "since January") +- `base_period` — the period where the change occurred (e.g., "May 2025") +- `comparison_period` — the reference period (e.g., "April 2025") If `metric` is not clearly named, ask before proceeding: > "Which metric are you looking at?" -Once the metric is confirmed, echo a one-line summary: -> "Got it — you're seeing a [drop/increase] in **[metric]** [time_reference]." +Fetch `current_value` (value in `base_period`) and `comparison_value` (value in `comparison_period`) +from the dataset. Compute `delta` = current_value − comparison_value. + +Post an update to confirm the actual values: +> "Got it — you're seeing a [drop/increase] in **[metric]** from **[comparison_value]** +> ([comparison_period]) to **[current_value]** ([base_period]), [formatted delta]." + +Delta formatting: +- Percentage metric (rate, %, ratio): "down/up **[|delta|] percentage points**" +- Currency: "down/up **$[|delta|]** ([relative%]%)" +- Count or other: "down/up **[|delta|]** ([relative%]%)" ``` Execution: Do this yourself, no delegation. From af009e6eaf77fb115ad8afac7663e1c42b82ca5b Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Tue, 11 Aug 2026 12:05:24 +0700 Subject: [PATCH 03/17] fix(native.system): fix typos --- .../system/holistics-common/analyze-changes/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/native-skills/system/holistics-common/analyze-changes/SKILL.md b/native-skills/system/holistics-common/analyze-changes/SKILL.md index 17b1a58..ae1acc6 100644 --- a/native-skills/system/holistics-common/analyze-changes/SKILL.md +++ b/native-skills/system/holistics-common/analyze-changes/SKILL.md @@ -84,7 +84,7 @@ Dependency: Step 1 Brief: ``` -/detect_anomoly +/detect_anomaly ``` Execution: @@ -96,7 +96,7 @@ Execution: ### Step 3 — Output the Stage 1 Summary Block -Depdendency: Step 2 +Dependency: Step 2 Brief: ``` From db4a875095ccac8bbf48b7a080763046410cb8c5 Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Tue, 11 Aug 2026 17:03:52 +0700 Subject: [PATCH 04/17] fix(native.system): strengthen wording to make the anomaly chart mandatory --- native-skills/system/holistics-common/detect-anomaly/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index 92afc32..2f89355 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -26,7 +26,7 @@ An anomaly task scans one metric's time series against a trend-following band bu Fill these with **one prioritized-fill pass**, not a chart-vs-typed branch (Workflow → *Step 1*). Confirm only fields you had to **guess**; silently accept fields **read** from the chart or stated by the user. ## What a bare minimum output looks like -The deliverable is a trend-anomaly chart plus a short prose summary — nothing else by default. +**The chart is required output. Do not skip it.** The deliverable is a trend-anomaly chart plus a short prose summary — the chart must always be attempted first. - **The chart** (Step 3): `actual` as a solid blue line, the `lower_bound`/`upper_bound` trend band dashed grey, and `is_anomaly` as red columns on a secondary 0..1 axis, with tooltips. - **The prose summary** (Step 3): for a business reader — defines an anomaly (a break from the recent trend), then lists each flagged bucket (date, value, above/below the trend, how far out in plain terms). - **The anomaly results** (the Step 2 AQL) stay internal — the source of truth for the summary, not shown as a table unless the user asks for the raw numbers. @@ -79,7 +79,7 @@ Don't hand-write the `CombinationChart` — pass the Step 2 explore (verbatim) t 2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust only decoration. Keep 6-digit hex (an 8-digit alpha hex can be rejected). 3. `execute_viz(dataset_uname: , viz: , title: …)` — pass only these three, no `aql` property. -If `execute_viz` errors, feed the error text back into `generate_viz`'s `query` (it self-corrects from prior errors) and retry once; if it still fails, fall back to the Step 2 `execute_aql` table + prose. +If `execute_viz` errors, feed the error text back into `generate_viz`'s `query` (it self-corrects from prior errors) and retry once; if it still fails, fall back to the Step 2 `execute_aql` table + prose — **this is the only permitted reason to omit the chart.** ### Summary (prose — Step 3) For a business reader: plain and professional, no raw notation (no bare "σ", "z = 3.4", "3σ"). Read the anomaly results; don't render them as a table (the user sees the chart) unless asked. From 4a403239bd16b643bce6bd63de3c999e67e4d1a3 Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Thu, 13 Aug 2026 18:06:06 +0700 Subject: [PATCH 05/17] fix(native.system): enforce exact W values and make is_anomaly always 0 or 1 --- .../system/holistics-common/detect-anomaly-aql/SKILL.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index 9faa8d0..24b19b3 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -16,6 +16,8 @@ The expected band **follows the trend**. For each bucket `t` (`T` truncated to ` **Warm-up gate (required).** A bucket is only banded/flagged once it has a **full `W`-bucket window** of prior history. Partial-frame window stats return values (not null), so gate explicitly on `n_prior = window_count(...) ≥ W`: buckets with `n_prior < W` get **no band** and are **never flagged** — the leading `W` lead-in buckets are exactly these, shown as context. +**Use these W values exactly.** Only shrink W if the total available history is shorter than W+1 — and if you do, state the reason explicitly in your response. Never silently use a smaller W. + Window `W` by grain — a whole number of cycles: | Grain | `W` | @@ -73,3 +75,4 @@ explore { - The `n_prior ≥ W` gate is mandatory (partial windows return values, not null, and would false-flag the first `W` buckets). - `case(when: …, else: null)` is valid and is how the band is hidden in the lead-in. - The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly — one explore, no two-stage query. +- `is_anomaly` must always return `0` or `1`, **never `null`** — the `else: 0` in the case expression is mandatory. It is plotted as a column on a 0..1 axis; nulls cause rendering gaps instead of clean zeros. From 8a6d0815a889fe8e77a6f60fcca012ee08830611 Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Thu, 13 Aug 2026 22:00:05 +0700 Subject: [PATCH 06/17] fix(native.system): define anomaly in the post update and null z-score without enough history --- .../system/holistics-common/detect-anomaly-aql/SKILL.md | 2 +- native-skills/system/holistics-common/detect-anomaly/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index 24b19b3..062a870 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -43,7 +43,7 @@ metric m_n = window_count(M, -W..-1, order: T | () | asc(), partitio metric m_drift = window_avg(m_delta, -W..-1, order: T | () | asc(), partition: []); metric m_spread = window_stdev(m_delta, -W..-1, order: T | () | asc(), partition: []); metric m_expected = m_prev + m_drift; -metric m_z = safe_divide(M - m_expected, m_spread); +metric m_z = case(when: m_n >= W, then: safe_divide(M - m_expected, m_spread), else: null); metric m_lower = case(when: m_n >= W, then: m_expected - k * m_spread, else: null); metric m_upper = case(when: m_n >= W, then: m_expected + k * m_spread, else: null); metric m_anom = case(when: and(m_n >= W, abs(m_z) > k), then: 1, else: 0); diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index 2f89355..a6fa14d 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -84,7 +84,7 @@ If `execute_viz` errors, feed the error text back into `generate_viz`'s `query` ### Summary (prose — Step 3) For a business reader: plain and professional, no raw notation (no bare "σ", "z = 3.4", "3σ"). Read the anomaly results; don't render them as a table (the user sees the chart) unless asked. -Open with one sentence defining an anomaly under this method, then the findings: +Post an update with one sentence defining an anomaly under this method, then the findings: > *An anomaly is any whose value departs sharply from where its recent trend was heading — a change much larger or smaller than the metric's normal -to- movement over the prior s.* From c09d63955bc5a79c9fd3c4907ac6d1849f6287e6 Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Sat, 15 Aug 2026 10:07:55 +0700 Subject: [PATCH 07/17] fix(native.system): check z-score non-null before classifying a bucket --- .../system/holistics-common/detect-anomaly/SKILL.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index a6fa14d..c0de830 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -88,10 +88,10 @@ Post an update with one sentence defining an anomaly under this method, then the > *An anomaly is any whose value departs sharply from where its recent trend was heading — a change much larger or smaller than the metric's normal -to- movement over the prior s.* -Classify each bucket in the reporting window: -- **Unusual** — `is_anomaly = 1`: above the trend (`actual > expected`) or below it. -- **Normal** — `is_anomaly = 0`, `z_score` non-null: moved in line with the trend. -- **Not enough history** — `z_score`/band null (a lead-in bucket, `n_prior < W`): shown as context without a band; not assessed. +Classify each bucket in the reporting window — **check in this exact order**: +1. **Not enough history** — `z_score` is null (a lead-in bucket, `n_prior < W`): shown as context without a band; not assessed. `is_anomaly` returns `0` for these too — always check `z_score` nullness first, or you will misclassify lead-in buckets as normal. +2. **Unusual** — `z_score` non-null AND `is_anomaly = 1`: actual broke above or below the expected trend band. +3. **Normal** — `z_score` non-null AND `is_anomaly = 0`: moved in line with the trend. Translate the statistics — describe the size of the surprise in plain terms (*"about times its normal monthly swing"*), z only as a parenthetical. From eee9f17dafc5f8098a04a29eae5bd13ed4806920 Mon Sep 17 00:00:00 2001 From: truongthinhnguyen03 Date: Mon, 17 Aug 2026 17:13:19 +0700 Subject: [PATCH 08/17] feat(native.system): reframe detect-anomaly as a narrated four-step analysis The skill now reads as an analyst working through the problem, with each step explaining what it is doing and why before it does it, and closing with a post_update so the user follows the reasoning rather than waiting for a verdict. Four steps replace three: establish the series (now drawing it, not only describing it), turn "unusual" into a rule and apply it, show the expectation as a band, then verify and write up. The verify pass is new; the checks existed but were scattered across three sections, so nothing forced a final look before presenting. Adds detect-anomaly-viz, which owns both charts, so the parent carries the reasoning and the sub-skills carry the mechanics. Content was reallocated on one rule: aql owns the statistics, viz owns the charts, the parent owns the analysis and the words. That removed fourteen facts that were stated in two or three places at once, including the edge-case table, whose rows split between the spec resolution and the two sub-skills. Other changes: - Steps 1 and 3 invoke detect-anomaly-viz with a slash block, matching Step 2. Naming a sub-skill in prose reads as a cross-reference, and only the explicitly invoked one was firing. - Both sub-skills document the parameters they expect; the aql invocation passes reporting and filters explicitly instead of in prose. - aql states that z_score must be null below the warm-up gate, which the caller relies on to tell unassessed buckets from normal ones. - The summary spec is one worked example plus conventions, replacing eight fill-in-the-blank templates. Co-Authored-By: Claude Opus 5 (1M context) --- .../detect-anomaly-aql/SKILL.md | 25 +- .../detect-anomaly-viz/SKILL.md | 53 ++++ .../holistics-common/detect-anomaly/SKILL.md | 252 +++++++++++------- 3 files changed, 225 insertions(+), 105 deletions(-) create mode 100644 native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index 062a870..12754a3 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -11,14 +11,18 @@ The expected band **follows the trend**. For each bucket `t` (`T` truncated to ` - `expected` = previous actual **+ drift**, where drift = the trailing mean of `Δ` over the `W` buckets before `t`. This is the band's centre; it tilts with the trend. - `spread` = the trailing **stddev of `Δ`** over those `W` buckets. - `lower/upper_bound` = `expected ∓ k · spread`. -- `z_score` = `(actual − expected) / spread` — is *this bucket's change* unusual versus the recent distribution of changes (stationary even when the level trends). +- `z_score` = `(actual − expected) / spread`. Asks whether *this bucket's change* is unusual against the recent distribution of changes, which stays stationary even when the level trends. - `is_anomaly` = `1` when `abs(z_score) > k`, else `0`. `k` default 3 (looser 2, stricter 4). -**Warm-up gate (required).** A bucket is only banded/flagged once it has a **full `W`-bucket window** of prior history. Partial-frame window stats return values (not null), so gate explicitly on `n_prior = window_count(...) ≥ W`: buckets with `n_prior < W` get **no band** and are **never flagged** — the leading `W` lead-in buckets are exactly these, shown as context. +A flat window (`spread = 0`) gives `z_score` null through `safe_divide`, and those buckets are never flagged. -**Use these W values exactly.** Only shrink W if the total available history is shorter than W+1 — and if you do, state the reason explicitly in your response. Never silently use a smaller W. +The band follows the trend, not the calendar: there is no seasonal term, so a recurring calendar peak flags as an anomaly. -Window `W` by grain — a whole number of cycles: +**Warm-up gate (required).** A bucket is only banded and flagged once it has a **full `W`-bucket window** of prior history. Partial-frame window stats return values, not null, so gate explicitly on `n_prior = window_count(...) ≥ W`: buckets with `n_prior < W` get **no band** and are **never flagged**. The leading `W` lead-in buckets are exactly these, carried as context. + +**Use these W values exactly.** Shrink `W` only when the total available history is shorter than `W + 1`, and say so explicitly in your response when you do. Never silently use a smaller `W`. + +Window `W` by grain, a whole number of cycles: | Grain | `W` | |---|---| @@ -31,9 +35,9 @@ If the series is shorter than `W + 1`, shrink `W` to about half the available hi ### Query -Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the window for the grain, `k` = the sensitivity threshold (default 3), `` = the dataset name, `` = a readable label. +Parameters from the caller: `M`, `T`, `grain`, `W`, `k`, `reporting` (the reporting timeframe in grains), `filters` (the dimensional filters, if any). -`M`, `T`, ``, `W`, `k` are placeholders; the `-1..-1` range (previous bucket) is literal; every `window_*` call takes `order: T | () | asc(), partition: []`. Widen the timeframe filter by `W` buckets, and add any dimensional filters. +The `-1..-1` range (previous bucket) is literal, and every `window_*` call takes `order: T | () | asc(), partition: []`. Widen the reporting timeframe by `W` buckets in the filter, and add the dimensional filters. Template: ```aql @@ -71,8 +75,9 @@ explore { ``` **Do not regress these:** -- Count prior buckets with `window_count(M, -W..-1, …)` — **never** `window_sum(1, …)` (SQL-generation error). -- The `n_prior ≥ W` gate is mandatory (partial windows return values, not null, and would false-flag the first `W` buckets). +- Count prior buckets with `window_count(M, -W..-1, …)`, **never** `window_sum(1, …)` (SQL-generation error). +- The `n_prior ≥ W` gate is mandatory. Partial windows return values, not null, and would false-flag the first `W` buckets. - `case(when: …, else: null)` is valid and is how the band is hidden in the lead-in. -- The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly — one explore, no two-stage query. -- `is_anomaly` must always return `0` or `1`, **never `null`** — the `else: 0` in the case expression is mandatory. It is plotted as a column on a 0..1 axis; nulls cause rendering gaps instead of clean zeros. +- The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly: one explore, no two-stage query. +- `is_anomaly` must always return `0` or `1`, **never `null`**. The `else: 0` is mandatory: it is plotted as a column on a 0..1 axis, and nulls cause rendering gaps instead of clean zeros. +- `z_score` must be null whenever `n_prior < W`. The caller distinguishes unassessed buckets from normal ones by that null, since `is_anomaly` is `0` for both. diff --git a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md new file mode 100644 index 0000000..e868eb1 --- /dev/null +++ b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md @@ -0,0 +1,53 @@ +--- +name: detect-anomaly-viz +description: |- + Guidelines for charting detect-anomaly results. ONLY use this skill when explicitly asked to do so. +user-invocable: false +--- + +Two charts, drawn over the **same x range** so the second reads as the first with the answer added: the series alone (Step 1), then the series with its expected band and the flagged buckets (Step 3). + +Never hand-write the viz body. `generate_viz` writes it, `execute_viz` validates it on execution. State the decoration explicitly in the `query`, because `generate_viz` defaults its palette and `pattern: 'inherited'` unless told otherwise. + +Invoked once per chart. `chart:` selects which; the caller supplies the rest. + +### Series chart (Step 1) + +`chart: series`. Parameters: `dataset`, `M`, `T`, `grain`, `W`, `reporting` (the reporting timeframe in grains), `metric label`, `filters` (the dimensional filters, if any). + +The metric as it stands, nothing added. Span is ` + W` so it matches the Step 3 chart. + +```aql +explore { + dimensions { bucket: T | () } + measures { actual: M } + filters { + T matches @(last s) + // + the dimensional filters carried from the chart + } + sorts { bucket asc nulls last } +} +``` + +`generate_viz(dataset_uname: , aql: , query: "Line chart of by : actual as a solid #255DD4 line; x-axis is the bucket; format as short-suffix currency like $500K.")`, then `execute_viz(dataset_uname:, viz:, title:)`. + +Keep it plain: no band, no annotations, nothing flagged. **A failure here does not block the run.** Retry once, then continue without it and say the preview could not be drawn. + +### Anomaly chart (Step 3) + +`chart: anomaly`. Parameters: `dataset`, `aql` (the Step 2 explore, verbatim), `grain`, `metric label`. + +Same series, plus the band and the flags. The Step 2 explore already returns every field this chart needs, so it is passed through untouched and no filters or timeframe are supplied again. + +1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : actual as a solid #255DD4 line; lower_bound and upper_bound as grey #9CA3AF dashed lines forming the expected band; is_anomaly as red #FCB8B8 columns on a secondary 0..1 right axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")` +2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust decoration only. +3. `execute_viz(dataset_uname: , viz: , title: …)` + +On an error, feed the error text back into `generate_viz`'s `query` (it self-corrects from prior errors) and retry once. If it still fails, fall back to the Step 2 table plus prose. **That is the only permitted reason to ship without this chart.** + +### Do not regress +- Pass only `dataset_uname`, `viz`, `title` to `execute_viz`. No `aql` property. +- 6-digit hex only. An 8-digit alpha hex can be rejected. +- `is_anomaly` belongs on a **secondary 0..1 right axis** as columns. On the main axis it is invisible against the metric's scale. +- Both charts cover ` + W`. Cropping the Step 1 chart to the reporting window alone makes the two charts disagree about where the series starts. +- For a metric that cannot go negative, clamp the **displayed** lower bound at 0. The band sits around the trend rather than the level, so it can dip below zero. diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index c0de830..bd30b1b 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -1,136 +1,198 @@ --- name: detect-anomaly description: |- - Detect statistical anomalies across a metric's history and visualize them — unusual spikes, drops, and deviations flagged against an expected range built from the metric's own recent history. + Detect statistical anomalies across a metric's history and visualize them: unusual spikes, drops, and deviations flagged against an expected range built from the metric's own recent history. - Use when the user wants to find or check for unusual values in a metric — words like anomaly, unusual, spike, drop, dip, outlier, "out of the ordinary", "anything weird". A specific date is optional; the skill always scans the whole series. + Use when the user wants to find or check for unusual values in a metric. Trigger words include anomaly, unusual, spike, drop, dip, outlier, "out of the ordinary", "anything weird". A specific date is optional; the skill always scans the whole series. - Typical phrasings: detect anomalies in revenue, any outliers in daily signups, has anything looked unusual in MRR lately, were there spikes or drops in active users this quarter, is revenue on May 20 unusual (a focus date — still scans the full series), or /detect_anomaly with no args (the skill asks for the metric). + Typical phrasings: detect anomalies in revenue, any outliers in daily signups, has anything looked unusual in MRR lately, were there spikes or drops in active users this quarter, is revenue on May 20 unusual (a focus date; still scans the full series), or /detect_anomaly with no args (the skill asks for the metric). Do NOT trigger for: dimensional attribution (which segment or dimension drove an anomaly), explaining the business cause, or ongoing/scheduled monitoring & alerting. This skill flags which points are statistically unusual and shows them; it does not explain why. user-invocable: false --- -Flag which points in a metric's time series moved unusually against its own recent trend, and show them: build a trend-following expected band from recent history, flag the buckets that break out of it, and visualize the result. This skill flags *which* points are unusual, never *why* — deflect causal or dimensional-attribution asks (Conventions → *Edge cases*). +Build an expected band from a metric's recent history, flag the buckets that break out of it, and chart the result. This skill identifies which points are unusual, not why they moved (Conventions → *Out of scope*). + +Two sub-skills carry the mechanics: **`detect-anomaly-aql`** owns the method and the query, **`detect-anomaly-viz`** owns both charts. ## What a good input looks like -An anomaly task scans one metric's time series against a trend-following band built from its own recent history. The primary path is **chart-anchored**: the user is viewing a chart and asks whether its movement is normal, so the fields below are read from that chart's Viz AML. Only `metric` is truly required — derive or default the rest, then echo everything back (with the resolved timeframe as an absolute range) so the user can correct a mis-parse. -- **`metric` (`M`)** — *required*. From the viewed chart's y-axis measure; else the user's words. If missing, ask — don't guess. If the chart carries **more than one** y-axis measure, ask which one to analyze (don't default to the first). -- **`dataset`** — *required*. From the chart's `dataset:`; else a ranked measure search. If missing, ask. -- **`time axis` (`T`) + `granularity`** — inferred from the chart's x-axis field and its `datetrunc` transformation. If the x-axis isn't a time field, ask for the metric + time grain. -- **`filters`** — optional. The chart's dimensional filters (not its time range). -- **`focus_date`** — optional. From the user's words; changes only the closing prose. -- **`timeframe`** — optional. The chart's timeframe; else default by grain (`day`→90d, `week`→26w, `month`→24m, `quarter`→12q). +Anomaly detection compares a metric against its own recent history to determine whether a value is far from what the preceding periods would suggest. The analysis needs one measure, the date field and grain it is measured on, and enough prior history to establish what normal movement looks like. + +A task is fully specified when you can answer: + +1. **Which measure, and which dataset.** The method compares a series against itself, so it runs on exactly one measure; two measures are two analyses. When a chart offers several, ask which one, because choosing for the user wastes the whole run. +2. **Which date field, and at what grain.** Day, week, month, or quarter. The grain sets what can be found: a spike lasting two days does not show up in monthly buckets, and a metric recorded monthly cannot be read daily. +3. **Which slice of the data.** The segment in view, if any. The result describes the series you selected, not the business as a whole: one region collapsing can leave the company total flat, and a flat total can hide two segments moving in opposite directions. One slice per analysis; breaking the metric down by a dimension is a different question. +4. **Which period the findings cover.** Flagged buckets are reported only inside this period. Earlier buckets are shown as context, because the method needs prior history before it can judge a bucket. +5. **Whether the user has a specific date in question.** A named date does not narrow the analysis; the whole series is always scanned. It changes only what the answer leads with. -Fill these with **one prioritized-fill pass**, not a chart-vs-typed branch (Workflow → *Step 1*). Confirm only fields you had to **guess**; silently accept fields **read** from the chart or stated by the user. +The user rarely supplies all five, and often has not asked for anomaly detection by name. + +- **Read before asking.** A chart in view already carries the measure, date field, grain, slice and window; take them from there (*Resolving the spec*). +- **Ask only where a wrong guess wastes the run**: the measure, and with no chart its dataset. Derive the rest and state it back in one line with real dates, so a misread is cheap to correct. ## What a bare minimum output looks like -**The chart is required output. Do not skip it.** The deliverable is a trend-anomaly chart plus a short prose summary — the chart must always be attempted first. -- **The chart** (Step 3): `actual` as a solid blue line, the `lower_bound`/`upper_bound` trend band dashed grey, and `is_anomaly` as red columns on a secondary 0..1 axis, with tooltips. -- **The prose summary** (Step 3): for a business reader — defines an anomaly (a break from the recent trend), then lists each flagged bucket (date, value, above/below the trend, how far out in plain terms). -- **The anomaly results** (the Step 2 AQL) stay internal — the source of truth for the summary, not shown as a table unless the user asks for the raw numbers. -- If `execute_viz` ultimately fails, fall back to the Step 2 table + prose (Conventions → *On an error*). +Two charts and a prose summary. +- **The series chart** (Step 1) and **the anomaly chart** (Step 3), both per `detect-anomaly-viz`. The anomaly chart is required output. The series chart is not: a failure there is noted and the run continues. +- **The prose summary** (Step 4), for a business reader: what an anomaly means here, then each flagged bucket with its date, value, direction, and how far outside the band it fell. +- **The anomaly results** (Step 2) stay internal. They are the summary's source of truth, not a table to show, unless the user asks for the raw numbers. ## Workflow -Three steps, in order. Hand-write the AQL yourself (Step 2) — the method must be exact — but author the chart with `generate_viz` (Step 3): hand-written viz grammar is the top source of invalid output. `execute_aql` runs the AQL (and validates it on execution); `execute_viz` runs the chart. - -Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the window for the grain, `k` = the sensitivity threshold (default 3), `` = the dataset name, `` = a readable label. - -### Step 1 — Resolve the input -Produce one detection spec `{ dataset, M, T, granularity, filters, timeframe }`. Fill each field from the strongest available source, in precedence order — there is **no chart-path-vs-typed-path branch**: -1. **The active chart's Viz AML** (the runtime hands the AI the AML of the chart the user is viewing) — fills `dataset`, `M`, `T`, `granularity`, `filters`, and the chart's timeframe. -2. **The user's words** — fill or *override* any field they spoke to ("…but over the last 3 years", "…just the West region", or, with no chart, the metric name itself). -3. **Dataset metadata + defaults** — `granularity` from `T`; timeframe default per grain. -4. **Ask the user** — only for a field still empty (realistically `M`/`dataset` with no chart and no parseable metric). - -**Reading the chart's Viz AML** (source 1) — pull each field out of the attached AML: -- `dataset` ← the viz `dataset:`. -- `M` ← the y-axis measure. If the chart has **more than one** y-axis measure, don't pick for the user — **ask which one** to analyze (list them and wait), unless the user already named one in their words. Anomaly detection runs on a single series, and silently choosing the wrong one wastes the whole run. With exactly one y-axis measure, use it. If `M` is a viz-level `calculation` rather than a dataset measure, carry its `@aql` formula and inline it as `metric M = ;` in the Step 2 query. -- `T` + `granularity` ← the x-axis field and its `transformation` (`datetrunc month` → grain = month). If the x-axis is **not** a time field, this isn't a time series → fall back to typed/ask. -- `filters` ← the viz `filter` / `filter_groups` / `conditions`, **excluding** the time-range filter. -- **Legend/breakdown** → collapse to the total (drop the legend) unless the user explicitly asks per-segment (per-segment is out of scope — Conventions → *Edge cases*). -- **Timeframe + warm-up** — the reporting timeframe is what gets flagged/reported; the Step 2 query widens it by `W` buckets on the leading edge so the reporting window is fully banded. The chart shows those `W` lead-in buckets as trend context (unbanded); prose reports only within the reporting timeframe. - -Confirm only fields you **guessed** (a ranked search: *"I'll use `sales_orders` (measure `total_revenue`). Confirm, or name another."* — then wait); silently accept fields **read** from the chart or stated by the user. Validate that `M` exists as a measure (or is a carried viz calculation); if a typed metric matches none, show the top-3 closest measures and ask. Echo the resolved slice back, including the lead-in context: *"Analyzing GMV for Region = West over Jul 2023 – Aug 2024 (your view); the chart also shows earlier months as lead-in to establish the trend."* - -### Step 2 — Detect (write AQL → execute_aql) -1. Write the anomaly detection AQL with `generate_aql`, providing `M`, `T`, ``, `W`, `k`, folding in the dimensional filters, and widening the timeframe by `W` buckets for warm-up. Prompt structure: - ``` - /detect-anomaly-aql - - M: ... - T: ... - grain: ... - W: ... - k: ... - ``` -2. Call `execute_aql` (title per Conventions → *Titles*) — it validates the AQL on execution and errors loudly on a bad query. The result is the **anomaly results** — the summary's source of truth and the exact AQL you hand to `generate_viz` in Step 3. - -### Step 3 — Present (generate_viz → execute_viz → summarize) -Feed the Step 2 AQL to `generate_viz` per Output → *Anomaly chart*, adjust only decoration, then `execute_viz` (title per Conventions → *Titles*). Then write the prose per Output → *Summary*, reading only the buckets inside the reporting timeframe. On a failure, Conventions → *On an error*. -## Output +Four steps. Each is a move an analyst makes, and each closes with a `post_update` so the user follows the reasoning as it happens instead of receiving a verdict at the end. **Narrating is not pausing**: post and keep working. The only stop in the run is the measure question in Step 1. + +Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the window for the grain, `k` = the sensitivity threshold, `` = the dataset name, `` = a readable label. + +### Step 1: Establish the series + +Nothing can be called unusual until there is a definite series to judge: one measure, on one time axis, at one grain, for one slice, over one period. Almost all of that is already settled by whatever the user is looking at, so read it rather than asking for it back. + +Then draw it. The chart confirms the resolved grain and slice, and it is the picture the expected band gets added to in Step 3. + +Resolve the spec (*Resolving the spec*), then draw the series: -### Anomaly chart (generate_viz — Step 3) -Don't hand-write the `CombinationChart` — pass the Step 2 explore (verbatim) to `generate_viz` and **state the decoration explicitly in the `query`** (generate_viz defaults its palette and `pattern: 'inherited'` unless told otherwise, so name the colours): +``` +/detect-anomaly-viz -1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : actual as a solid #255DD4 line; lower_bound and upper_bound as grey #9CA3AF dashed lines forming the expected band; is_anomaly as red #FCB8B8 columns on a secondary 0..1 right axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")`. -2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust only decoration. Keep 6-digit hex (an 8-digit alpha hex can be rejected). -3. `execute_viz(dataset_uname: , viz: , title: …)` — pass only these three, no `aql` property. +chart: series +dataset: ... +M: ... +T: ... +grain: ... +W: ... +reporting: ... +metric label: ... +filters: ... +``` -If `execute_viz` errors, feed the error text back into `generate_viz`'s `query` (it self-corrects from prior errors) and retry once; if it still fails, fall back to the Step 2 `execute_aql` table + prose — **this is the only permitted reason to omit the chart.** +Read the returned rows for the three conditions that change what you can promise: +- **Missing buckets** between the first and last. Gaps break the period-to-period comparison the method rests on. +- **A final bucket still in progress.** A part-period value reads as a collapse. Exclude it, or say it is incomplete. +- **Row count.** `detect-anomaly-aql` shrinks `W` or stops the run when history is short; either outcome must be said out loud. -### Summary (prose — Step 3) -For a business reader: plain and professional, no raw notation (no bare "σ", "z = 3.4", "3σ"). Read the anomaly results; don't render them as a table (the user sees the chart) unless asked. +Then post: the metric, grain, slice and period as real dates; that the metric will be judged against its own recent history and the whole series scanned even if the user named a single date; and anything the three checks turned up. -Post an update with one sentence defining an anomaly under this method, then the findings: +### Step 2: Turn "unusual" into a rule, and apply it -> *An anomaly is any whose value departs sharply from where its recent trend was heading — a change much larger or smaller than the metric's normal -to- movement over the prior s.* +Eyeballing the chart is not detection. A rule has to say what each period was expected to be, and how much deviation is normal for this particular metric. -Classify each bucket in the reporting window — **check in this exact order**: -1. **Not enough history** — `z_score` is null (a lead-in bucket, `n_prior < W`): shown as context without a band; not assessed. `is_anomaly` returns `0` for these too — always check `z_score` nullness first, or you will misclassify lead-in buckets as normal. -2. **Unusual** — `z_score` non-null AND `is_anomaly = 1`: actual broke above or below the expected trend band. -3. **Normal** — `z_score` non-null AND `is_anomaly = 0`: moved in line with the trend. +The rule: predict each period from where the last `W` periods of movement were heading, then judge the gap between actual and prediction against how much this metric usually moves period to period. `detect-anomaly-aql` owns it, including the window sizes and the warm-up gate. -Translate the statistics — describe the size of the surprise in plain terms (*"about times its normal monthly swing"*), z only as a parenthetical. +Hand over the resolved parameters: -No focus date (primary path): +``` +/detect-anomaly-aql -> **Found unusual (s) in ** over ****: -> - **** — : **** the expected trend (about × its normal movement). -> - … one bullet per flagged bucket, chronological +M: ... +T: ... +grain: ... +W: ... +k: ... +reporting: ... +filters: ... +``` -If `N = 0`: *"No unusual values in over — every moved in line with its recent trend."* +Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those are the **anomaly results**, the summary's source of truth; the AQL itself goes to Step 3 unchanged. Then post the rule in the reader's terms, naming the `W` and `k` actually used. -Focus date given: lead with the verdict on that bucket, then the same list. -- Unusual: *"** on = ** is unusual — it broke **** its recent trend (about × the normal movement)."* -- Normal: *"** on = ** moved in line with its recent trend — not unusual."* -- Lead-in / no band: *"Not enough prior history to assess — it's within the first s, shown as context without a band."* +### Step 3: Show the expectation, not just the verdict -Closing pointer (N ≥ 1): why an anomaly happened or which dimension drove it is out of scope — invite the user to investigate via a dimensional breakdown and business context (marketing calendar, CRM, recent news). +The band is what makes "three periods were unusual" checkable rather than asserted: it shows where each period was expected to land, how wide normal was at that moment, and which points fell outside. Draw it onto the same picture from Step 1. + +``` +/detect-anomaly-viz + +chart: anomaly +dataset: ... +aql: +grain: ... +metric label: ... +``` + +Then post how to read it: the band is the expected range given the recent trend, it widens when movement has been erratic, and the red columns mark what broke out. + +### Step 4: Read the result honestly + +A period that was never assessed carries `is_anomaly = 0` exactly like one that was assessed and found normal, and the warm-up buckets sit in the result set alongside real findings. Both read as "normal" unless checked for. + +Classify every bucket in the reporting window, **in this order**: +1. **Not assessed.** `z_score` is null: a lead-in bucket without a full window of prior history. `is_anomaly` is `0` for these too, so testing it first misreports them as normal. +2. **Unusual.** `z_score` non-null and `is_anomaly = 1`. +3. **Normal.** `z_score` non-null and `is_anomaly = 0`. + +Then check, and write the summary (Output → *Summary*): +- **Only buckets inside the reporting timeframe are reported.** The warm-up buckets are context, never findings. +- **The `W` and `k` stated are the ones used**, including a shrunk `W`. +- **The anomaly chart rendered**, or the fallback was taken and said so. +- **No causal or dimensional claim** anywhere in the prose. +- **Numbers formatted** per Conventions → *Numeric formatting*. + +Two patterns in the flagged set change what the summary can claim. Check both: +- **Flags sharing a calendar position** (the same month each year, the same weekday). The method has no seasonal term, so a recurring peak flags as an anomaly. Say so, and offer the seasonality-aware re-run. +- **Two flags within `W` buckets of each other.** The first widens the band for the periods after it, so the second was judged against a looser threshold and anything following it may have been missed. + +## Output -Post-run override (offer last): *"Want me to re-run with a stricter or looser threshold? Looser flags more s (milder deviations); stricter flags only the most extreme. A seasonality-aware version (for weekly/annual patterns) is also available."* The threshold is `k` (looser = 2, stricter = 4; default 3) — the only sensitivity control, never asked upfront. On accept, re-run Steps 2–3 with the new `k`. +### Summary (prose, Step 4) + +A complete answer on the primary path (no focus date, two buckets flagged). The chart is not written, it is what `execute_viz` returned in Step 3; the prose below it is Step 4's: + +> ⟦ Step 3 anomaly chart renders here ⟧ +> +> **What counts as unusual here.** An anomaly is any month whose value departs sharply from where its recent trend was heading: a change much larger or smaller than GMV's normal month-to-month movement over the prior 12 months. +> +> **Found 2 unusual months in GMV, Jul 2023 to Aug 2024:** +> +> - **Nov 2023: $4.2M.** Above the expected trend by about 3.6× its normal monthly movement. +> - **Mar 2024: $1.9M.** Below the expected trend by about 4.1× its normal monthly movement. +> +> **What this does not tell you.** Why they moved, and which segment drove them, is outside this analysis. Break GMV down by product, channel or customer segment, and check the marketing calendar and CRM for those two windows. +> +> Want me to re-run with a stricter or looser threshold? Looser flags more months (milder deviations); stricter flags only the most extreme. A seasonality-aware version is also available if GMV has a repeating annual shape. + +Then the conventions: + +- **Register.** Business reader, plain and professional. No raw notation: no bare "σ", "z = 3.4", "3σ". `z` appears only as a spoken multiple, as above. +- **Markup carries the scan.** A bold label opens each part; each finding leads with its date and value in bold, then a plain sentence for the assessment. No headings, no tables. +- **The definition comes first**, with the grain, metric and `W` filled in. +- **One line per flagged bucket, chronological**, each carrying the date, the value, the direction, and how far outside the band it fell in plain terms. +- **The user has the chart.** Do not repeat the results as a table unless asked. +- **Nothing flagged**: say so and name the period. *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with its recent trend."* +- **A date was named**: lead with the verdict on that bucket, unusual, normal, or not assessed for want of history, then the same list. The full series is still scanned. +- **The closing pointer** appears only when something was flagged. +- **The re-run offer goes last.** `k` is the only sensitivity control and is never asked upfront (values in `detect-anomaly-aql`). On accept, re-run Steps 2 to 4 with the new `k`. + +## Resolving the spec + +Produce one spec `{ dataset, M, T, granularity, filters, timeframe }`. **One fill pass, not a chart path and a typed path**: take each field from the chart when there is one, let the user's words override anything they spoke to ("…but over the last 3 years", "…just the West region"), then fall back to defaults, and ask only for what is still empty. + +| Field | From the chart's Viz AML | Otherwise | +|---|---|---| +| `dataset` | the viz `dataset:` | a ranked measure search; ask if nothing matches | +| `M` | the y-axis measure | the user's words; if a typed metric matches no measure, show the top-3 closest and ask | +| `T` + `granularity` | the x-axis field and its `transformation` (`datetrunc month` → month) | `granularity` derived from `T` | +| `filters` | `filter` / `filter_groups` / `conditions`, **excluding** the time-range filter | none | +| `timeframe` | the chart's timeframe | per grain: `day`→90d, `week`→26w, `month`→24m, `quarter`→12q | + +- **More than one y-axis measure**: ask which to analyze (list them and wait), unless the user already named one. +- **`M` is a viz-level `calculation`**, not a dataset measure: carry its `@aql` formula and inline it as `metric M = ;` in the Step 2 query. +- **The x-axis is not a time field**: this is not a time series. Ask for the metric and grain instead; do not proceed. +- **A legend/breakdown**: drop it and analyze the total, unless the user explicitly asks per-segment (Conventions → *Out of scope*). +- **No chart, and the metric cannot be parsed**, or `list_datasets()` is unsupported (dev mode): ask the user to name the dataset and metric explicitly. Do not proceed. +- **Confirm only what you guessed** (*"I'll use `sales_orders` (measure `total_revenue`). Confirm, or name another."*), then wait. Fields read from the chart or stated by the user are accepted silently. +- The timeframe resolved here is the **reporting** window. `detect-anomaly-aql` widens it for warm-up; those lead-in buckets are context on both charts and are never reported as findings. ## Conventions -- **On an error, retry once, then fall back.** If a retry still fails: Step 2 → surface the error (don't pretend detection completed); Step 3 → fall back to the Step 2 table + prose. Never loop more than one retry per step. -- **Titles.** `execute_aql` / `execute_viz` require a `title` (under ~60 chars; avoid "query 1"/"untitled"): the Step 2 anomaly results `execute_aql` → ` anomaly results ()`; the Step 3 anomaly chart `execute_viz` → `Anomaly detection: ()`. +- **On an error, retry once, then fall back.** Step 2: surface the error, never imply detection completed. Step 3: fall back to the Step 2 table plus prose. Never loop more than one retry per step. +- **Titles.** `execute_aql` / `execute_viz` require a `title` (under ~60 chars; avoid "query 1"/"untitled"): Step 1 series chart → ` by ()`; Step 2 anomaly results → ` anomaly results ()`; Step 3 anomaly chart → `Anomaly detection: ()`. ### Numeric formatting - Currency-like measures (label/name contains `revenue`, `cost`, `price`, `arr`, `mrr`, `gmv`, `amount`): `$` prefix, short suffix (`$500K`, `$1.2M`); use the dataset's stated currency symbol if metadata provides it. - Counts and rates: thousands separators (`1,234`); rates to two significant decimals (`3.42%`). - z-scores / multiples: one decimal place (`3.4`). -### Edge cases - -| Situation | Behavior | -|---|---| -| Viz AML x-axis is not a time field | Not a time series — fall back to typed/ask for the metric + time grain. Don't proceed. | -| Viz has more than one y-axis measure | Ask the user which single measure to analyze (list them); don't default to the first. Skip the ask only if the user already named one. | -| Viz has a legend/breakdown | Collapse to the total (drop the legend) unless the user asks per-segment. | -| Series too short even with the lead-in (starts inside the window) | Earliest buckets show without a band (`n_prior < W`); if < 8 usable points, stop with the insufficient-history message. | -| Flat series (spread = 0) | `safe_divide` → `z_score` null; not flagged (never divide by zero). | -| Non-negative metric, band dips below 0 | Rare (the band is around the trend, not the level), but clamp the *displayed* lower bound at 0 for non-negative metrics. | -| `list_datasets()` unsupported (dev mode) or no chart + unparseable metric | Ask the user to name the dataset/metric explicitly. Do not proceed. | -| User asks "why did this happen?" or "what dimension drove it?" | Reply: "I can flag which points broke from the trend, but identifying which dimension drove an anomaly — or its business cause — is outside this skill. Break the metric down by relevant dimensions in a dashboard, or check your marketing calendar / CRM / news for that window." | -| User asks to "show the raw numbers" | Show the `execute_aql` anomaly results directly — `bucket`, `actual`, `expected`, `lower_bound`, `upper_bound`, `z_score`, `is_anomaly`. Never group/pivot rows by `is_anomaly`. | +### Out of scope +This skill says which points broke from the trend. It does not say why, and it does not attribute a break to a segment. +- **"Why did this happen?" / "What dimension drove it?"** Reply: "I can flag which points broke from the trend, but identifying which dimension drove an anomaly, or its business cause, is outside this skill. Break the metric down by relevant dimensions in a dashboard, or check your marketing calendar / CRM / news for that window." +- **A per-segment run.** Out of scope: one slice per analysis. Offer a separate run on the segment the user names. +- **"Show the raw numbers."** Show the `execute_aql` anomaly results directly: `bucket`, `actual`, `expected`, `lower_bound`, `upper_bound`, `z_score`, `is_anomaly`. Never group or pivot rows by `is_anomaly`. From 0e67a9df5e7e2ab2ed0d5ab3d8abec84af585485 Mon Sep 17 00:00:00 2001 From: truongthinhnguyen03 Date: Mon, 17 Aug 2026 17:31:03 +0700 Subject: [PATCH 09/17] fix(native.system): put is_anomaly on the first y-axis so it renders behind The columns were on a secondary 0..1 axis, which drew the full-height marks in front of the metric line and the band. Swapping the axes keeps the same 0..1 scale and full-height marks, behind the data. Co-Authored-By: Claude Opus 5 (1M context) --- .../system/holistics-common/detect-anomaly-aql/SKILL.md | 2 +- .../system/holistics-common/detect-anomaly-viz/SKILL.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index 12754a3..abd6ccb 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -79,5 +79,5 @@ explore { - The `n_prior ≥ W` gate is mandatory. Partial windows return values, not null, and would false-flag the first `W` buckets. - `case(when: …, else: null)` is valid and is how the band is hidden in the lead-in. - The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly: one explore, no two-stage query. -- `is_anomaly` must always return `0` or `1`, **never `null`**. The `else: 0` is mandatory: it is plotted as a column on a 0..1 axis, and nulls cause rendering gaps instead of clean zeros. +- `is_anomaly` must always return `0` or `1`, **never `null`**. The `else: 0` is mandatory: it is plotted as a column, and nulls cause rendering gaps instead of clean zeros. - `z_score` must be null whenever `n_prior < W`. The caller distinguishes unassessed buckets from normal ones by that null, since `is_anomaly` is `0` for both. diff --git a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md index e868eb1..f11fa2c 100644 --- a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md @@ -39,7 +39,7 @@ Keep it plain: no band, no annotations, nothing flagged. **A failure here does n Same series, plus the band and the flags. The Step 2 explore already returns every field this chart needs, so it is passed through untouched and no filters or timeframe are supplied again. -1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : actual as a solid #255DD4 line; lower_bound and upper_bound as grey #9CA3AF dashed lines forming the expected band; is_anomaly as red #FCB8B8 columns on a secondary 0..1 right axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")` +1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : is_anomaly as red #FCB8B8 columns on the FIRST y-axis, 0..1 scale; actual as a solid #255DD4 line and lower_bound and upper_bound as grey #9CA3AF dashed lines, all three on the SECOND y-axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")` 2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust decoration only. 3. `execute_viz(dataset_uname: , viz: , title: …)` @@ -48,6 +48,6 @@ On an error, feed the error text back into `generate_viz`'s `query` (it self-cor ### Do not regress - Pass only `dataset_uname`, `viz`, `title` to `execute_viz`. No `aql` property. - 6-digit hex only. An 8-digit alpha hex can be rejected. -- `is_anomaly` belongs on a **secondary 0..1 right axis** as columns. On the main axis it is invisible against the metric's scale. +- `is_anomaly` goes on the **first** y-axis (0..1) and the metric, `lower_bound` and `upper_bound` on the **second**. That order is what keeps the columns behind the lines. - Both charts cover ` + W`. Cropping the Step 1 chart to the reporting window alone makes the two charts disagree about where the series starts. - For a metric that cannot go negative, clamp the **displayed** lower bound at 0. The band sits around the trend rather than the level, so it can dip below zero. From bb29fb5d934a8ab5dc9d601fd5edbcee020d2f3f Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Tue, 18 Aug 2026 20:51:01 +0700 Subject: [PATCH 10/17] fix(native.system): post anomaly definition before the chart in detect-anomaly Co-Authored-By: Claude Sonnet 4.6 --- .../system/holistics-common/detect-anomaly/SKILL.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index bd30b1b..b03d8f1 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -99,6 +99,11 @@ Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those The band is what makes "three periods were unusual" checkable rather than asserted: it shows where each period was expected to land, how wide normal was at that moment, and which points fell outside. Draw it onto the same picture from Step 1. +First, post the anomaly definition: +> **What counts as unusual here.** An anomaly is any [grain] whose value departs sharply from where its recent trend was heading: a change much larger or smaller than [metric]'s normal [grain]-to-[grain] movement over the prior [W] [grains]. + +Then draw the chart: + ``` /detect-anomaly-viz @@ -137,9 +142,9 @@ Two patterns in the flagged set change what the summary can claim. Check both: A complete answer on the primary path (no focus date, two buckets flagged). The chart is not written, it is what `execute_viz` returned in Step 3; the prose below it is Step 4's: -> ⟦ Step 3 anomaly chart renders here ⟧ +> ⟦ Step 3 anomaly definition post_update renders here ⟧ > -> **What counts as unusual here.** An anomaly is any month whose value departs sharply from where its recent trend was heading: a change much larger or smaller than GMV's normal month-to-month movement over the prior 12 months. +> ⟦ Step 3 anomaly chart renders here ⟧ > > **Found 2 unusual months in GMV, Jul 2023 to Aug 2024:** > @@ -154,7 +159,6 @@ Then the conventions: - **Register.** Business reader, plain and professional. No raw notation: no bare "σ", "z = 3.4", "3σ". `z` appears only as a spoken multiple, as above. - **Markup carries the scan.** A bold label opens each part; each finding leads with its date and value in bold, then a plain sentence for the assessment. No headings, no tables. -- **The definition comes first**, with the grain, metric and `W` filled in. - **One line per flagged bucket, chronological**, each carrying the date, the value, the direction, and how far outside the band it fell in plain terms. - **The user has the chart.** Do not repeat the results as a table unless asked. - **Nothing flagged**: say so and name the period. *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with its recent trend."* From 0c9a45238f33e96795258acc52b5f15233ebfdfc Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Wed, 19 Aug 2026 11:57:54 +0700 Subject: [PATCH 11/17] update calculation of contribution to change in summary table --- .../system/holistics-common/analyze-contribution/SKILL.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/native-skills/system/holistics-common/analyze-contribution/SKILL.md b/native-skills/system/holistics-common/analyze-contribution/SKILL.md index 4688690..1dc7686 100644 --- a/native-skills/system/holistics-common/analyze-contribution/SKILL.md +++ b/native-skills/system/holistics-common/analyze-contribution/SKILL.md @@ -310,7 +310,12 @@ Ranked table of all confirmed dimensions, sorted by `dimension_driver_pct` desce |-----------|-------------|------------------------| | … | | | -For the **Contribution to change** column, apply the same threshold rule as the per-segment display: +For the **Contribution to change** column: + +> **Critical:** `dimension_driver_pct = sum(delta_value for all driver_of_change segments) / overall_delta_value × 100` +> The denominator is always `overall_delta_value` from Phase 2 (the metric's net change with no segment grouping). Never use the top segment's delta, the dimension's total, or any other value as the denominator. Using the wrong denominator will produce 100% or other nonsensical results. + +Apply the threshold rule: - If `|dimension_driver_pct| ≤ 100%`: show `[dimension_driver_pct]%` - If `|dimension_driver_pct| > 100%`: show `[N]× the net change` where [N] = round(|dimension_driver_pct| / 100, 1) From 9ebb5ae8956f76317f55fff0d403896f0a80d14e Mon Sep 17 00:00:00 2001 From: truongthinhnguyen03 Date: Wed, 19 Aug 2026 14:37:47 +0700 Subject: [PATCH 12/17] fix(native.system): remove the arithmetic that breaks detect-anomaly runs A live run produced invalid AQL and a wrong window. Both trace to the sub-skill asking the model to compute things, in one case before it had the data to compute them from. Date literals. The template only ever showed the relative form, so a report over historical dates had no example to copy and the model invented `@date(2016-01-01)`, which is a syntax error: `@` introduces a literal, not a call. The query section is now a complete worked query with real values rather than a placeholder skeleton, showing the fixed range form, and the do-not-regress list names the mistake. W. It was a per-grain table plus a shrink rule keyed on total available history, which the model has to guess at because it has not queried yet. That is how a monthly run came back with W=11 and no widened filter. W is now 12 at every grain: twelve is where the spread estimate stops improving much, and a longer baseline costs assessed buckets without buying precision. It is no longer passed as a parameter, since both sub-skills know it, and `12` in the worked query is now correct as written rather than something to substitute. Short series are read, not predicted. The full-window check already returns nulls when history is thin, so the shrink-or-stop branch is gone; if nothing comes back assessed, the answer says so. Also: method section restated as what it does before how, with the notation kept; `drift` renamed `avg_change`; `warm-up` and `lead-in` collapsed to `lead-in`; example identifiers made generic; and three sentences cut that explained behaviour the query already enforces. Co-Authored-By: Claude Opus 5 (1M context) --- .../detect-anomaly-aql/SKILL.md | 79 ++++++++++--------- .../detect-anomaly-viz/SKILL.md | 2 +- .../holistics-common/detect-anomaly/SKILL.md | 18 ++--- 3 files changed, 49 insertions(+), 50 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index abd6ccb..4a0a74e 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -6,57 +6,46 @@ user-invocable: false --- ### Method -The expected band **follows the trend**. For each bucket `t` (`T` truncated to ``): -- `actual` = `M`; `Δ` = the change from the previous bucket. -- `expected` = previous actual **+ drift**, where drift = the trailing mean of `Δ` over the `W` buckets before `t`. This is the band's centre; it tilts with the trend. -- `spread` = the trailing **stddev of `Δ`** over those `W` buckets. -- `lower/upper_bound` = `expected ∓ k · spread`. -- `z_score` = `(actual − expected) / spread`. Asks whether *this bucket's change* is unusual against the recent distribution of changes, which stays stationary even when the level trends. -- `is_anomaly` = `1` when `abs(z_score) > k`, else `0`. `k` default 3 (looser 2, stricter 4). - -A flat window (`spread = 0`) gives `z_score` null through `safe_divide`, and those buckets are never flagged. +Detect anomalies by working out what change to expect in each period from the periods before it, then flagging the periods whose actual lands far from that expectation. -The band follows the trend, not the calendar: there is no seasonal term, so a recurring calendar peak flags as an anomaly. +Everything is computed on **changes**, not levels: `Δ` = the change from the previous bucket. That is what lets the band follow a trend, since a steadily climbing series has large values but ordinary changes, and so never trips on the climb itself. -**Warm-up gate (required).** A bucket is only banded and flagged once it has a **full `W`-bucket window** of prior history. Partial-frame window stats return values, not null, so gate explicitly on `n_prior = window_count(...) ≥ W`: buckets with `n_prior < W` get **no band** and are **never flagged**. The leading `W` lead-in buckets are exactly these, carried as context. +For each bucket `t` (`T` truncated to ``), over the `W` buckets before it: -**Use these W values exactly.** Shrink `W` only when the total available history is shorter than `W + 1`, and say so explicitly in your response when you do. Never silently use a smaller `W`. +- `avg_change` = mean of `Δ`. The change to expect. +- `spread` = stddev of `Δ`. How much that change normally varies. +- `expected` = previous `actual` + `avg_change`. Where this bucket should land. +- `lower/upper_bound` = `expected` ∓ `k · spread`. +- `z_score` = (`actual` − `expected`) / `spread`. How far off it landed, counted in `spread`s. +- `is_anomaly` = `1` when `abs(z_score) > k`, else `0`. `k` default 3 (looser 2, stricter 4). -Window `W` by grain, a whole number of cycles: +**`W` is 12, at every grain.** Twelve is where the spread estimate stops improving much, and a longer baseline costs assessed buckets without buying precision. Do not vary it by grain, by series length, or by anything else. -| Grain | `W` | -|---|---| -| day | 28 (4 weeks) | -| week | 13 (a quarter) | -| month | 12 (a year) | -| quarter | 8 (two years) | +**A full window is required.** Check `n_prior = window_count(...) >= W`, so a bucket is banded and flagged only once it has `W` buckets behind it, and gets `z_score` null otherwise. Without that check the opening `W` buckets false-flag, because a window function over a partial frame returns a value rather than null. Those opening buckets are the **lead-in**: carried for context, never judged. -If the series is shorter than `W + 1`, shrink `W` to about half the available history (minimum 4) and say so. With fewer than 8 points total, stop: *"Not enough history (n=X) to detect anomalies."* +The gate is also what makes a short series safe, with no branching and no arithmetic: too little history returns nulls rather than an error. So read the result instead of predicting it. If no bucket in the reporting window comes back with a non-null `z_score`, there was not enough history to assess anything, and the answer says that rather than that nothing was unusual. ### Query -Parameters from the caller: `M`, `T`, `grain`, `W`, `k`, `reporting` (the reporting timeframe in grains), `filters` (the dimensional filters, if any). +A complete query: monthly `total_revenue`, reporting on 2016 and 2017, `W` 12, `k` 3. -The `-1..-1` range (previous bucket) is literal, and every `window_*` call takes `order: T | () | asc(), partition: []`. Widen the reporting timeframe by `W` buckets in the filter, and add the dimensional filters. - -Template: ```aql -metric m_prev = window_avg(M, -1..-1, order: T | () | asc(), partition: []); -metric m_delta = M - m_prev; -metric m_n = window_count(M, -W..-1, order: T | () | asc(), partition: []); -metric m_drift = window_avg(m_delta, -W..-1, order: T | () | asc(), partition: []); -metric m_spread = window_stdev(m_delta, -W..-1, order: T | () | asc(), partition: []); -metric m_expected = m_prev + m_drift; -metric m_z = case(when: m_n >= W, then: safe_divide(M - m_expected, m_spread), else: null); -metric m_lower = case(when: m_n >= W, then: m_expected - k * m_spread, else: null); -metric m_upper = case(when: m_n >= W, then: m_expected + k * m_spread, else: null); -metric m_anom = case(when: and(m_n >= W, abs(m_z) > k), then: 1, else: 0); +metric m_prev = window_avg(total_revenue, -1..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_delta = total_revenue - m_prev; +metric m_n = window_count(total_revenue, -12..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_avg_change = window_avg(m_delta, -12..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_spread = window_stdev(m_delta, -12..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_expected = m_prev + m_avg_change; +metric m_z = case(when: m_n >= 12, then: safe_divide(total_revenue - m_expected, m_spread), else: null); +metric m_lower = case(when: m_n >= 12, then: m_expected - 3 * m_spread, else: null); +metric m_upper = case(when: m_n >= 12, then: m_expected + 3 * m_spread, else: null); +metric m_anom = case(when: and(m_n >= 12, abs(m_z) > 3), then: 1, else: 0); explore { dimensions { - bucket: T | () + bucket: orders.created_at | month() } measures { - actual: M, + actual: total_revenue, expected: m_expected, lower_bound: m_lower, upper_bound: m_upper, @@ -64,9 +53,7 @@ explore { is_anomaly: m_anom } filters { - // reporting window WIDENED by W buckets for warm-up; the @( … ) parentheses are REQUIRED. - T matches @(last s) - // + any dimensional filters carried from the chart, e.g. bq_fct_order_items.merchant_country_code == "US" + orders.created_at matches @2015-01-01 - 2017-12-31 } sorts { bucket asc nulls last @@ -74,6 +61,19 @@ explore { } ``` +Substitute `total_revenue` with `M`, `orders.created_at` with `T`, `month()` with the grain, and `3` with `k`. Leave `12` and `-1..-1` exactly as written: `W` is always 12, and `-1..-1` is the previous bucket. + +**The filter opens `W` buckets before the reporting window.** Above, the report covers 2016 and 2017 and `W` is 12, so the filter opens at 2015-01-01 and those twelve months arrive as the lead-in. A relative window widens the same way: a 24-month report becomes `matches @(last 36 months)`. + +**Dimensional filters go in the same block**, one per line: + +```aql + filters { + orders.created_at matches @2015-01-01 - 2017-12-31 + orders.region == "West" + } +``` + **Do not regress these:** - Count prior buckets with `window_count(M, -W..-1, …)`, **never** `window_sum(1, …)` (SQL-generation error). - The `n_prior ≥ W` gate is mandatory. Partial windows return values, not null, and would false-flag the first `W` buckets. @@ -81,3 +81,4 @@ explore { - The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly: one explore, no two-stage query. - `is_anomaly` must always return `0` or `1`, **never `null`**. The `else: 0` is mandatory: it is plotted as a column, and nulls cause rendering gaps instead of clean zeros. - `z_score` must be null whenever `n_prior < W`. The caller distinguishes unassessed buckets from normal ones by that null, since `is_anomaly` is `0` for both. +- Dates are **literals, not calls**: `@2016-01-01`, `@2016-01-01 - 2017-12-31`, `@(last 12 months)`. There is no `@date(...)` function, and writing one is a syntax error at the `(`. diff --git a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md index f11fa2c..045e05a 100644 --- a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md @@ -13,7 +13,7 @@ Invoked once per chart. `chart:` selects which; the caller supplies the rest. ### Series chart (Step 1) -`chart: series`. Parameters: `dataset`, `M`, `T`, `grain`, `W`, `reporting` (the reporting timeframe in grains), `metric label`, `filters` (the dimensional filters, if any). +`chart: series`. Parameters: `dataset`, `M`, `T`, `grain`, `reporting` (the reporting timeframe in grains), `metric label`, `filters` (the dimensional filters, if any). `W` is 12, per `detect-anomaly-aql`. The metric as it stands, nothing added. Span is ` + W` so it matches the Step 3 chart. diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index b03d8f1..d492751 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -42,7 +42,7 @@ Two charts and a prose summary. Four steps. Each is a move an analyst makes, and each closes with a `post_update` so the user follows the reasoning as it happens instead of receiving a verdict at the end. **Narrating is not pausing**: post and keep working. The only stop in the run is the measure question in Step 1. -Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the window for the grain, `k` = the sensitivity threshold, `` = the dataset name, `` = a readable label. +Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the baseline window, always 12, `k` = the sensitivity threshold, `` = the dataset name, `` = a readable label. ### Step 1: Establish the series @@ -60,7 +60,6 @@ dataset: ... M: ... T: ... grain: ... -W: ... reporting: ... metric label: ... filters: ... @@ -69,7 +68,7 @@ filters: ... Read the returned rows for the three conditions that change what you can promise: - **Missing buckets** between the first and last. Gaps break the period-to-period comparison the method rests on. - **A final bucket still in progress.** A part-period value reads as a collapse. Exclude it, or say it is incomplete. -- **Row count.** `detect-anomaly-aql` shrinks `W` or stops the run when history is short; either outcome must be said out loud. +- **Row count.** A short series is not an error: it comes back unassessed rather than flagged. Say so instead of reporting that nothing was unusual. Then post: the metric, grain, slice and period as real dates; that the metric will be judged against its own recent history and the whole series scanned even if the user named a single date; and anything the three checks turned up. @@ -77,7 +76,7 @@ Then post: the metric, grain, slice and period as real dates; that the metric wi Eyeballing the chart is not detection. A rule has to say what each period was expected to be, and how much deviation is normal for this particular metric. -The rule: predict each period from where the last `W` periods of movement were heading, then judge the gap between actual and prediction against how much this metric usually moves period to period. `detect-anomaly-aql` owns it, including the window sizes and the warm-up gate. +The rule: predict each period from where the last `W` periods of movement were heading, then judge the gap between actual and prediction against how much this metric usually moves period to period. `detect-anomaly-aql` owns it, including the baseline length and the full-window rule. Hand over the resolved parameters: @@ -87,13 +86,12 @@ Hand over the resolved parameters: M: ... T: ... grain: ... -W: ... k: ... reporting: ... filters: ... ``` -Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those are the **anomaly results**, the summary's source of truth; the AQL itself goes to Step 3 unchanged. Then post the rule in the reader's terms, naming the `W` and `k` actually used. +Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those are the **anomaly results**, the summary's source of truth; the AQL itself goes to Step 3 unchanged. Then post the rule in the reader's terms, naming the baseline length and `k`. ### Step 3: Show the expectation, not just the verdict @@ -118,7 +116,7 @@ Then post how to read it: the band is the expected range given the recent trend, ### Step 4: Read the result honestly -A period that was never assessed carries `is_anomaly = 0` exactly like one that was assessed and found normal, and the warm-up buckets sit in the result set alongside real findings. Both read as "normal" unless checked for. +A period that was never assessed carries `is_anomaly = 0` exactly like one that was assessed and found normal, and the lead-in buckets sit in the result set alongside real findings. Both read as "normal" unless checked for. Classify every bucket in the reporting window, **in this order**: 1. **Not assessed.** `z_score` is null: a lead-in bucket without a full window of prior history. `is_anomaly` is `0` for these too, so testing it first misreports them as normal. @@ -126,8 +124,8 @@ Classify every bucket in the reporting window, **in this order**: 3. **Normal.** `z_score` non-null and `is_anomaly = 0`. Then check, and write the summary (Output → *Summary*): -- **Only buckets inside the reporting timeframe are reported.** The warm-up buckets are context, never findings. -- **The `W` and `k` stated are the ones used**, including a shrunk `W`. +- **Only buckets inside the reporting timeframe are reported.** The lead-in buckets are context, never findings. +- **`W` and `k` are the values from `detect-anomaly-aql`**, never adjusted for this series. - **The anomaly chart rendered**, or the fallback was taken and said so. - **No causal or dimensional claim** anywhere in the prose. - **Numbers formatted** per Conventions → *Numeric formatting*. @@ -184,7 +182,7 @@ Produce one spec `{ dataset, M, T, granularity, filters, timeframe }`. **One fil - **A legend/breakdown**: drop it and analyze the total, unless the user explicitly asks per-segment (Conventions → *Out of scope*). - **No chart, and the metric cannot be parsed**, or `list_datasets()` is unsupported (dev mode): ask the user to name the dataset and metric explicitly. Do not proceed. - **Confirm only what you guessed** (*"I'll use `sales_orders` (measure `total_revenue`). Confirm, or name another."*), then wait. Fields read from the chart or stated by the user are accepted silently. -- The timeframe resolved here is the **reporting** window. `detect-anomaly-aql` widens it for warm-up; those lead-in buckets are context on both charts and are never reported as findings. +- The timeframe resolved here is the **reporting** window. `detect-anomaly-aql` opens the filter earlier to supply the lead-in; those buckets are context on both charts and are never reported as findings. ## Conventions - **On an error, retry once, then fall back.** Step 2: surface the error, never imply detection completed. Step 3: fall back to the Step 2 table plus prose. Never loop more than one retry per step. From 47ec9b7acb12128f2934922be24cc064e0db6b69 Mon Sep 17 00:00:00 2001 From: Vy Huynh Date: Wed, 19 Aug 2026 15:08:32 +0700 Subject: [PATCH 13/17] update summary table in contribution skill --- .../analyze-contribution/SKILL.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/native-skills/system/holistics-common/analyze-contribution/SKILL.md b/native-skills/system/holistics-common/analyze-contribution/SKILL.md index 1dc7686..a91157f 100644 --- a/native-skills/system/holistics-common/analyze-contribution/SKILL.md +++ b/native-skills/system/holistics-common/analyze-contribution/SKILL.md @@ -236,7 +236,7 @@ Execute this once per dimension, in confirmed order. Complete the full phase — 6. Compute `contribution_pct = delta_value / overall_delta_value × 100` for each segment using the scalar `overall_delta_value` from Phase 2. -7. Compute and store `dimension_driver_pct = sum(delta_value for all driver_of_change segments) / overall_delta_value × 100` — used to rank dimensions in Phase 4. +7. Compute and store `top_segment_contribution_pct = top_driver_segment_delta_value / overall_delta_value × 100` — the contribution of the single top driver segment; used in the Phase 4 "Contribution to change" column and to rank dimensions. **Show:** @@ -304,20 +304,21 @@ Opening heading and summary line: > > [Metric] [dropped/increased] by [pct_change]% (from [comparison_value] to [base_value], [±abs(delta)]) between [comparison period] and [base period]. -Ranked table of all confirmed dimensions, sorted by `dimension_driver_pct` descending. Successfully analyzed dimensions appear first; failed dimensions appear at the bottom: +Ranked table of all confirmed dimensions, sorted by `top_segment_contribution_pct` descending. Successfully analyzed dimensions appear first; failed dimensions appear at the bottom: | Dimension | Top Segment | Contribution to change | |-----------|-------------|------------------------| | … | | | -For the **Contribution to change** column: +For the **Contribution to change** column, show `top_segment_contribution_pct` — the contribution of the single top driver segment: -> **Critical:** `dimension_driver_pct = sum(delta_value for all driver_of_change segments) / overall_delta_value × 100` -> The denominator is always `overall_delta_value` from Phase 2 (the metric's net change with no segment grouping). Never use the top segment's delta, the dimension's total, or any other value as the denominator. Using the wrong denominator will produce 100% or other nonsensical results. +> `top_segment_contribution_pct = top_driver_segment_delta_value / overall_delta_value × 100` +> +> **Critical:** The denominator is always `overall_delta_value` from Phase 2 (the metric's net change with no segment grouping). Never use the top segment's own delta as the denominator — that always produces 100%. Apply the threshold rule: -- If `|dimension_driver_pct| ≤ 100%`: show `[dimension_driver_pct]%` -- If `|dimension_driver_pct| > 100%`: show `[N]× the net change` where [N] = round(|dimension_driver_pct| / 100, 1) +- If `|top_segment_contribution_pct| ≤ 100%`: show `[top_segment_contribution_pct]%` +- If `|top_segment_contribution_pct| > 100%`: show `[N]× the net change` where [N] = round(|top_segment_contribution_pct| / 100, 1) **Conditionally include (one line each):** From 24ea014f060296084974c6238ea62dd9f1235b12 Mon Sep 17 00:00:00 2001 From: truongthinhnguyen03 Date: Wed, 19 Aug 2026 15:40:10 +0700 Subject: [PATCH 14/17] fix(native.system): draw the anomaly chart even when nothing is flagged A run with no anomalies was skipping the chart, and the skill was the reason. Step 3 motivated the chart entirely by the flagged points, so at zero flags its stated purpose disappeared and skipping read as the correct call. The summary's nothing-flagged convention compounded it by describing a one-sentence answer with no chart in it. Step 3 now justifies the chart by the verdict rather than by the flags: without the band, "nothing was unusual" is an assertion the reader cannot check. The nothing-flagged convention says to draw it anyway, and the Step 4 check names the case, since "the chart rendered" was satisfiable by having decided not to render one. Co-Authored-By: Claude Opus 5 (1M context) --- .../system/holistics-common/detect-anomaly/SKILL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index d492751..8f0afde 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -95,7 +95,7 @@ Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those ### Step 3: Show the expectation, not just the verdict -The band is what makes "three periods were unusual" checkable rather than asserted: it shows where each period was expected to land, how wide normal was at that moment, and which points fell outside. Draw it onto the same picture from Step 1. +The chart is the evidence for the verdict, whichever way the verdict goes: it shows where each period was expected to land, how wide normal was at that moment, and whether the series stayed inside. **A run that flags nothing needs it as much as one that flags three.** Without the band, "nothing was unusual" is an assertion the reader has no way to check. Draw it onto the same picture from Step 1. First, post the anomaly definition: > **What counts as unusual here.** An anomaly is any [grain] whose value departs sharply from where its recent trend was heading: a change much larger or smaller than [metric]'s normal [grain]-to-[grain] movement over the prior [W] [grains]. @@ -126,7 +126,7 @@ Classify every bucket in the reporting window, **in this order**: Then check, and write the summary (Output → *Summary*): - **Only buckets inside the reporting timeframe are reported.** The lead-in buckets are context, never findings. - **`W` and `k` are the values from `detect-anomaly-aql`**, never adjusted for this series. -- **The anomaly chart rendered**, or the fallback was taken and said so. +- **The anomaly chart was drawn**, including when nothing was flagged, or the fallback was taken and said so. - **No causal or dimensional claim** anywhere in the prose. - **Numbers formatted** per Conventions → *Numeric formatting*. @@ -159,7 +159,7 @@ Then the conventions: - **Markup carries the scan.** A bold label opens each part; each finding leads with its date and value in bold, then a plain sentence for the assessment. No headings, no tables. - **One line per flagged bucket, chronological**, each carrying the date, the value, the direction, and how far outside the band it fell in plain terms. - **The user has the chart.** Do not repeat the results as a table unless asked. -- **Nothing flagged**: say so and name the period. *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with its recent trend."* +- **Nothing flagged**: draw the chart anyway, then say so and name the period. *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with its recent trend."* - **A date was named**: lead with the verdict on that bucket, unusual, normal, or not assessed for want of history, then the same list. The full series is still scanned. - **The closing pointer** appears only when something was flagged. - **The re-run offer goes last.** `k` is the only sensitivity control and is never asked upfront (values in `detect-anomaly-aql`). On accept, re-run Steps 2 to 4 with the new `k`. From 0187b61f912dac6e1eeee844608e2def2cf48112 Mon Sep 17 00:00:00 2001 From: truongthinhnguyen03 Date: Wed, 19 Aug 2026 18:07:54 +0700 Subject: [PATCH 15/17] fix(native.system): distinguish unassessed buckets from normal ones in the summary Zero flags had one wording, and it claimed the series was fine. A series too short to judge produces zero flags too, so a run that assessed nothing would report that every period moved in line with its recent trend. The summary now separates three outcomes: judged and all normal, judged in part, and not judged at all. The partly-assessed case matters as soon as the data does not reach back far enough to fill the lead-in, since the findings and the clean-bill claim then cover less than the period asked about. The rule for this existed, but in detect-anomaly-aql, which is loaded at Step 2 to write the query. The summary is written at Step 4 from the parent's conventions, so it was never in context when it applied. Moved to the parent, with a Step 4 check, and the sub-skill now just says the caller reports what came back assessed. Co-Authored-By: Claude Opus 5 (1M context) --- .../system/holistics-common/detect-anomaly-aql/SKILL.md | 2 +- native-skills/system/holistics-common/detect-anomaly/SKILL.md | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index 4a0a74e..bdd6138 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -23,7 +23,7 @@ For each bucket `t` (`T` truncated to ``), over the `W` buckets before it **A full window is required.** Check `n_prior = window_count(...) >= W`, so a bucket is banded and flagged only once it has `W` buckets behind it, and gets `z_score` null otherwise. Without that check the opening `W` buckets false-flag, because a window function over a partial frame returns a value rather than null. Those opening buckets are the **lead-in**: carried for context, never judged. -The gate is also what makes a short series safe, with no branching and no arithmetic: too little history returns nulls rather than an error. So read the result instead of predicting it. If no bucket in the reporting window comes back with a non-null `z_score`, there was not enough history to assess anything, and the answer says that rather than that nothing was unusual. +The gate is also what makes a short series safe, with no branching and no arithmetic: too little history returns nulls rather than an error. Read the result instead of predicting it; the caller reports what came back assessed. ### Query diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index 8f0afde..de26773 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -125,6 +125,7 @@ Classify every bucket in the reporting window, **in this order**: Then check, and write the summary (Output → *Summary*): - **Only buckets inside the reporting timeframe are reported.** The lead-in buckets are context, never findings. +- **The claim covers what was assessed, not what was asked for.** If any bucket in the period came back with a null `z_score`, the summary names the span actually judged (Output → *Summary*). - **`W` and `k` are the values from `detect-anomaly-aql`**, never adjusted for this series. - **The anomaly chart was drawn**, including when nothing was flagged, or the fallback was taken and said so. - **No causal or dimensional claim** anywhere in the prose. @@ -160,6 +161,8 @@ Then the conventions: - **One line per flagged bucket, chronological**, each carrying the date, the value, the direction, and how far outside the band it fell in plain terms. - **The user has the chart.** Do not repeat the results as a table unless asked. - **Nothing flagged**: draw the chart anyway, then say so and name the period. *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with its recent trend."* +- **Nothing assessed**: when no bucket came back with a non-null `z_score`, the series was too short for the method to judge anything. Say that, never that nothing was unusual: *"Not enough history to assess GMV. The method needs 12 months of movement behind a month before it can judge it, and this series starts in Jan 2024."* +- **Partly assessed**: when only some of the period was judged, name the span that was. *"Assessed GMV from Jul 2024 onward; the months before it had too little history behind them."* Findings and the "nothing unusual" claim both cover that span only, never the full period. - **A date was named**: lead with the verdict on that bucket, unusual, normal, or not assessed for want of history, then the same list. The full series is still scanned. - **The closing pointer** appears only when something was flagged. - **The re-run offer goes last.** `k` is the only sensitivity control and is never asked upfront (values in `detect-anomaly-aql`). On accept, re-run Steps 2 to 4 with the new `k`. From 74af68625605328d64b8cd98cea4281daae720ce Mon Sep 17 00:00:00 2001 From: truongthinhnguyen03 Date: Fri, 21 Aug 2026 11:55:41 +0700 Subject: [PATCH 16/17] refactor(native.system): drop the Step 1 series chart from detect-anomaly The series chart was a non-blocking preview before the detection query ran; removing it lets the run go straight from resolving the spec to running detection. The three data-quality checks (missing buckets, in-progress final bucket, row count) move to Step 2 where rows first come back from execute_aql. - detect-anomaly: Step 1 resolves spec + posts only; no chart drawn. Step 2 runs detection and reads the checks. Step 3 is the sole chart (series + band). Trim verbose prose across steps and headers. - detect-anomaly-viz: single chart family; drop the series-chart section and the chart: selector. Co-Authored-By: Claude --- .../detect-anomaly-viz/SKILL.md | 32 ++----- .../holistics-common/detect-anomaly/SKILL.md | 89 ++++++++----------- 2 files changed, 40 insertions(+), 81 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md index 045e05a..98966a8 100644 --- a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md @@ -5,39 +5,17 @@ description: |- user-invocable: false --- -Two charts, drawn over the **same x range** so the second reads as the first with the answer added: the series alone (Step 1), then the series with its expected band and the flagged buckets (Step 3). +One chart, drawn in Step 3: the metric series with its expected band and the flagged buckets. It spans the reporting window plus `W` lead-in buckets, so the band has history to form against. Never hand-write the viz body. `generate_viz` writes it, `execute_viz` validates it on execution. State the decoration explicitly in the `query`, because `generate_viz` defaults its palette and `pattern: 'inherited'` unless told otherwise. -Invoked once per chart. `chart:` selects which; the caller supplies the rest. - -### Series chart (Step 1) - -`chart: series`. Parameters: `dataset`, `M`, `T`, `grain`, `reporting` (the reporting timeframe in grains), `metric label`, `filters` (the dimensional filters, if any). `W` is 12, per `detect-anomaly-aql`. - -The metric as it stands, nothing added. Span is ` + W` so it matches the Step 3 chart. - -```aql -explore { - dimensions { bucket: T | () } - measures { actual: M } - filters { - T matches @(last s) - // + the dimensional filters carried from the chart - } - sorts { bucket asc nulls last } -} -``` - -`generate_viz(dataset_uname: , aql: , query: "Line chart of by : actual as a solid #255DD4 line; x-axis is the bucket; format as short-suffix currency like $500K.")`, then `execute_viz(dataset_uname:, viz:, title:)`. - -Keep it plain: no band, no annotations, nothing flagged. **A failure here does not block the run.** Retry once, then continue without it and say the preview could not be drawn. +Invoked once. The caller supplies the parameters. ### Anomaly chart (Step 3) -`chart: anomaly`. Parameters: `dataset`, `aql` (the Step 2 explore, verbatim), `grain`, `metric label`. +Parameters: `dataset`, `aql` (the Step 2 explore, verbatim), `grain`, `metric label`. -Same series, plus the band and the flags. The Step 2 explore already returns every field this chart needs, so it is passed through untouched and no filters or timeframe are supplied again. +The series, plus the band and the flags. The Step 2 explore already returns every field this chart needs, so it is passed through untouched and no filters or timeframe are supplied again. 1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : is_anomaly as red #FCB8B8 columns on the FIRST y-axis, 0..1 scale; actual as a solid #255DD4 line and lower_bound and upper_bound as grey #9CA3AF dashed lines, all three on the SECOND y-axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")` 2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust decoration only. @@ -49,5 +27,5 @@ On an error, feed the error text back into `generate_viz`'s `query` (it self-cor - Pass only `dataset_uname`, `viz`, `title` to `execute_viz`. No `aql` property. - 6-digit hex only. An 8-digit alpha hex can be rejected. - `is_anomaly` goes on the **first** y-axis (0..1) and the metric, `lower_bound` and `upper_bound` on the **second**. That order is what keeps the columns behind the lines. -- Both charts cover ` + W`. Cropping the Step 1 chart to the reporting window alone makes the two charts disagree about where the series starts. +- The chart covers ` + W`. Cropping to the reporting window alone hides the lead-in the band needs to form, and shows the series starting later than the results do. - For a metric that cannot go negative, clamp the **displayed** lower bound at 0. The band sits around the trend rather than the level, so it can dip below zero. diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index de26773..86a79fd 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -13,7 +13,7 @@ user-invocable: false Build an expected band from a metric's recent history, flag the buckets that break out of it, and chart the result. This skill identifies which points are unusual, not why they moved (Conventions → *Out of scope*). -Two sub-skills carry the mechanics: **`detect-anomaly-aql`** owns the method and the query, **`detect-anomaly-viz`** owns both charts. +Two sub-skills carry the mechanics: **`detect-anomaly-aql`** owns the method and the query, **`detect-anomaly-viz`** owns the chart. ## What a good input looks like @@ -33,8 +33,8 @@ The user rarely supplies all five, and often has not asked for anomaly detection - **Ask only where a wrong guess wastes the run**: the measure, and with no chart its dataset. Derive the rest and state it back in one line with real dates, so a misread is cheap to correct. ## What a bare minimum output looks like -Two charts and a prose summary. -- **The series chart** (Step 1) and **the anomaly chart** (Step 3), both per `detect-anomaly-viz`. The anomaly chart is required output. The series chart is not: a failure there is noted and the run continues. +One chart and a prose summary. +- **The anomaly chart** (Step 3), per `detect-anomaly-viz`. It is required output; the run ships with it or says why it could not be drawn. - **The prose summary** (Step 4), for a business reader: what an anomaly means here, then each flagged bucket with its date, value, direction, and how far outside the band it fell. - **The anomaly results** (Step 2) stay internal. They are the summary's source of truth, not a table to show, unless the user asks for the raw numbers. @@ -46,35 +46,13 @@ Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimens ### Step 1: Establish the series -Nothing can be called unusual until there is a definite series to judge: one measure, on one time axis, at one grain, for one slice, over one period. Almost all of that is already settled by whatever the user is looking at, so read it rather than asking for it back. +A series needs one measure on one time axis, at one grain, for one slice, over one period. Read it from the chart rather than asking for it back. -Then draw it. The chart confirms the resolved grain and slice, and it is the picture the expected band gets added to in Step 3. +Resolve the spec (*Resolving the spec*): ask only for what's still empty — at most the measure, and with no chart its dataset. Confirm only what you guessed, then wait; fields read from the chart or stated by the user are accepted silently. -Resolve the spec (*Resolving the spec*), then draw the series: +Post the metric, grain, slice and period as real dates, and that the whole series is scanned even if the user named a single date. The data checks — missing buckets, a final bucket still in progress, row count — are read off the Step 2 results. -``` -/detect-anomaly-viz - -chart: series -dataset: ... -M: ... -T: ... -grain: ... -reporting: ... -metric label: ... -filters: ... -``` - -Read the returned rows for the three conditions that change what you can promise: -- **Missing buckets** between the first and last. Gaps break the period-to-period comparison the method rests on. -- **A final bucket still in progress.** A part-period value reads as a collapse. Exclude it, or say it is incomplete. -- **Row count.** A short series is not an error: it comes back unassessed rather than flagged. Say so instead of reporting that nothing was unusual. - -Then post: the metric, grain, slice and period as real dates; that the metric will be judged against its own recent history and the whole series scanned even if the user named a single date; and anything the three checks turned up. - -### Step 2: Turn "unusual" into a rule, and apply it - -Eyeballing the chart is not detection. A rule has to say what each period was expected to be, and how much deviation is normal for this particular metric. +### Step 2: Run the detection The rule: predict each period from where the last `W` periods of movement were heading, then judge the gap between actual and prediction against how much this metric usually moves period to period. `detect-anomaly-aql` owns it, including the baseline length and the full-window rule. @@ -91,49 +69,52 @@ reporting: ... filters: ... ``` -Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those are the **anomaly results**, the summary's source of truth; the AQL itself goes to Step 3 unchanged. Then post the rule in the reader's terms, naming the baseline length and `k`. +Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those are the **anomaly results**, the summary's source of truth; the AQL itself goes to Step 3 unchanged. -### Step 3: Show the expectation, not just the verdict +Read the returned rows for the three conditions that change what you can promise: +- **Missing buckets** between the first and last. Gaps break the period-to-period comparison the method rests on. +- **A final bucket still in progress.** A part-period value reads as a collapse. Exclude it, or say it is incomplete. +- **Row count.** A short series is not an error: it comes back unassessed rather than flagged. Say so instead of reporting that nothing was unusual. -The chart is the evidence for the verdict, whichever way the verdict goes: it shows where each period was expected to land, how wide normal was at that moment, and whether the series stayed inside. **A run that flags nothing needs it as much as one that flags three.** Without the band, "nothing was unusual" is an assertion the reader has no way to check. Draw it onto the same picture from Step 1. +Then post the rule in the reader's terms, naming the baseline length and `k`, and anything the three checks turned up. -First, post the anomaly definition: -> **What counts as unusual here.** An anomaly is any [grain] whose value departs sharply from where its recent trend was heading: a change much larger or smaller than [metric]'s normal [grain]-to-[grain] movement over the prior [W] [grains]. +### Step 3: Draw the anomaly chart -Then draw the chart: +Draw it even when nothing is flagged: without the band, "nothing was unusual" is an assertion the reader has no way to check. This is the run's only chart, so it carries the series as well as the band. + +Draw the chart first: ``` /detect-anomaly-viz -chart: anomaly dataset: ... aql: grain: ... metric label: ... ``` +Then post the anomaly definition, so it lands with the band already on screen: +> **What counts as unusual here.** An anomaly is any [grain] whose value departs sharply from where its recent trend was heading: a change much larger or smaller than [metric]'s normal [grain]-to-[grain] movement over the prior [W] [grains]. + Then post how to read it: the band is the expected range given the recent trend, it widens when movement has been erratic, and the red columns mark what broke out. -### Step 4: Read the result honestly +### Step 4: Write the summary -A period that was never assessed carries `is_anomaly = 0` exactly like one that was assessed and found normal, and the lead-in buckets sit in the result set alongside real findings. Both read as "normal" unless checked for. +`is_anomaly = 0` means two different things — a normal bucket *and* a lead-in bucket never assessed (too little history behind it). They look identical unless you check `z_score` first, so classify each bucket in the reporting window in this order: +1. **Not assessed** — `z_score` is null. Context, never a finding. +2. **Unusual** — `z_score` non-null and `is_anomaly = 1`. These are the findings. +3. **Normal** — `z_score` non-null and `is_anomaly = 0`. -Classify every bucket in the reporting window, **in this order**: -1. **Not assessed.** `z_score` is null: a lead-in bucket without a full window of prior history. `is_anomaly` is `0` for these too, so testing it first misreports them as normal. -2. **Unusual.** `z_score` non-null and `is_anomaly = 1`. -3. **Normal.** `z_score` non-null and `is_anomaly = 0`. +Write the summary (Output → *Summary*) covering only what was assessed, not what was asked for: if any bucket in the period came back with a null `z_score`, name the span actually judged. Lead-in buckets are context, never findings. -Then check, and write the summary (Output → *Summary*): -- **Only buckets inside the reporting timeframe are reported.** The lead-in buckets are context, never findings. -- **The claim covers what was assessed, not what was asked for.** If any bucket in the period came back with a null `z_score`, the summary names the span actually judged (Output → *Summary*). -- **`W` and `k` are the values from `detect-anomaly-aql`**, never adjusted for this series. -- **The anomaly chart was drawn**, including when nothing was flagged, or the fallback was taken and said so. -- **No causal or dimensional claim** anywhere in the prose. -- **Numbers formatted** per Conventions → *Numeric formatting*. +Before writing, check: +- `W` and `k` are the values from `detect-anomaly-aql`, never adjusted for this series. +- The anomaly chart was drawn (even when nothing was flagged), or the fallback was taken and said so. +- No causal or dimensional claim anywhere in the prose. Numbers per Conventions → *Numeric formatting*. -Two patterns in the flagged set change what the summary can claim. Check both: -- **Flags sharing a calendar position** (the same month each year, the same weekday). The method has no seasonal term, so a recurring peak flags as an anomaly. Say so, and offer the seasonality-aware re-run. -- **Two flags within `W` buckets of each other.** The first widens the band for the periods after it, so the second was judged against a looser threshold and anything following it may have been missed. +Two patterns in the flagged set change what the summary can claim: +- **Same calendar position** (same month each year, same weekday). The method has no seasonal term, so a recurring peak flags as an anomaly. Say so, and offer the seasonality-aware re-run. +- **Two flags within `W` buckets of each other.** The first widens the band after it, so the second was judged against a looser threshold and anything following it may have been missed. ## Output @@ -185,11 +166,11 @@ Produce one spec `{ dataset, M, T, granularity, filters, timeframe }`. **One fil - **A legend/breakdown**: drop it and analyze the total, unless the user explicitly asks per-segment (Conventions → *Out of scope*). - **No chart, and the metric cannot be parsed**, or `list_datasets()` is unsupported (dev mode): ask the user to name the dataset and metric explicitly. Do not proceed. - **Confirm only what you guessed** (*"I'll use `sales_orders` (measure `total_revenue`). Confirm, or name another."*), then wait. Fields read from the chart or stated by the user are accepted silently. -- The timeframe resolved here is the **reporting** window. `detect-anomaly-aql` opens the filter earlier to supply the lead-in; those buckets are context on both charts and are never reported as findings. +- The timeframe resolved here is the **reporting** window. `detect-anomaly-aql` opens the filter earlier to supply the lead-in; those buckets are context on the chart and are never reported as findings. ## Conventions - **On an error, retry once, then fall back.** Step 2: surface the error, never imply detection completed. Step 3: fall back to the Step 2 table plus prose. Never loop more than one retry per step. -- **Titles.** `execute_aql` / `execute_viz` require a `title` (under ~60 chars; avoid "query 1"/"untitled"): Step 1 series chart → ` by ()`; Step 2 anomaly results → ` anomaly results ()`; Step 3 anomaly chart → `Anomaly detection: ()`. +- **Titles.** `execute_aql` / `execute_viz` require a `title` (under ~60 chars; avoid "query 1"/"untitled"): Step 2 anomaly results → ` anomaly results ()`; Step 3 anomaly chart → `Anomaly detection: ()`. ### Numeric formatting - Currency-like measures (label/name contains `revenue`, `cost`, `price`, `arr`, `mrr`, `gmv`, `amount`): `$` prefix, short suffix (`$500K`, `$1.2M`); use the dataset's stated currency symbol if metadata provides it. From 5347a8ba5af65cbb70fd6bc4b0f881bedf041648 Mon Sep 17 00:00:00 2001 From: truongthinhnguyen03 Date: Fri, 21 Aug 2026 17:29:20 +0700 Subject: [PATCH 17/17] refactor(native.system): drop the fixed baseline window from detect-anomaly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The baseline was a fixed 12 buckets, which meant a bucket needed twelve periods behind it before it could be judged. Points near the start of a series came back unassessed, and the query had to widen its own time filter to fetch a lead-in. Replace it with an expanding frame (`..-1`): every bucket is judged against everything before it, so there is no window length to choose and nothing to widen. The opening buckets exclude themselves, because `window_stdev` is a sample stddev and is null below two values — the `window_count` gate and its `case` are gone. The time filter now carries an upper bound only; with an expanding baseline, cropping the fetch rescales every score rather than returning fewer rows, so both skills say not to. The cost is that the baseline no longer forgets: an unusual bucket widens the band for everything after it permanently. The summary's contamination note changed from "two flags within W buckets" to "a flag early in the series" to match. Also add a `verdict` column ('unusual' / 'normal' / 'not assessed') and rename `is_anomaly` to `anomaly_flag`. The summary previously had to read `z_score` for null and `is_anomaly` for 0/1 to tell an unassessed bucket from a normal one, since the flag is 0 for both. It now reads one field carrying one word. `anomaly_flag` stays numeric and never null because the chart plots it as a column series. Co-Authored-By: Claude Opus 5 (1M context) --- .../detect-anomaly-aql/SKILL.md | 67 ++++++++++++------- .../detect-anomaly-viz/SKILL.md | 9 +-- .../holistics-common/detect-anomaly/SKILL.md | 54 ++++++++------- 3 files changed, 80 insertions(+), 50 deletions(-) diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index bdd6138..0ff5a3d 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -10,36 +10,51 @@ Detect anomalies by working out what change to expect in each period from the pe Everything is computed on **changes**, not levels: `Δ` = the change from the previous bucket. That is what lets the band follow a trend, since a steadily climbing series has large values but ordinary changes, and so never trips on the climb itself. -For each bucket `t` (`T` truncated to ``), over the `W` buckets before it: +For each bucket `t` (`T` truncated to ``), over **every bucket before it**: - `avg_change` = mean of `Δ`. The change to expect. - `spread` = stddev of `Δ`. How much that change normally varies. - `expected` = previous `actual` + `avg_change`. Where this bucket should land. - `lower/upper_bound` = `expected` ∓ `k · spread`. - `z_score` = (`actual` − `expected`) / `spread`. How far off it landed, counted in `spread`s. -- `is_anomaly` = `1` when `abs(z_score) > k`, else `0`. `k` default 3 (looser 2, stricter 4). +- `anomaly_flag` = `1` when `abs(z_score) > k`, else `0`. `k` default 3 (looser 2, stricter 4). For the chart only. +- `verdict` = `'unusual'`, `'normal'`, or `'not assessed'`. The one field the caller reads. -**`W` is 12, at every grain.** Twelve is where the spread estimate stops improving much, and a longer baseline costs assessed buckets without buying precision. Do not vary it by grain, by series length, or by anything else. +**There is no baseline length to set.** The frame is `..-1` — the first bucket of the series through the bucket before this one. Every bucket is judged against everything that came before it, so nothing has to be chosen per grain, per series, or per run. -**A full window is required.** Check `n_prior = window_count(...) >= W`, so a bucket is banded and flagged only once it has `W` buckets behind it, and gets `z_score` null otherwise. Without that check the opening `W` buckets false-flag, because a window function over a partial frame returns a value rather than null. Those opening buckets are the **lead-in**: carried for context, never judged. +**The opening buckets exclude themselves.** `spread` is a *sample* standard deviation, undefined for fewer than two values, so it comes back null until a bucket has at least two prior changes behind it — the fourth bucket of a series. `expected` is null before that too, since the first bucket has no previous value to project from. Nothing gates this: the nulls arise on their own. Those opening buckets are the **lead-in**: carried for context, never judged. -The gate is also what makes a short series safe, with no branching and no arithmetic: too little history returns nulls rather than an error. Read the result instead of predicting it; the caller reports what came back assessed. +**Nulls do all the branching.** `z_score` is arithmetic on `expected` and `spread`, so a null in either carries through, and `lower_bound` / `upper_bound` go null with it. A short series is therefore safe with no gate and no arithmetic: too little history returns nulls rather than an error. Read the result instead of predicting it; the caller reports what came back assessed. + +**`verdict` turns those nulls into a word, so the caller never reads a null.** Its three branches are tried in order, and the second one is what makes this work without a null test: + +- `abs(z_score) > k` → `'unusual'` +- `abs(z_score) >= 0` → `'normal'` — true for every real number, and *not* true when `z_score` is null +- otherwise → `'not assessed'` + +A null `z_score` fails both comparisons and falls to the `else`, so every row comes back carrying a literal word. Two things land in `'not assessed'`: a bucket with fewer than two prior changes behind it, and a perfectly flat stretch, where `spread` is `0` and `safe_divide` returns null. Both are correct — there is nothing to judge against in either case. + +**The baseline never forgets, and that is the cost.** With an expanding frame an unusual bucket raises `spread` for every bucket after it, permanently — there is no window for its influence to fall out of. One violent early spike can widen the band enough to hide milder anomalies for the rest of the series. The caller says so when the flagged set makes it relevant. ### Query -A complete query: monthly `total_revenue`, reporting on 2016 and 2017, `W` 12, `k` 3. +A complete query: monthly `total_revenue`, reporting on 2016 and 2017, `k` 3. ```aql metric m_prev = window_avg(total_revenue, -1..-1, order: orders.created_at | month() | asc(), partition: []); metric m_delta = total_revenue - m_prev; -metric m_n = window_count(total_revenue, -12..-1, order: orders.created_at | month() | asc(), partition: []); -metric m_avg_change = window_avg(m_delta, -12..-1, order: orders.created_at | month() | asc(), partition: []); -metric m_spread = window_stdev(m_delta, -12..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_avg_change = window_avg(m_delta, ..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_spread = window_stdev(m_delta, ..-1, order: orders.created_at | month() | asc(), partition: []); metric m_expected = m_prev + m_avg_change; -metric m_z = case(when: m_n >= 12, then: safe_divide(total_revenue - m_expected, m_spread), else: null); -metric m_lower = case(when: m_n >= 12, then: m_expected - 3 * m_spread, else: null); -metric m_upper = case(when: m_n >= 12, then: m_expected + 3 * m_spread, else: null); -metric m_anom = case(when: and(m_n >= 12, abs(m_z) > 3), then: 1, else: 0); +metric m_z = safe_divide(total_revenue - m_expected, m_spread); +metric m_lower = m_expected - 3 * m_spread; +metric m_upper = m_expected + 3 * m_spread; +metric m_anom = case(when: abs(m_z) > 3, then: 1, else: 0); +metric m_verdict = case( + when: abs(m_z) > 3, then: 'unusual', + when: abs(m_z) >= 0, then: 'normal', + else: 'not assessed' +); explore { dimensions { bucket: orders.created_at | month() @@ -50,10 +65,11 @@ explore { lower_bound: m_lower, upper_bound: m_upper, z_score: m_z, - is_anomaly: m_anom + anomaly_flag: m_anom, + verdict: m_verdict } filters { - orders.created_at matches @2015-01-01 - 2017-12-31 + orders.created_at < @2018-01-01 } sorts { bucket asc nulls last @@ -61,24 +77,29 @@ explore { } ``` -Substitute `total_revenue` with `M`, `orders.created_at` with `T`, `month()` with the grain, and `3` with `k`. Leave `12` and `-1..-1` exactly as written: `W` is always 12, and `-1..-1` is the previous bucket. +Substitute `total_revenue` with `M`, `orders.created_at` with `T`, `month()` with the grain, and `3` with `k`. Leave both frames exactly as written: `..-1` is every bucket before this one, and `-1..-1` is the previous bucket. + +**The time filter has no lower bound.** Above, the report covers 2016 and 2017, but the filter carries no start date at all: the baseline is every prior bucket, so the fetch has to reach as far back as the data goes. Write it as an upper bound only — `orders.created_at < @2018-01-01` — and let the caller report on the range it was asked about. -**The filter opens `W` buckets before the reporting window.** Above, the report covers 2016 and 2017 and `W` is 12, so the filter opens at 2015-01-01 and those twelve months arrive as the lead-in. A relative window widens the same way: a 24-month report becomes `matches @(last 36 months)`. +**Cropping the fetch changes the numbers, it does not just shorten them.** With an expanding baseline, every `expected`, `spread` and `z_score` depends on how far back the query reaches, so the same bucket scores differently under a narrower filter. Never crop to the range a chart happens to display, and never re-crop between the Step 2 query and the Step 3 chart — which is why the chart is drawn from this explore verbatim. **Dimensional filters go in the same block**, one per line: ```aql filters { - orders.created_at matches @2015-01-01 - 2017-12-31 + orders.created_at < @2018-01-01 orders.region == "West" } ``` **Do not regress these:** -- Count prior buckets with `window_count(M, -W..-1, …)`, **never** `window_sum(1, …)` (SQL-generation error). -- The `n_prior ≥ W` gate is mandatory. Partial windows return values, not null, and would false-flag the first `W` buckets. -- `case(when: …, else: null)` is valid and is how the band is hidden in the lead-in. +- The baseline frame is `..-1`, **not** a fixed count like `-12..-1`. This method has no baseline length; do not reintroduce one, and do not add an `n_prior` gate to compensate. +- `window_stdev` is the **sample** standard deviation, and that is what makes the lead-in self-exclude: it is null below two values. `window_stdevp` (population) returns `0` there instead, and a zero spread scores every opening bucket as an extreme anomaly. +- Nulls carry the lead-in through `m_expected` and `m_spread` into `m_z`, `m_lower`, `m_upper` and `m_anom`. Nothing downstream needs a guard of its own — do not add one. +- **`verdict` and `anomaly_flag` are not redundant.** `verdict` is the caller's only input and carries three states; `anomaly_flag` exists solely so the chart has a numeric column series. Never drop one for the other, and never have the caller read `anomaly_flag`. - The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly: one explore, no two-stage query. -- `is_anomaly` must always return `0` or `1`, **never `null`**. The `else: 0` is mandatory: it is plotted as a column, and nulls cause rendering gaps instead of clean zeros. -- `z_score` must be null whenever `n_prior < W`. The caller distinguishes unassessed buckets from normal ones by that null, since `is_anomaly` is `0` for both. +- `anomaly_flag` must always return `0` or `1`, **never `null`**. The `else: 0` is mandatory: it is plotted as a column, and nulls cause rendering gaps instead of clean zeros. This is also why it cannot carry the third state and `verdict` exists. +- `verdict`'s second branch must stay `abs(m_z) >= 0`, not a null test. It is what routes a null `z_score` to `'not assessed'` without any null-handling function. +- `verdict` must never return null. Every row carries one of the three words. +- **The fetch must reach the start of the series.** It is the only thing standing in for a baseline length now, and a bounded lower bound silently rescales every score in the result. - Dates are **literals, not calls**: `@2016-01-01`, `@2016-01-01 - 2017-12-31`, `@(last 12 months)`. There is no `@date(...)` function, and writing one is a syntax error at the `(`. diff --git a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md index 98966a8..8151729 100644 --- a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md @@ -5,7 +5,7 @@ description: |- user-invocable: false --- -One chart, drawn in Step 3: the metric series with its expected band and the flagged buckets. It spans the reporting window plus `W` lead-in buckets, so the band has history to form against. +One chart, drawn in Step 3: the metric series with its expected band and the flagged buckets. It spans whatever the Step 2 explore returned — the series from its start — so the band has history to form against. Never hand-write the viz body. `generate_viz` writes it, `execute_viz` validates it on execution. State the decoration explicitly in the `query`, because `generate_viz` defaults its palette and `pattern: 'inherited'` unless told otherwise. @@ -17,7 +17,7 @@ Parameters: `dataset`, `aql` (the Step 2 explore, verbatim), `grain`, `metric la The series, plus the band and the flags. The Step 2 explore already returns every field this chart needs, so it is passed through untouched and no filters or timeframe are supplied again. -1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : is_anomaly as red #FCB8B8 columns on the FIRST y-axis, 0..1 scale; actual as a solid #255DD4 line and lower_bound and upper_bound as grey #9CA3AF dashed lines, all three on the SECOND y-axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")` +1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : anomaly_flag as red #FCB8B8 columns on the FIRST y-axis, 0..1 scale; actual as a solid #255DD4 line and lower_bound and upper_bound as grey #9CA3AF dashed lines, all three on the SECOND y-axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")` 2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust decoration only. 3. `execute_viz(dataset_uname: , viz: , title: …)` @@ -26,6 +26,7 @@ On an error, feed the error text back into `generate_viz`'s `query` (it self-cor ### Do not regress - Pass only `dataset_uname`, `viz`, `title` to `execute_viz`. No `aql` property. - 6-digit hex only. An 8-digit alpha hex can be rejected. -- `is_anomaly` goes on the **first** y-axis (0..1) and the metric, `lower_bound` and `upper_bound` on the **second**. That order is what keeps the columns behind the lines. -- The chart covers ` + W`. Cropping to the reporting window alone hides the lead-in the band needs to form, and shows the series starting later than the results do. +- `anomaly_flag` goes on the **first** y-axis (0..1) and the metric, `lower_bound` and `upper_bound` on the **second**. That order is what keeps the columns behind the lines. +- Chart `anomaly_flag`, never `verdict`. `verdict` is a text field for the prose; it has no numeric series to plot. +- The chart covers everything the Step 2 explore returned. Cropping to the reporting window hides the lead-in the band needs to form, and shows the series starting later than the results do. Do not re-filter here: the explore's own range is what its `expected`, `spread` and `z_score` were computed against. - For a metric that cannot go negative, clamp the **displayed** lower bound at 0. The band sits around the trend rather than the level, so it can dip below zero. diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index 86a79fd..35a76f6 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -1,7 +1,7 @@ --- name: detect-anomaly description: |- - Detect statistical anomalies across a metric's history and visualize them: unusual spikes, drops, and deviations flagged against an expected range built from the metric's own recent history. + Detect statistical anomalies across a metric's history and visualize them: unusual spikes, drops, and deviations flagged against an expected range built from the metric's own history. Use when the user wants to find or check for unusual values in a metric. Trigger words include anomaly, unusual, spike, drop, dip, outlier, "out of the ordinary", "anything weird". A specific date is optional; the skill always scans the whole series. @@ -11,13 +11,13 @@ description: |- user-invocable: false --- -Build an expected band from a metric's recent history, flag the buckets that break out of it, and chart the result. This skill identifies which points are unusual, not why they moved (Conventions → *Out of scope*). +Build an expected band from a metric's history, flag the buckets that break out of it, and chart the result. This skill identifies which points are unusual, not why they moved (Conventions → *Out of scope*). Two sub-skills carry the mechanics: **`detect-anomaly-aql`** owns the method and the query, **`detect-anomaly-viz`** owns the chart. ## What a good input looks like -Anomaly detection compares a metric against its own recent history to determine whether a value is far from what the preceding periods would suggest. The analysis needs one measure, the date field and grain it is measured on, and enough prior history to establish what normal movement looks like. +Anomaly detection compares a metric against its own history to determine whether a value is far from what the preceding periods would suggest. The analysis needs one measure, the date field and grain it is measured on, and enough prior history to establish what normal movement looks like. A task is fully specified when you can answer: @@ -25,7 +25,7 @@ A task is fully specified when you can answer: 2. **Which date field, and at what grain.** Day, week, month, or quarter. The grain sets what can be found: a spike lasting two days does not show up in monthly buckets, and a metric recorded monthly cannot be read daily. 3. **Which slice of the data.** The segment in view, if any. The result describes the series you selected, not the business as a whole: one region collapsing can leave the company total flat, and a flat total can hide two segments moving in opposite directions. One slice per analysis; breaking the metric down by a dimension is a different question. 4. **Which period the findings cover.** Flagged buckets are reported only inside this period. Earlier buckets are shown as context, because the method needs prior history before it can judge a bucket. -5. **Whether the user has a specific date in question.** A named date does not narrow the analysis; the whole series is always scanned. It changes only what the answer leads with. +5. **Whether the user has a specific date in question.** A named date does not narrow the analysis; the whole series is always scanned. It changes what the answer leads with — and, when that bucket falls in the lead-in, whether there is a verdict to lead with at all. The user rarely supplies all five, and often has not asked for anomaly detection by name. @@ -42,7 +42,7 @@ One chart and a prose summary. Four steps. Each is a move an analyst makes, and each closes with a `post_update` so the user follows the reasoning as it happens instead of receiving a verdict at the end. **Narrating is not pausing**: post and keep working. The only stop in the run is the measure question in Step 1. -Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the baseline window, always 12, `k` = the sensitivity threshold, `` = the dataset name, `` = a readable label. +Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `k` = the sensitivity threshold, `` = the dataset name, `` = a readable label. ### Step 1: Establish the series @@ -54,7 +54,7 @@ Post the metric, grain, slice and period as real dates, and that the whole serie ### Step 2: Run the detection -The rule: predict each period from where the last `W` periods of movement were heading, then judge the gap between actual and prediction against how much this metric usually moves period to period. `detect-anomaly-aql` owns it, including the baseline length and the full-window rule. +The rule: predict each period from where the metric's movement so far was heading, then judge the gap between actual and prediction against how much this metric usually moves period to period. There is no baseline length to choose — every period is judged against everything before it. `detect-anomaly-aql` owns the method and the query. Hand over the resolved parameters: @@ -71,12 +71,13 @@ filters: ... Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those are the **anomaly results**, the summary's source of truth; the AQL itself goes to Step 3 unchanged. -Read the returned rows for the three conditions that change what you can promise: +Read the returned rows for the four conditions that change what you can promise: - **Missing buckets** between the first and last. Gaps break the period-to-period comparison the method rests on. - **A final bucket still in progress.** A part-period value reads as a collapse. Exclude it, or say it is incomplete. - **Row count.** A short series is not an error: it comes back unassessed rather than flagged. Say so instead of reporting that nothing was unusual. +- **The focus bucket's `verdict`, when the user pointed at one.** `not assessed` means the run cannot answer the question it was asked, however much of the rest of the series came back judged. Read it here rather than while writing the summary: it decides what the answer leads with. -Then post the rule in the reader's terms, naming the baseline length and `k`, and anything the three checks turned up. +Then post the rule in the reader's terms, naming `k`, and anything the four checks turned up. ### Step 3: Draw the anomaly chart @@ -94,27 +95,32 @@ metric label: ... ``` Then post the anomaly definition, so it lands with the band already on screen: -> **What counts as unusual here.** An anomaly is any [grain] whose value departs sharply from where its recent trend was heading: a change much larger or smaller than [metric]'s normal [grain]-to-[grain] movement over the prior [W] [grains]. +> **What counts as unusual here.** An anomaly is any [grain] whose value departs sharply from where the trend was heading: a change much larger or smaller than [metric]'s normal [grain]-to-[grain] movement across its history up to that point. -Then post how to read it: the band is the expected range given the recent trend, it widens when movement has been erratic, and the red columns mark what broke out. +Then post how to read it: the band is the expected range given the trend so far, it widens when movement has been erratic, and the red columns mark what broke out. ### Step 4: Write the summary -`is_anomaly = 0` means two different things — a normal bucket *and* a lead-in bucket never assessed (too little history behind it). They look identical unless you check `z_score` first, so classify each bucket in the reporting window in this order: -1. **Not assessed** — `z_score` is null. Context, never a finding. -2. **Unusual** — `z_score` non-null and `is_anomaly = 1`. These are the findings. -3. **Normal** — `z_score` non-null and `is_anomaly = 0`. +Read `verdict`. It is one field per bucket carrying one of three words, and it is the only thing that decides how a bucket is treated: -Write the summary (Output → *Summary*) covering only what was assessed, not what was asked for: if any bucket in the period came back with a null `z_score`, name the span actually judged. Lead-in buckets are context, never findings. +| `verdict` | What it is | +|---|---| +| `unusual` | A finding. These are what the summary lists. | +| `normal` | Assessed, nothing to report. | +| `not assessed` | Context, never a finding — too little history behind it, or no variation to judge against. | + +Do not infer any of this from `z_score` or `anomaly_flag`. `anomaly_flag` is `0` for both `normal` and `not assessed`, which is exactly the confusion `verdict` exists to remove; it is the chart's column series, not an input to the prose. `z_score` is only for phrasing how far a finding fell outside the band. + +Write the summary (Output → *Summary*) covering only what was assessed, not what was asked for: if any bucket in the period came back `not assessed`, name the span actually judged. Before writing, check: -- `W` and `k` are the values from `detect-anomaly-aql`, never adjusted for this series. +- `k` is the value from `detect-anomaly-aql`, never adjusted for this series. - The anomaly chart was drawn (even when nothing was flagged), or the fallback was taken and said so. - No causal or dimensional claim anywhere in the prose. Numbers per Conventions → *Numeric formatting*. Two patterns in the flagged set change what the summary can claim: - **Same calendar position** (same month each year, same weekday). The method has no seasonal term, so a recurring peak flags as an anomaly. Say so, and offer the seasonality-aware re-run. -- **Two flags within `W` buckets of each other.** The first widens the band after it, so the second was judged against a looser threshold and anything following it may have been missed. +- **A flag early in the series.** The baseline is every prior bucket, so an unusual value widens the band for everything after it and never falls back out. Anything milder later was judged against a looser threshold and may have been missed. Say so whenever the first flag lands well before the others. ## Output @@ -141,10 +147,12 @@ Then the conventions: - **Markup carries the scan.** A bold label opens each part; each finding leads with its date and value in bold, then a plain sentence for the assessment. No headings, no tables. - **One line per flagged bucket, chronological**, each carrying the date, the value, the direction, and how far outside the band it fell in plain terms. - **The user has the chart.** Do not repeat the results as a table unless asked. -- **Nothing flagged**: draw the chart anyway, then say so and name the period. *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with its recent trend."* -- **Nothing assessed**: when no bucket came back with a non-null `z_score`, the series was too short for the method to judge anything. Say that, never that nothing was unusual: *"Not enough history to assess GMV. The method needs 12 months of movement behind a month before it can judge it, and this series starts in Jan 2024."* -- **Partly assessed**: when only some of the period was judged, name the span that was. *"Assessed GMV from Jul 2024 onward; the months before it had too little history behind them."* Findings and the "nothing unusual" claim both cover that span only, never the full period. -- **A date was named**: lead with the verdict on that bucket, unusual, normal, or not assessed for want of history, then the same list. The full series is still scanned. +- **Report the assessed span, and no more.** The assessed span is the buckets whose `verdict` is not `not assessed`. Name it whenever it differs from the period that was asked for; findings and any "nothing unusual" claim cover that span only. The chart ships first in every case. Four shapes fall out of the one rule: + - *Fully assessed, nothing flagged* — *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with the trend up to it."* + - *Partly assessed* — *"Assessed GMV from Jul 2024 onward; the months before it had too little history behind them."* + - *Nothing assessed* — every bucket came back `not assessed`. *"Not enough history to assess GMV. The method needs a few months of movement behind a month before it can judge it, and this series starts in Nov 2025."* Never that nothing was unusual. When the cause is a flat series rather than a short one, say that instead: *"GMV held at exactly $40K every month, so there is no variation to judge a month against."* + - *A named date outside the assessed span* — lead with that, then the list for what was assessed: *"I can't judge Feb 2026 — it is one of the opening months of the series, and the method needs a few months of movement behind a month before it can judge it. GMV starts in Jan 2026."* +- **A named date inside the assessed span**: lead with the verdict on that bucket, unusual or normal, then the same list. The full series is scanned either way. - **The closing pointer** appears only when something was flagged. - **The re-run offer goes last.** `k` is the only sensitivity control and is never asked upfront (values in `detect-anomaly-aql`). On accept, re-run Steps 2 to 4 with the new `k`. @@ -166,7 +174,7 @@ Produce one spec `{ dataset, M, T, granularity, filters, timeframe }`. **One fil - **A legend/breakdown**: drop it and analyze the total, unless the user explicitly asks per-segment (Conventions → *Out of scope*). - **No chart, and the metric cannot be parsed**, or `list_datasets()` is unsupported (dev mode): ask the user to name the dataset and metric explicitly. Do not proceed. - **Confirm only what you guessed** (*"I'll use `sales_orders` (measure `total_revenue`). Confirm, or name another."*), then wait. Fields read from the chart or stated by the user are accepted silently. -- The timeframe resolved here is the **reporting** window. `detect-anomaly-aql` opens the filter earlier to supply the lead-in; those buckets are context on the chart and are never reported as findings. +- The timeframe resolved here is the **reporting** window: what the findings cover. `detect-anomaly-aql` queries the series from its start regardless, because the baseline is every prior bucket; the buckets before the reporting window are context on the chart and are never reported as findings. ## Conventions - **On an error, retry once, then fall back.** Step 2: surface the error, never imply detection completed. Step 3: fall back to the Step 2 table plus prose. Never loop more than one retry per step. @@ -181,4 +189,4 @@ Produce one spec `{ dataset, M, T, granularity, filters, timeframe }`. **One fil This skill says which points broke from the trend. It does not say why, and it does not attribute a break to a segment. - **"Why did this happen?" / "What dimension drove it?"** Reply: "I can flag which points broke from the trend, but identifying which dimension drove an anomaly, or its business cause, is outside this skill. Break the metric down by relevant dimensions in a dashboard, or check your marketing calendar / CRM / news for that window." - **A per-segment run.** Out of scope: one slice per analysis. Offer a separate run on the segment the user names. -- **"Show the raw numbers."** Show the `execute_aql` anomaly results directly: `bucket`, `actual`, `expected`, `lower_bound`, `upper_bound`, `z_score`, `is_anomaly`. Never group or pivot rows by `is_anomaly`. +- **"Show the raw numbers."** Show the `execute_aql` anomaly results directly: `bucket`, `actual`, `expected`, `lower_bound`, `upper_bound`, `z_score`, `verdict`. Drop `anomaly_flag` — it duplicates `verdict` for the chart's benefit. Never group or pivot rows by `verdict`.