diff --git a/native-skills/system/holistics-common/analyze-changes/SKILL.md b/native-skills/system/holistics-common/analyze-changes/SKILL.md index 27c9747..ae1acc6 100644 --- a/native-skills/system/holistics-common/analyze-changes/SKILL.md +++ b/native-skills/system/holistics-common/analyze-changes/SKILL.md @@ -57,13 +57,23 @@ Brief: Parse the user's message for: - `metric` — the metric that changed (e.g., "revenue", "close rate", "daily signups") - `direction` — drop or increase -- `time_reference` — when the change occurred (e.g., "in 2026", "last quarter", "since January") +- `base_period` — the period where the change occurred (e.g., "May 2025") +- `comparison_period` — the reference period (e.g., "April 2025") If `metric` is not clearly named, ask before proceeding: > "Which metric are you looking at?" -Once the metric is confirmed, echo a one-line summary: -> "Got it — you're seeing a [drop/increase] in **[metric]** [time_reference]." +Fetch `current_value` (value in `base_period`) and `comparison_value` (value in `comparison_period`) +from the dataset. Compute `delta` = current_value − comparison_value. + +Post an update to confirm the actual values: +> "Got it — you're seeing a [drop/increase] in **[metric]** from **[comparison_value]** +> ([comparison_period]) to **[current_value]** ([base_period]), [formatted delta]." + +Delta formatting: +- Percentage metric (rate, %, ratio): "down/up **[|delta|] percentage points**" +- Currency: "down/up **$[|delta|]** ([relative%]%)" +- Count or other: "down/up **[|delta|]** ([relative%]%)" ``` Execution: Do this yourself, no delegation. @@ -74,7 +84,7 @@ Dependency: Step 1 Brief: ``` -/detect_anomoly +/detect_anomaly ``` Execution: @@ -86,7 +96,7 @@ Execution: ### Step 3 — Output the Stage 1 Summary Block -Depdendency: Step 2 +Dependency: Step 2 Brief: ``` diff --git a/native-skills/system/holistics-common/analyze-contribution/SKILL.md b/native-skills/system/holistics-common/analyze-contribution/SKILL.md index cdd2b7d..a91157f 100644 --- a/native-skills/system/holistics-common/analyze-contribution/SKILL.md +++ b/native-skills/system/holistics-common/analyze-contribution/SKILL.md @@ -163,7 +163,7 @@ Run only Steps 1 and 2, then proceed directly to Phase 3 using `user_specified_d **If `user_specified_dimensions` is empty — standard path:** -Execute these five steps in the exact order below. Do not reorder, skip, or batch any step. +Execute these four steps in the exact order below. Do not reorder, skip, or batch any step. **Step 1 — Output the overview sentence as plain text:** @@ -175,11 +175,11 @@ Execute these five steps in the exact order below. Do not reorder, skip, or batc The table must include these columns: Comparison Value, Base Value, Delta Value, Pct Change. -**Step 3 — Output the transition sentence as plain text:** +**Step 3 — Post this update** > **There are multiple ways to explain what drove this change. In this analysis, I'll break it down by dimensions to identify which segments moved the most.** Here are my top [N] proposed dimensions: -**Step 4 — Output the dimension list as plain text.** This is a text output step, not a tool call. Write the numbered list directly in your response. Each item must include a short reason — **5–8 words max, business-focused, no technical jargon**: +**Step 4 — Output the dimension list as plain text, then proceed directly to Phase 3.** This is a text output step, not a tool call. Write the numbered list directly in your response. Each item must include a short, business-meaningful reason — **1 sentence max, focused on why this dimension is relevant to the metric change**: > **1. Dimension Name** — short business reason (e.g. "varies most by sales team", "differs by product line") > @@ -187,37 +187,7 @@ The table must include these columns: Comparison Value, Base Value, Delta Value, > > **3. Dimension Name** — short business reason -**Step 5 — Output the Stage 1 Summary Block as plain text, then STOP:** - -> --- -> Here's a summary of what we're going to analyze: -> -> - **Metric**: [metric name] -> - **Comparison period**: [comparison_period_literal] ([comparison_period_name]) -> - **Base period**: [base_period_literal] ([base_period_name]) -> - **Overall change**: [metric] [dropped/increased] [pct_change]%, from [comparison_value] to [base_value] (Δ [±overall_delta_value]) -> - **Dimension 1**: [dim1] -> - **Dimension 2**: [dim2] -> - **Dimension 3**: [dim3] -> -> Shall I proceed to break down **[metric]** by each of these dimensions to see where the [drop/increase] happened the most? -> --- - -**Do NOT call `ask_user` here.** Stop after outputting this block and wait for the user to reply in the chat. - ---- - -### Stage 2 Detection — check before Phase 3 - -Before running Phase 3, check the conversation history: - -- **If a Stage 1 Summary Block is present** (contains confirmed metric, periods, overall delta, and dimension list): - - Check the user's most recent reply: - - **Affirmative** (e.g. "yes", "proceed", "looks good") → load `metric`, `base_period_literal`, `comparison_period_literal`, `overall_delta_value`, and confirmed dimension list from the block. Do NOT re-run Phases 1 or 2. Proceed directly to Phase 3 with dimension #1. - - **Requests changes** (e.g. "remove X", "swap Y for Z", "add Y", "use X instead") → update the dimension list based on the feedback, output a new Stage 1 Summary Block with the updated list, then proceed directly to Phase 3. Do not stop for another confirmation. - - **Negative** (e.g. "no", "stop", "that's enough", "skip it") → stop. Output a single closing sentence: "Got it — stopping here. Let me know if you'd like to explore any dimension further." Do not proceed to Phase 3. - -- **If no Stage 1 Summary Block is present**: this is a fresh invocation — run from Phase 1. +After outputting the dimension list, proceed immediately to Phase 3. Do NOT stop or wait for user input. --- @@ -225,15 +195,7 @@ Before running Phase 3, check the conversation history: **Plan (once, before starting any dimension analysis):** -Post an update about the confirmed dimension list: -> # Analyzing these dimensions -> 1. [dim1] -> 2. [dim2] -> 3. [dim3] - -(Always show this update, even when the user accepted the proposed list without changes.) - -Then add new tasks to analyze the dimensions. +Add new tasks to analyze the dimensions. NOTE: If you are going to delegate, make sure to reference `/analyze-contribution` in the brief, so that sub-agent can follow the prompt structures correctly. @@ -274,7 +236,7 @@ Execute this once per dimension, in confirmed order. Complete the full phase — 6. Compute `contribution_pct = delta_value / overall_delta_value × 100` for each segment using the scalar `overall_delta_value` from Phase 2. -7. Compute and store `dimension_driver_pct = sum(delta_value for all driver_of_change segments) / overall_delta_value × 100` — used to rank dimensions in Phase 4. +7. Compute and store `top_segment_contribution_pct = top_driver_segment_delta_value / overall_delta_value × 100` — the contribution of the single top driver segment; used in the Phase 4 "Contribution to change" column and to rank dimensions. **Show:** @@ -342,15 +304,21 @@ Opening heading and summary line: > > [Metric] [dropped/increased] by [pct_change]% (from [comparison_value] to [base_value], [±abs(delta)]) between [comparison period] and [base period]. -Ranked table of all confirmed dimensions, sorted by `dimension_driver_pct` descending. Successfully analyzed dimensions appear first; failed dimensions appear at the bottom: +Ranked table of all confirmed dimensions, sorted by `top_segment_contribution_pct` descending. Successfully analyzed dimensions appear first; failed dimensions appear at the bottom: | Dimension | Top Segment | Contribution to change | |-----------|-------------|------------------------| | … | | | -For the **Contribution to change** column, apply the same threshold rule as the per-segment display: -- If `|dimension_driver_pct| ≤ 100%`: show `[dimension_driver_pct]%` -- If `|dimension_driver_pct| > 100%`: show `[N]× the net change` where [N] = round(|dimension_driver_pct| / 100, 1) +For the **Contribution to change** column, show `top_segment_contribution_pct` — the contribution of the single top driver segment: + +> `top_segment_contribution_pct = top_driver_segment_delta_value / overall_delta_value × 100` +> +> **Critical:** The denominator is always `overall_delta_value` from Phase 2 (the metric's net change with no segment grouping). Never use the top segment's own delta as the denominator — that always produces 100%. + +Apply the threshold rule: +- If `|top_segment_contribution_pct| ≤ 100%`: show `[top_segment_contribution_pct]%` +- If `|top_segment_contribution_pct| > 100%`: show `[N]× the net change` where [N] = round(|top_segment_contribution_pct| / 100, 1) **Conditionally include (one line each):** diff --git a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md index 9faa8d0..0ff5a3d 100644 --- a/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly-aql/SKILL.md @@ -6,61 +6,70 @@ user-invocable: false --- ### Method -The expected band **follows the trend**. For each bucket `t` (`T` truncated to ``): -- `actual` = `M`; `Δ` = the change from the previous bucket. -- `expected` = previous actual **+ drift**, where drift = the trailing mean of `Δ` over the `W` buckets before `t`. This is the band's centre; it tilts with the trend. -- `spread` = the trailing **stddev of `Δ`** over those `W` buckets. -- `lower/upper_bound` = `expected ∓ k · spread`. -- `z_score` = `(actual − expected) / spread` — is *this bucket's change* unusual versus the recent distribution of changes (stationary even when the level trends). -- `is_anomaly` = `1` when `abs(z_score) > k`, else `0`. `k` default 3 (looser 2, stricter 4). +Detect anomalies by working out what change to expect in each period from the periods before it, then flagging the periods whose actual lands far from that expectation. -**Warm-up gate (required).** A bucket is only banded/flagged once it has a **full `W`-bucket window** of prior history. Partial-frame window stats return values (not null), so gate explicitly on `n_prior = window_count(...) ≥ W`: buckets with `n_prior < W` get **no band** and are **never flagged** — the leading `W` lead-in buckets are exactly these, shown as context. +Everything is computed on **changes**, not levels: `Δ` = the change from the previous bucket. That is what lets the band follow a trend, since a steadily climbing series has large values but ordinary changes, and so never trips on the climb itself. -Window `W` by grain — a whole number of cycles: +For each bucket `t` (`T` truncated to ``), over **every bucket before it**: -| Grain | `W` | -|---|---| -| day | 28 (4 weeks) | -| week | 13 (a quarter) | -| month | 12 (a year) | -| quarter | 8 (two years) | +- `avg_change` = mean of `Δ`. The change to expect. +- `spread` = stddev of `Δ`. How much that change normally varies. +- `expected` = previous `actual` + `avg_change`. Where this bucket should land. +- `lower/upper_bound` = `expected` ∓ `k · spread`. +- `z_score` = (`actual` − `expected`) / `spread`. How far off it landed, counted in `spread`s. +- `anomaly_flag` = `1` when `abs(z_score) > k`, else `0`. `k` default 3 (looser 2, stricter 4). For the chart only. +- `verdict` = `'unusual'`, `'normal'`, or `'not assessed'`. The one field the caller reads. -If the series is shorter than `W + 1`, shrink `W` to about half the available history (minimum 4) and say so. With fewer than 8 points total, stop: *"Not enough history (n=X) to detect anomalies."* +**There is no baseline length to set.** The frame is `..-1` — the first bucket of the series through the bucket before this one. Every bucket is judged against everything that came before it, so nothing has to be chosen per grain, per series, or per run. -### Query +**The opening buckets exclude themselves.** `spread` is a *sample* standard deviation, undefined for fewer than two values, so it comes back null until a bucket has at least two prior changes behind it — the fourth bucket of a series. `expected` is null before that too, since the first bucket has no previous value to project from. Nothing gates this: the nulls arise on their own. Those opening buckets are the **lead-in**: carried for context, never judged. + +**Nulls do all the branching.** `z_score` is arithmetic on `expected` and `spread`, so a null in either carries through, and `lower_bound` / `upper_bound` go null with it. A short series is therefore safe with no gate and no arithmetic: too little history returns nulls rather than an error. Read the result instead of predicting it; the caller reports what came back assessed. + +**`verdict` turns those nulls into a word, so the caller never reads a null.** Its three branches are tried in order, and the second one is what makes this work without a null test: + +- `abs(z_score) > k` → `'unusual'` +- `abs(z_score) >= 0` → `'normal'` — true for every real number, and *not* true when `z_score` is null +- otherwise → `'not assessed'` -Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the window for the grain, `k` = the sensitivity threshold (default 3), `` = the dataset name, `` = a readable label. +A null `z_score` fails both comparisons and falls to the `else`, so every row comes back carrying a literal word. Two things land in `'not assessed'`: a bucket with fewer than two prior changes behind it, and a perfectly flat stretch, where `spread` is `0` and `safe_divide` returns null. Both are correct — there is nothing to judge against in either case. -`M`, `T`, ``, `W`, `k` are placeholders; the `-1..-1` range (previous bucket) is literal; every `window_*` call takes `order: T | () | asc(), partition: []`. Widen the timeframe filter by `W` buckets, and add any dimensional filters. +**The baseline never forgets, and that is the cost.** With an expanding frame an unusual bucket raises `spread` for every bucket after it, permanently — there is no window for its influence to fall out of. One violent early spike can widen the band enough to hide milder anomalies for the rest of the series. The caller says so when the flagged set makes it relevant. + +### Query + +A complete query: monthly `total_revenue`, reporting on 2016 and 2017, `k` 3. -Template: ```aql -metric m_prev = window_avg(M, -1..-1, order: T | () | asc(), partition: []); -metric m_delta = M - m_prev; -metric m_n = window_count(M, -W..-1, order: T | () | asc(), partition: []); -metric m_drift = window_avg(m_delta, -W..-1, order: T | () | asc(), partition: []); -metric m_spread = window_stdev(m_delta, -W..-1, order: T | () | asc(), partition: []); -metric m_expected = m_prev + m_drift; -metric m_z = safe_divide(M - m_expected, m_spread); -metric m_lower = case(when: m_n >= W, then: m_expected - k * m_spread, else: null); -metric m_upper = case(when: m_n >= W, then: m_expected + k * m_spread, else: null); -metric m_anom = case(when: and(m_n >= W, abs(m_z) > k), then: 1, else: 0); +metric m_prev = window_avg(total_revenue, -1..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_delta = total_revenue - m_prev; +metric m_avg_change = window_avg(m_delta, ..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_spread = window_stdev(m_delta, ..-1, order: orders.created_at | month() | asc(), partition: []); +metric m_expected = m_prev + m_avg_change; +metric m_z = safe_divide(total_revenue - m_expected, m_spread); +metric m_lower = m_expected - 3 * m_spread; +metric m_upper = m_expected + 3 * m_spread; +metric m_anom = case(when: abs(m_z) > 3, then: 1, else: 0); +metric m_verdict = case( + when: abs(m_z) > 3, then: 'unusual', + when: abs(m_z) >= 0, then: 'normal', + else: 'not assessed' +); explore { dimensions { - bucket: T | () + bucket: orders.created_at | month() } measures { - actual: M, + actual: total_revenue, expected: m_expected, lower_bound: m_lower, upper_bound: m_upper, z_score: m_z, - is_anomaly: m_anom + anomaly_flag: m_anom, + verdict: m_verdict } filters { - // reporting window WIDENED by W buckets for warm-up; the @( … ) parentheses are REQUIRED. - T matches @(last s) - // + any dimensional filters carried from the chart, e.g. bq_fct_order_items.merchant_country_code == "US" + orders.created_at < @2018-01-01 } sorts { bucket asc nulls last @@ -68,8 +77,29 @@ explore { } ``` +Substitute `total_revenue` with `M`, `orders.created_at` with `T`, `month()` with the grain, and `3` with `k`. Leave both frames exactly as written: `..-1` is every bucket before this one, and `-1..-1` is the previous bucket. + +**The time filter has no lower bound.** Above, the report covers 2016 and 2017, but the filter carries no start date at all: the baseline is every prior bucket, so the fetch has to reach as far back as the data goes. Write it as an upper bound only — `orders.created_at < @2018-01-01` — and let the caller report on the range it was asked about. + +**Cropping the fetch changes the numbers, it does not just shorten them.** With an expanding baseline, every `expected`, `spread` and `z_score` depends on how far back the query reaches, so the same bucket scores differently under a narrower filter. Never crop to the range a chart happens to display, and never re-crop between the Step 2 query and the Step 3 chart — which is why the chart is drawn from this explore verbatim. + +**Dimensional filters go in the same block**, one per line: + +```aql + filters { + orders.created_at < @2018-01-01 + orders.region == "West" + } +``` + **Do not regress these:** -- Count prior buckets with `window_count(M, -W..-1, …)` — **never** `window_sum(1, …)` (SQL-generation error). -- The `n_prior ≥ W` gate is mandatory (partial windows return values, not null, and would false-flag the first `W` buckets). -- `case(when: …, else: null)` is valid and is how the band is hidden in the lead-in. -- The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly — one explore, no two-stage query. +- The baseline frame is `..-1`, **not** a fixed count like `-12..-1`. This method has no baseline length; do not reintroduce one, and do not add an `n_prior` gate to compensate. +- `window_stdev` is the **sample** standard deviation, and that is what makes the lead-in self-exclude: it is null below two values. `window_stdevp` (population) returns `0` there instead, and a zero spread scores every opening bucket as an extreme anomaly. +- Nulls carry the lead-in through `m_expected` and `m_spread` into `m_z`, `m_lower`, `m_upper` and `m_anom`. Nothing downstream needs a guard of its own — do not add one. +- **`verdict` and `anomaly_flag` are not redundant.** `verdict` is the caller's only input and carries three states; `anomaly_flag` exists solely so the chart has a numeric column series. Never drop one for the other, and never have the caller read `anomaly_flag`. +- The nested window (`window_stdev(m_delta, …)` where `m_delta` contains a window) compiles directly: one explore, no two-stage query. +- `anomaly_flag` must always return `0` or `1`, **never `null`**. The `else: 0` is mandatory: it is plotted as a column, and nulls cause rendering gaps instead of clean zeros. This is also why it cannot carry the third state and `verdict` exists. +- `verdict`'s second branch must stay `abs(m_z) >= 0`, not a null test. It is what routes a null `z_score` to `'not assessed'` without any null-handling function. +- `verdict` must never return null. Every row carries one of the three words. +- **The fetch must reach the start of the series.** It is the only thing standing in for a baseline length now, and a bounded lower bound silently rescales every score in the result. +- Dates are **literals, not calls**: `@2016-01-01`, `@2016-01-01 - 2017-12-31`, `@(last 12 months)`. There is no `@date(...)` function, and writing one is a syntax error at the `(`. diff --git a/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md new file mode 100644 index 0000000..8151729 --- /dev/null +++ b/native-skills/system/holistics-common/detect-anomaly-viz/SKILL.md @@ -0,0 +1,32 @@ +--- +name: detect-anomaly-viz +description: |- + Guidelines for charting detect-anomaly results. ONLY use this skill when explicitly asked to do so. +user-invocable: false +--- + +One chart, drawn in Step 3: the metric series with its expected band and the flagged buckets. It spans whatever the Step 2 explore returned — the series from its start — so the band has history to form against. + +Never hand-write the viz body. `generate_viz` writes it, `execute_viz` validates it on execution. State the decoration explicitly in the `query`, because `generate_viz` defaults its palette and `pattern: 'inherited'` unless told otherwise. + +Invoked once. The caller supplies the parameters. + +### Anomaly chart (Step 3) + +Parameters: `dataset`, `aql` (the Step 2 explore, verbatim), `grain`, `metric label`. + +The series, plus the band and the flags. The Step 2 explore already returns every field this chart needs, so it is passed through untouched and no filters or timeframe are supplied again. + +1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : anomaly_flag as red #FCB8B8 columns on the FIRST y-axis, 0..1 scale; actual as a solid #255DD4 line and lower_bound and upper_bound as grey #9CA3AF dashed lines, all three on the SECOND y-axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")` +2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust decoration only. +3. `execute_viz(dataset_uname: , viz: , title: …)` + +On an error, feed the error text back into `generate_viz`'s `query` (it self-corrects from prior errors) and retry once. If it still fails, fall back to the Step 2 table plus prose. **That is the only permitted reason to ship without this chart.** + +### Do not regress +- Pass only `dataset_uname`, `viz`, `title` to `execute_viz`. No `aql` property. +- 6-digit hex only. An 8-digit alpha hex can be rejected. +- `anomaly_flag` goes on the **first** y-axis (0..1) and the metric, `lower_bound` and `upper_bound` on the **second**. That order is what keeps the columns behind the lines. +- Chart `anomaly_flag`, never `verdict`. `verdict` is a text field for the prose; it has no numeric series to plot. +- The chart covers everything the Step 2 explore returned. Cropping to the reporting window hides the lead-in the band needs to form, and shows the series starting later than the results do. Do not re-filter here: the explore's own range is what its `expected`, `spread` and `z_score` were computed against. +- For a metric that cannot go negative, clamp the **displayed** lower bound at 0. The band sits around the trend rather than the level, so it can dip below zero. diff --git a/native-skills/system/holistics-common/detect-anomaly/SKILL.md b/native-skills/system/holistics-common/detect-anomaly/SKILL.md index 92afc32..35a76f6 100644 --- a/native-skills/system/holistics-common/detect-anomaly/SKILL.md +++ b/native-skills/system/holistics-common/detect-anomaly/SKILL.md @@ -1,136 +1,192 @@ --- name: detect-anomaly description: |- - Detect statistical anomalies across a metric's history and visualize them — unusual spikes, drops, and deviations flagged against an expected range built from the metric's own recent history. + Detect statistical anomalies across a metric's history and visualize them: unusual spikes, drops, and deviations flagged against an expected range built from the metric's own history. - Use when the user wants to find or check for unusual values in a metric — words like anomaly, unusual, spike, drop, dip, outlier, "out of the ordinary", "anything weird". A specific date is optional; the skill always scans the whole series. + Use when the user wants to find or check for unusual values in a metric. Trigger words include anomaly, unusual, spike, drop, dip, outlier, "out of the ordinary", "anything weird". A specific date is optional; the skill always scans the whole series. - Typical phrasings: detect anomalies in revenue, any outliers in daily signups, has anything looked unusual in MRR lately, were there spikes or drops in active users this quarter, is revenue on May 20 unusual (a focus date — still scans the full series), or /detect_anomaly with no args (the skill asks for the metric). + Typical phrasings: detect anomalies in revenue, any outliers in daily signups, has anything looked unusual in MRR lately, were there spikes or drops in active users this quarter, is revenue on May 20 unusual (a focus date; still scans the full series), or /detect_anomaly with no args (the skill asks for the metric). Do NOT trigger for: dimensional attribution (which segment or dimension drove an anomaly), explaining the business cause, or ongoing/scheduled monitoring & alerting. This skill flags which points are statistically unusual and shows them; it does not explain why. user-invocable: false --- -Flag which points in a metric's time series moved unusually against its own recent trend, and show them: build a trend-following expected band from recent history, flag the buckets that break out of it, and visualize the result. This skill flags *which* points are unusual, never *why* — deflect causal or dimensional-attribution asks (Conventions → *Edge cases*). +Build an expected band from a metric's history, flag the buckets that break out of it, and chart the result. This skill identifies which points are unusual, not why they moved (Conventions → *Out of scope*). + +Two sub-skills carry the mechanics: **`detect-anomaly-aql`** owns the method and the query, **`detect-anomaly-viz`** owns the chart. ## What a good input looks like -An anomaly task scans one metric's time series against a trend-following band built from its own recent history. The primary path is **chart-anchored**: the user is viewing a chart and asks whether its movement is normal, so the fields below are read from that chart's Viz AML. Only `metric` is truly required — derive or default the rest, then echo everything back (with the resolved timeframe as an absolute range) so the user can correct a mis-parse. -- **`metric` (`M`)** — *required*. From the viewed chart's y-axis measure; else the user's words. If missing, ask — don't guess. If the chart carries **more than one** y-axis measure, ask which one to analyze (don't default to the first). -- **`dataset`** — *required*. From the chart's `dataset:`; else a ranked measure search. If missing, ask. -- **`time axis` (`T`) + `granularity`** — inferred from the chart's x-axis field and its `datetrunc` transformation. If the x-axis isn't a time field, ask for the metric + time grain. -- **`filters`** — optional. The chart's dimensional filters (not its time range). -- **`focus_date`** — optional. From the user's words; changes only the closing prose. -- **`timeframe`** — optional. The chart's timeframe; else default by grain (`day`→90d, `week`→26w, `month`→24m, `quarter`→12q). +Anomaly detection compares a metric against its own history to determine whether a value is far from what the preceding periods would suggest. The analysis needs one measure, the date field and grain it is measured on, and enough prior history to establish what normal movement looks like. + +A task is fully specified when you can answer: + +1. **Which measure, and which dataset.** The method compares a series against itself, so it runs on exactly one measure; two measures are two analyses. When a chart offers several, ask which one, because choosing for the user wastes the whole run. +2. **Which date field, and at what grain.** Day, week, month, or quarter. The grain sets what can be found: a spike lasting two days does not show up in monthly buckets, and a metric recorded monthly cannot be read daily. +3. **Which slice of the data.** The segment in view, if any. The result describes the series you selected, not the business as a whole: one region collapsing can leave the company total flat, and a flat total can hide two segments moving in opposite directions. One slice per analysis; breaking the metric down by a dimension is a different question. +4. **Which period the findings cover.** Flagged buckets are reported only inside this period. Earlier buckets are shown as context, because the method needs prior history before it can judge a bucket. +5. **Whether the user has a specific date in question.** A named date does not narrow the analysis; the whole series is always scanned. It changes what the answer leads with — and, when that bucket falls in the lead-in, whether there is a verdict to lead with at all. -Fill these with **one prioritized-fill pass**, not a chart-vs-typed branch (Workflow → *Step 1*). Confirm only fields you had to **guess**; silently accept fields **read** from the chart or stated by the user. +The user rarely supplies all five, and often has not asked for anomaly detection by name. + +- **Read before asking.** A chart in view already carries the measure, date field, grain, slice and window; take them from there (*Resolving the spec*). +- **Ask only where a wrong guess wastes the run**: the measure, and with no chart its dataset. Derive the rest and state it back in one line with real dates, so a misread is cheap to correct. ## What a bare minimum output looks like -The deliverable is a trend-anomaly chart plus a short prose summary — nothing else by default. -- **The chart** (Step 3): `actual` as a solid blue line, the `lower_bound`/`upper_bound` trend band dashed grey, and `is_anomaly` as red columns on a secondary 0..1 axis, with tooltips. -- **The prose summary** (Step 3): for a business reader — defines an anomaly (a break from the recent trend), then lists each flagged bucket (date, value, above/below the trend, how far out in plain terms). -- **The anomaly results** (the Step 2 AQL) stay internal — the source of truth for the summary, not shown as a table unless the user asks for the raw numbers. -- If `execute_viz` ultimately fails, fall back to the Step 2 table + prose (Conventions → *On an error*). +One chart and a prose summary. +- **The anomaly chart** (Step 3), per `detect-anomaly-viz`. It is required output; the run ships with it or says why it could not be drawn. +- **The prose summary** (Step 4), for a business reader: what an anomaly means here, then each flagged bucket with its date, value, direction, and how far outside the band it fell. +- **The anomaly results** (Step 2) stay internal. They are the summary's source of truth, not a table to show, unless the user asks for the raw numbers. ## Workflow -Three steps, in order. Hand-write the AQL yourself (Step 2) — the method must be exact — but author the chart with `generate_viz` (Step 3): hand-written viz grammar is the top source of invalid output. `execute_aql` runs the AQL (and validates it on execution); `execute_viz` runs the chart. - -Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `W` = the window for the grain, `k` = the sensitivity threshold (default 3), `` = the dataset name, `` = a readable label. - -### Step 1 — Resolve the input -Produce one detection spec `{ dataset, M, T, granularity, filters, timeframe }`. Fill each field from the strongest available source, in precedence order — there is **no chart-path-vs-typed-path branch**: -1. **The active chart's Viz AML** (the runtime hands the AI the AML of the chart the user is viewing) — fills `dataset`, `M`, `T`, `granularity`, `filters`, and the chart's timeframe. -2. **The user's words** — fill or *override* any field they spoke to ("…but over the last 3 years", "…just the West region", or, with no chart, the metric name itself). -3. **Dataset metadata + defaults** — `granularity` from `T`; timeframe default per grain. -4. **Ask the user** — only for a field still empty (realistically `M`/`dataset` with no chart and no parseable metric). - -**Reading the chart's Viz AML** (source 1) — pull each field out of the attached AML: -- `dataset` ← the viz `dataset:`. -- `M` ← the y-axis measure. If the chart has **more than one** y-axis measure, don't pick for the user — **ask which one** to analyze (list them and wait), unless the user already named one in their words. Anomaly detection runs on a single series, and silently choosing the wrong one wastes the whole run. With exactly one y-axis measure, use it. If `M` is a viz-level `calculation` rather than a dataset measure, carry its `@aql` formula and inline it as `metric M = ;` in the Step 2 query. -- `T` + `granularity` ← the x-axis field and its `transformation` (`datetrunc month` → grain = month). If the x-axis is **not** a time field, this isn't a time series → fall back to typed/ask. -- `filters` ← the viz `filter` / `filter_groups` / `conditions`, **excluding** the time-range filter. -- **Legend/breakdown** → collapse to the total (drop the legend) unless the user explicitly asks per-segment (per-segment is out of scope — Conventions → *Edge cases*). -- **Timeframe + warm-up** — the reporting timeframe is what gets flagged/reported; the Step 2 query widens it by `W` buckets on the leading edge so the reporting window is fully banded. The chart shows those `W` lead-in buckets as trend context (unbanded); prose reports only within the reporting timeframe. - -Confirm only fields you **guessed** (a ranked search: *"I'll use `sales_orders` (measure `total_revenue`). Confirm, or name another."* — then wait); silently accept fields **read** from the chart or stated by the user. Validate that `M` exists as a measure (or is a carried viz calculation); if a typed metric matches none, show the top-3 closest measures and ask. Echo the resolved slice back, including the lead-in context: *"Analyzing GMV for Region = West over Jul 2023 – Aug 2024 (your view); the chart also shows earlier months as lead-in to establish the trend."* - -### Step 2 — Detect (write AQL → execute_aql) -1. Write the anomaly detection AQL with `generate_aql`, providing `M`, `T`, ``, `W`, `k`, folding in the dimensional filters, and widening the timeframe by `W` buckets for warm-up. Prompt structure: - ``` - /detect-anomaly-aql - - M: ... - T: ... - grain: ... - W: ... - k: ... - ``` -2. Call `execute_aql` (title per Conventions → *Titles*) — it validates the AQL on execution and errors loudly on a bad query. The result is the **anomaly results** — the summary's source of truth and the exact AQL you hand to `generate_viz` in Step 3. - -### Step 3 — Present (generate_viz → execute_viz → summarize) -Feed the Step 2 AQL to `generate_viz` per Output → *Anomaly chart*, adjust only decoration, then `execute_viz` (title per Conventions → *Titles*). Then write the prose per Output → *Summary*, reading only the buckets inside the reporting timeframe. On a failure, Conventions → *On an error*. -## Output +Four steps. Each is a move an analyst makes, and each closes with a `post_update` so the user follows the reasoning as it happens instead of receiving a verdict at the end. **Narrating is not pausing**: post and keep working. The only stop in the run is the measure question in Step 1. + +Placeholders: `M` = the metric measure (e.g. `gmv`), `T` = the bound time dimension (e.g. `bq_fct_order_items.created_date`), `` = the time grain, `k` = the sensitivity threshold, `` = the dataset name, `` = a readable label. + +### Step 1: Establish the series + +A series needs one measure on one time axis, at one grain, for one slice, over one period. Read it from the chart rather than asking for it back. + +Resolve the spec (*Resolving the spec*): ask only for what's still empty — at most the measure, and with no chart its dataset. Confirm only what you guessed, then wait; fields read from the chart or stated by the user are accepted silently. + +Post the metric, grain, slice and period as real dates, and that the whole series is scanned even if the user named a single date. The data checks — missing buckets, a final bucket still in progress, row count — are read off the Step 2 results. + +### Step 2: Run the detection + +The rule: predict each period from where the metric's movement so far was heading, then judge the gap between actual and prediction against how much this metric usually moves period to period. There is no baseline length to choose — every period is judged against everything before it. `detect-anomaly-aql` owns the method and the query. -### Anomaly chart (generate_viz — Step 3) -Don't hand-write the `CombinationChart` — pass the Step 2 explore (verbatim) to `generate_viz` and **state the decoration explicitly in the `query`** (generate_viz defaults its palette and `pattern: 'inherited'` unless told otherwise, so name the colours): +Hand over the resolved parameters: -1. `generate_viz(dataset_uname: , aql: , query: "Combination chart of by : actual as a solid #255DD4 line; lower_bound and upper_bound as grey #9CA3AF dashed lines forming the expected band; is_anomaly as red #FCB8B8 columns on a secondary 0..1 right axis; x-axis is the bucket; format as short-suffix currency like $500K; tooltips for expected, , and z_score.")`. -2. Keep the structure `generate_viz` produced (axes, series, calculations); adjust only decoration. Keep 6-digit hex (an 8-digit alpha hex can be rejected). -3. `execute_viz(dataset_uname: , viz: , title: …)` — pass only these three, no `aql` property. +``` +/detect-anomaly-aql -If `execute_viz` errors, feed the error text back into `generate_viz`'s `query` (it self-corrects from prior errors) and retry once; if it still fails, fall back to the Step 2 `execute_aql` table + prose. +M: ... +T: ... +grain: ... +k: ... +reporting: ... +filters: ... +``` -### Summary (prose — Step 3) -For a business reader: plain and professional, no raw notation (no bare "σ", "z = 3.4", "3σ"). Read the anomaly results; don't render them as a table (the user sees the chart) unless asked. +Run `execute_aql` on what comes back (title per Conventions → *Titles*). Those are the **anomaly results**, the summary's source of truth; the AQL itself goes to Step 3 unchanged. -Open with one sentence defining an anomaly under this method, then the findings: +Read the returned rows for the four conditions that change what you can promise: +- **Missing buckets** between the first and last. Gaps break the period-to-period comparison the method rests on. +- **A final bucket still in progress.** A part-period value reads as a collapse. Exclude it, or say it is incomplete. +- **Row count.** A short series is not an error: it comes back unassessed rather than flagged. Say so instead of reporting that nothing was unusual. +- **The focus bucket's `verdict`, when the user pointed at one.** `not assessed` means the run cannot answer the question it was asked, however much of the rest of the series came back judged. Read it here rather than while writing the summary: it decides what the answer leads with. -> *An anomaly is any whose value departs sharply from where its recent trend was heading — a change much larger or smaller than the metric's normal -to- movement over the prior s.* +Then post the rule in the reader's terms, naming `k`, and anything the four checks turned up. -Classify each bucket in the reporting window: -- **Unusual** — `is_anomaly = 1`: above the trend (`actual > expected`) or below it. -- **Normal** — `is_anomaly = 0`, `z_score` non-null: moved in line with the trend. -- **Not enough history** — `z_score`/band null (a lead-in bucket, `n_prior < W`): shown as context without a band; not assessed. +### Step 3: Draw the anomaly chart -Translate the statistics — describe the size of the surprise in plain terms (*"about times its normal monthly swing"*), z only as a parenthetical. +Draw it even when nothing is flagged: without the band, "nothing was unusual" is an assertion the reader has no way to check. This is the run's only chart, so it carries the series as well as the band. -No focus date (primary path): +Draw the chart first: -> **Found unusual (s) in ** over ****: -> - **** — : **** the expected trend (about × its normal movement). -> - … one bullet per flagged bucket, chronological +``` +/detect-anomaly-viz -If `N = 0`: *"No unusual values in over — every moved in line with its recent trend."* +dataset: ... +aql: +grain: ... +metric label: ... +``` -Focus date given: lead with the verdict on that bucket, then the same list. -- Unusual: *"** on = ** is unusual — it broke **** its recent trend (about × the normal movement)."* -- Normal: *"** on = ** moved in line with its recent trend — not unusual."* -- Lead-in / no band: *"Not enough prior history to assess — it's within the first s, shown as context without a band."* +Then post the anomaly definition, so it lands with the band already on screen: +> **What counts as unusual here.** An anomaly is any [grain] whose value departs sharply from where the trend was heading: a change much larger or smaller than [metric]'s normal [grain]-to-[grain] movement across its history up to that point. -Closing pointer (N ≥ 1): why an anomaly happened or which dimension drove it is out of scope — invite the user to investigate via a dimensional breakdown and business context (marketing calendar, CRM, recent news). +Then post how to read it: the band is the expected range given the trend so far, it widens when movement has been erratic, and the red columns mark what broke out. -Post-run override (offer last): *"Want me to re-run with a stricter or looser threshold? Looser flags more s (milder deviations); stricter flags only the most extreme. A seasonality-aware version (for weekly/annual patterns) is also available."* The threshold is `k` (looser = 2, stricter = 4; default 3) — the only sensitivity control, never asked upfront. On accept, re-run Steps 2–3 with the new `k`. +### Step 4: Write the summary + +Read `verdict`. It is one field per bucket carrying one of three words, and it is the only thing that decides how a bucket is treated: + +| `verdict` | What it is | +|---|---| +| `unusual` | A finding. These are what the summary lists. | +| `normal` | Assessed, nothing to report. | +| `not assessed` | Context, never a finding — too little history behind it, or no variation to judge against. | + +Do not infer any of this from `z_score` or `anomaly_flag`. `anomaly_flag` is `0` for both `normal` and `not assessed`, which is exactly the confusion `verdict` exists to remove; it is the chart's column series, not an input to the prose. `z_score` is only for phrasing how far a finding fell outside the band. + +Write the summary (Output → *Summary*) covering only what was assessed, not what was asked for: if any bucket in the period came back `not assessed`, name the span actually judged. + +Before writing, check: +- `k` is the value from `detect-anomaly-aql`, never adjusted for this series. +- The anomaly chart was drawn (even when nothing was flagged), or the fallback was taken and said so. +- No causal or dimensional claim anywhere in the prose. Numbers per Conventions → *Numeric formatting*. + +Two patterns in the flagged set change what the summary can claim: +- **Same calendar position** (same month each year, same weekday). The method has no seasonal term, so a recurring peak flags as an anomaly. Say so, and offer the seasonality-aware re-run. +- **A flag early in the series.** The baseline is every prior bucket, so an unusual value widens the band for everything after it and never falls back out. Anything milder later was judged against a looser threshold and may have been missed. Say so whenever the first flag lands well before the others. + +## Output + +### Summary (prose, Step 4) + +A complete answer on the primary path (no focus date, two buckets flagged). The chart is not written, it is what `execute_viz` returned in Step 3; the prose below it is Step 4's: + +> ⟦ Step 3 anomaly definition post_update renders here ⟧ +> +> ⟦ Step 3 anomaly chart renders here ⟧ +> +> **Found 2 unusual months in GMV, Jul 2023 to Aug 2024:** +> +> - **Nov 2023: $4.2M.** Above the expected trend by about 3.6× its normal monthly movement. +> - **Mar 2024: $1.9M.** Below the expected trend by about 4.1× its normal monthly movement. +> +> **What this does not tell you.** Why they moved, and which segment drove them, is outside this analysis. Break GMV down by product, channel or customer segment, and check the marketing calendar and CRM for those two windows. +> +> Want me to re-run with a stricter or looser threshold? Looser flags more months (milder deviations); stricter flags only the most extreme. A seasonality-aware version is also available if GMV has a repeating annual shape. + +Then the conventions: + +- **Register.** Business reader, plain and professional. No raw notation: no bare "σ", "z = 3.4", "3σ". `z` appears only as a spoken multiple, as above. +- **Markup carries the scan.** A bold label opens each part; each finding leads with its date and value in bold, then a plain sentence for the assessment. No headings, no tables. +- **One line per flagged bucket, chronological**, each carrying the date, the value, the direction, and how far outside the band it fell in plain terms. +- **The user has the chart.** Do not repeat the results as a table unless asked. +- **Report the assessed span, and no more.** The assessed span is the buckets whose `verdict` is not `not assessed`. Name it whenever it differs from the period that was asked for; findings and any "nothing unusual" claim cover that span only. The chart ships first in every case. Four shapes fall out of the one rule: + - *Fully assessed, nothing flagged* — *"No unusual values in GMV over Jul 2023 to Aug 2024; every month moved in line with the trend up to it."* + - *Partly assessed* — *"Assessed GMV from Jul 2024 onward; the months before it had too little history behind them."* + - *Nothing assessed* — every bucket came back `not assessed`. *"Not enough history to assess GMV. The method needs a few months of movement behind a month before it can judge it, and this series starts in Nov 2025."* Never that nothing was unusual. When the cause is a flat series rather than a short one, say that instead: *"GMV held at exactly $40K every month, so there is no variation to judge a month against."* + - *A named date outside the assessed span* — lead with that, then the list for what was assessed: *"I can't judge Feb 2026 — it is one of the opening months of the series, and the method needs a few months of movement behind a month before it can judge it. GMV starts in Jan 2026."* +- **A named date inside the assessed span**: lead with the verdict on that bucket, unusual or normal, then the same list. The full series is scanned either way. +- **The closing pointer** appears only when something was flagged. +- **The re-run offer goes last.** `k` is the only sensitivity control and is never asked upfront (values in `detect-anomaly-aql`). On accept, re-run Steps 2 to 4 with the new `k`. + +## Resolving the spec + +Produce one spec `{ dataset, M, T, granularity, filters, timeframe }`. **One fill pass, not a chart path and a typed path**: take each field from the chart when there is one, let the user's words override anything they spoke to ("…but over the last 3 years", "…just the West region"), then fall back to defaults, and ask only for what is still empty. + +| Field | From the chart's Viz AML | Otherwise | +|---|---|---| +| `dataset` | the viz `dataset:` | a ranked measure search; ask if nothing matches | +| `M` | the y-axis measure | the user's words; if a typed metric matches no measure, show the top-3 closest and ask | +| `T` + `granularity` | the x-axis field and its `transformation` (`datetrunc month` → month) | `granularity` derived from `T` | +| `filters` | `filter` / `filter_groups` / `conditions`, **excluding** the time-range filter | none | +| `timeframe` | the chart's timeframe | per grain: `day`→90d, `week`→26w, `month`→24m, `quarter`→12q | + +- **More than one y-axis measure**: ask which to analyze (list them and wait), unless the user already named one. +- **`M` is a viz-level `calculation`**, not a dataset measure: carry its `@aql` formula and inline it as `metric M = ;` in the Step 2 query. +- **The x-axis is not a time field**: this is not a time series. Ask for the metric and grain instead; do not proceed. +- **A legend/breakdown**: drop it and analyze the total, unless the user explicitly asks per-segment (Conventions → *Out of scope*). +- **No chart, and the metric cannot be parsed**, or `list_datasets()` is unsupported (dev mode): ask the user to name the dataset and metric explicitly. Do not proceed. +- **Confirm only what you guessed** (*"I'll use `sales_orders` (measure `total_revenue`). Confirm, or name another."*), then wait. Fields read from the chart or stated by the user are accepted silently. +- The timeframe resolved here is the **reporting** window: what the findings cover. `detect-anomaly-aql` queries the series from its start regardless, because the baseline is every prior bucket; the buckets before the reporting window are context on the chart and are never reported as findings. ## Conventions -- **On an error, retry once, then fall back.** If a retry still fails: Step 2 → surface the error (don't pretend detection completed); Step 3 → fall back to the Step 2 table + prose. Never loop more than one retry per step. -- **Titles.** `execute_aql` / `execute_viz` require a `title` (under ~60 chars; avoid "query 1"/"untitled"): the Step 2 anomaly results `execute_aql` → ` anomaly results ()`; the Step 3 anomaly chart `execute_viz` → `Anomaly detection: ()`. +- **On an error, retry once, then fall back.** Step 2: surface the error, never imply detection completed. Step 3: fall back to the Step 2 table plus prose. Never loop more than one retry per step. +- **Titles.** `execute_aql` / `execute_viz` require a `title` (under ~60 chars; avoid "query 1"/"untitled"): Step 2 anomaly results → ` anomaly results ()`; Step 3 anomaly chart → `Anomaly detection: ()`. ### Numeric formatting - Currency-like measures (label/name contains `revenue`, `cost`, `price`, `arr`, `mrr`, `gmv`, `amount`): `$` prefix, short suffix (`$500K`, `$1.2M`); use the dataset's stated currency symbol if metadata provides it. - Counts and rates: thousands separators (`1,234`); rates to two significant decimals (`3.42%`). - z-scores / multiples: one decimal place (`3.4`). -### Edge cases - -| Situation | Behavior | -|---|---| -| Viz AML x-axis is not a time field | Not a time series — fall back to typed/ask for the metric + time grain. Don't proceed. | -| Viz has more than one y-axis measure | Ask the user which single measure to analyze (list them); don't default to the first. Skip the ask only if the user already named one. | -| Viz has a legend/breakdown | Collapse to the total (drop the legend) unless the user asks per-segment. | -| Series too short even with the lead-in (starts inside the window) | Earliest buckets show without a band (`n_prior < W`); if < 8 usable points, stop with the insufficient-history message. | -| Flat series (spread = 0) | `safe_divide` → `z_score` null; not flagged (never divide by zero). | -| Non-negative metric, band dips below 0 | Rare (the band is around the trend, not the level), but clamp the *displayed* lower bound at 0 for non-negative metrics. | -| `list_datasets()` unsupported (dev mode) or no chart + unparseable metric | Ask the user to name the dataset/metric explicitly. Do not proceed. | -| User asks "why did this happen?" or "what dimension drove it?" | Reply: "I can flag which points broke from the trend, but identifying which dimension drove an anomaly — or its business cause — is outside this skill. Break the metric down by relevant dimensions in a dashboard, or check your marketing calendar / CRM / news for that window." | -| User asks to "show the raw numbers" | Show the `execute_aql` anomaly results directly — `bucket`, `actual`, `expected`, `lower_bound`, `upper_bound`, `z_score`, `is_anomaly`. Never group/pivot rows by `is_anomaly`. | +### Out of scope +This skill says which points broke from the trend. It does not say why, and it does not attribute a break to a segment. +- **"Why did this happen?" / "What dimension drove it?"** Reply: "I can flag which points broke from the trend, but identifying which dimension drove an anomaly, or its business cause, is outside this skill. Break the metric down by relevant dimensions in a dashboard, or check your marketing calendar / CRM / news for that window." +- **A per-segment run.** Out of scope: one slice per analysis. Offer a separate run on the segment the user names. +- **"Show the raw numbers."** Show the `execute_aql` anomaly results directly: `bucket`, `actual`, `expected`, `lower_bound`, `upper_bound`, `z_score`, `verdict`. Drop `anomaly_flag` — it duplicates `verdict` for the chart's benefit. Never group or pivot rows by `verdict`.