improve change analysis: confirmed data echo, mandatory anomaly chart, remove stop-and-wait - #32
Open
vyhuynh-product wants to merge 17 commits into
Open
improve change analysis: confirmed data echo, mandatory anomaly chart, remove stop-and-wait#32vyhuynh-product wants to merge 17 commits into
vyhuynh-product wants to merge 17 commits into
Conversation
…isting dimension proposal skill now proceed to breakdown immediately
…e without enough history
thinhnguyentruong
force-pushed
the
system/vy-improve-change-analysis4
branch
from
August 17, 2026 07:41
bd8688e to
620d28a
Compare
2 tasks
thinhnguyentruong
force-pushed
the
system/vy-improve-change-analysis4
branch
from
August 17, 2026 07:51
620d28a to
c09d639
Compare
…nalysis The skill now reads as an analyst working through the problem, with each step explaining what it is doing and why before it does it, and closing with a post_update so the user follows the reasoning rather than waiting for a verdict. Four steps replace three: establish the series (now drawing it, not only describing it), turn "unusual" into a rule and apply it, show the expectation as a band, then verify and write up. The verify pass is new; the checks existed but were scattered across three sections, so nothing forced a final look before presenting. Adds detect-anomaly-viz, which owns both charts, so the parent carries the reasoning and the sub-skills carry the mechanics. Content was reallocated on one rule: aql owns the statistics, viz owns the charts, the parent owns the analysis and the words. That removed fourteen facts that were stated in two or three places at once, including the edge-case table, whose rows split between the spec resolution and the two sub-skills. Other changes: - Steps 1 and 3 invoke detect-anomaly-viz with a slash block, matching Step 2. Naming a sub-skill in prose reads as a cross-reference, and only the explicitly invoked one was firing. - Both sub-skills document the parameters they expect; the aql invocation passes reporting and filters explicitly instead of in prose. - aql states that z_score must be null below the warm-up gate, which the caller relies on to tell unassessed buckets from normal ones. - The summary spec is one worked example plus conventions, replacing eight fill-in-the-blank templates. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…behind The columns were on a secondary 0..1 axis, which drew the full-height marks in front of the metric line and the band. Swapping the axes keeps the same 0..1 scale and full-height marks, behind the data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t-anomaly Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…runs A live run produced invalid AQL and a wrong window. Both trace to the sub-skill asking the model to compute things, in one case before it had the data to compute them from. Date literals. The template only ever showed the relative form, so a report over historical dates had no example to copy and the model invented `@date(2016-01-01)`, which is a syntax error: `@` introduces a literal, not a call. The query section is now a complete worked query with real values rather than a placeholder skeleton, showing the fixed range form, and the do-not-regress list names the mistake. W. It was a per-grain table plus a shrink rule keyed on total available history, which the model has to guess at because it has not queried yet. That is how a monthly run came back with W=11 and no widened filter. W is now 12 at every grain: twelve is where the spread estimate stops improving much, and a longer baseline costs assessed buckets without buying precision. It is no longer passed as a parameter, since both sub-skills know it, and `12` in the worked query is now correct as written rather than something to substitute. Short series are read, not predicted. The full-window check already returns nulls when history is thin, so the shrink-or-stop branch is gone; if nothing comes back assessed, the answer says so. Also: method section restated as what it does before how, with the notation kept; `drift` renamed `avg_change`; `warm-up` and `lead-in` collapsed to `lead-in`; example identifiers made generic; and three sentences cut that explained behaviour the query already enforces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run with no anomalies was skipping the chart, and the skill was the reason. Step 3 motivated the chart entirely by the flagged points, so at zero flags its stated purpose disappeared and skipping read as the correct call. The summary's nothing-flagged convention compounded it by describing a one-sentence answer with no chart in it. Step 3 now justifies the chart by the verdict rather than by the flags: without the band, "nothing was unusual" is an assertion the reader cannot check. The nothing-flagged convention says to draw it anyway, and the Step 4 check names the case, since "the chart rendered" was satisfiable by having decided not to render one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n the summary Zero flags had one wording, and it claimed the series was fine. A series too short to judge produces zero flags too, so a run that assessed nothing would report that every period moved in line with its recent trend. The summary now separates three outcomes: judged and all normal, judged in part, and not judged at all. The partly-assessed case matters as soon as the data does not reach back far enough to fill the lead-in, since the findings and the clean-bill claim then cover less than the period asked about. The rule for this existed, but in detect-anomaly-aql, which is loaded at Step 2 to write the query. The summary is written at Step 4 from the parent's conventions, so it was never in context when it applied. Moved to the parent, with a Step 4 check, and the sub-skill now just says the caller reports what came back assessed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…maly The series chart was a non-blocking preview before the detection query ran; removing it lets the run go straight from resolving the spec to running detection. The three data-quality checks (missing buckets, in-progress final bucket, row count) move to Step 2 where rows first come back from execute_aql. - detect-anomaly: Step 1 resolves spec + posts only; no chart drawn. Step 2 runs detection and reads the checks. Step 3 is the sole chart (series + band). Trim verbose prose across steps and headers. - detect-anomaly-viz: single chart family; drop the series-chart section and the chart: selector. Co-Authored-By: Claude <noreply@anthropic.com>
…nomaly
The baseline was a fixed 12 buckets, which meant a bucket needed twelve
periods behind it before it could be judged. Points near the start of a
series came back unassessed, and the query had to widen its own time
filter to fetch a lead-in.
Replace it with an expanding frame (`..-1`): every bucket is judged
against everything before it, so there is no window length to choose and
nothing to widen. The opening buckets exclude themselves, because
`window_stdev` is a sample stddev and is null below two values — the
`window_count` gate and its `case` are gone. The time filter now carries
an upper bound only; with an expanding baseline, cropping the fetch
rescales every score rather than returning fewer rows, so both skills say
not to.
The cost is that the baseline no longer forgets: an unusual bucket widens
the band for everything after it permanently. The summary's contamination
note changed from "two flags within W buckets" to "a flag early in the
series" to match.
Also add a `verdict` column ('unusual' / 'normal' / 'not assessed') and
rename `is_anomaly` to `anomaly_flag`. The summary previously had to read
`z_score` for null and `is_anomaly` for 0/1 to tell an unassessed bucket
from a normal one, since the flag is 0 for both. It now reads one field
carrying one word. `anomaly_flag` stays numeric and never null because the
chart plots it as a column series.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
base_period/comparison_period), fix typos (/detect_anomoly→/detect_anomaly,Depdendency→Dependency)execute_vizretry failureTest plan
analyze-changes— verify it echoes actual metric values with delta before running anomaly checkanalyze-contribution— verify it no longer pauses for user confirmation after listing dimensionsdetect-anomaly— verify chart is always attempted and not silently skipped🤖 Generated with Claude Code