test(data): spike check for collab activity vs per-user trailing baseline (#1321)

[SharedQA]

Follow-up to the silver non-negative business-rule test. It guards the "too high" case the non-negative check can't see: a count that looks fine on its own but is wildly out of line with the same user's own recent history — the kind of jump a feeder double-count or a unit change produces.

Not a fixed limit. The baseline is computed per (tenant, user, product, metric) from each series' own trailing 28 observed days (excluding the current day). A user-day is flagged only when all of these hold:

• at least 14 prior observations exist (cold-start guard)
• the baseline has non-zero spread (avoids divide-by-noise on flat series)
• value > mean + 5*stddev (z-score outlier)
• value > 10*mean (massive relative jump)

The z-score and the relative-jump condition together keep it to genuinely large spikes, so it stays quiet on normal variation.

Safe by construction. It joins the data_quality catalog as severity=warn + store_failures — advisory and monitored, never blocks a build. Reads FINAL.

Why draft. The thresholds (28-day window, 14-day minimum, 5σ, 10×) are conservative starting points, not yet calibrated against real tenant data — there's no warehouse available locally to run dbt. The intent is to tune them from the first scheduled-run findings before marking ready. Review welcome on the method and the threshold choices.

Summary by CodeRabbit

• New Features
  • Added automated anomaly detection for collaboration document activity spikes. The system continuously monitors activity patterns and generates real-time alerts when unusual increases are detected in collaboration metrics. This provides visibility into potential anomalies across your workspace, enabling proactive oversight of collaboration activities, usage patterns, and team productivity metrics.

[SharedQA]

Good catches — applied both, with one adjustment on the performance fix.

1. Full-table-scan / re-alerting. Filtering only the final SELECT does stop re-alerting on old spikes, but the window function still has to sort and scan the whole history first, so the compute cost you flagged doesn't actually go away. So I bounded both ends:

• input capped to the last 120 days (a new recent CTE) — a 28-observed-day baseline for any active user fits inside that, so the window sort never grows with total history;
• output capped to date >= today() - 3 — the scheduled daily run only reports newly-appearing spikes.

A user with fewer than 14 active days inside the 120-day window simply isn't evaluated, which the prior_n >= 14 guard already enforces.

2. Observed vs calendar days. You're right that ROWS means observed days, and the return-from-leave case is the real risk. Rather than leave it as a documented caveat, I added a staleness guard: date - prev_date <= 35, where prev_date is the most recent prior observed day. So a baseline that is itself months old won't be used, which removes that false-positive class. I kept the ROWS window (vs ClickHouse's clunky RANGE BETWEEN INTERVAL) — the guard gives the calendar-recency property without the syntax pain.

Still severity=warn and uncalibrated; the 120/3/35-day and 5σ/10× values are starting points to tune from the first scheduled-run findings.

[coderabbitai]

Warning

Review limit reached

@SharedQA, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 56 minutes and 52 seconds. Learn how PR review limits work.

To continue reviewing without waiting, enable usage-based billing in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses rolling per-developer review limits. Reviews become available again as older review attempts age out of the rolling limit window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2559c6ed-323c-4ef3-b9c2-959600ab8d14

📥 Commits

Reviewing files that changed from the base of the PR and between 09e8474 and e1d4575.

📒 Files selected for processing (1)

• src/ingestion/dbt/tests/collaboration/assert_collab_document_counts_no_spike.sql

📝 Walkthrough

Walkthrough

A new dbt SQL test assert_collab_document_counts_no_spike.sql is added. It queries class_collab_document_activity over a 120-day window, unpivots five activity counters into metric rows, computes a 28-day trailing baseline (mean, stddev), and emits warn-severity findings for values exceeding both a z-score and a 10× relative threshold within the last 3 days.

Changes

Collaboration Document Spike Detection Test

Layer / File(s)	Summary
Test config and methodology docs src/ingestion/dbt/tests/collaboration/assert_collab_document_counts_no_spike.sql	dbt config block sets tags, severity = 'warn', store_failures = true, and descriptive metadata; SQL comments document the trailing-window spike-detection approach and advisory behavior.
Data scoping and unpivot CTEs src/ingestion/dbt/tests/collaboration/assert_collab_document_counts_no_spike.sql	recent CTE limits source data to the last 120 days using FINAL; unpivoted CTE array-joins five activity counter columns into (metric, value) rows, dropping nulls.
Baseline stats and spike filter src/ingestion/dbt/tests/collaboration/assert_collab_document_counts_no_spike.sql	baselined CTE applies window functions per (tenant_id, person_key, product, data_source, metric) to compute prior_n, base_mean, base_sd, and prev_date over the 28 prior observed rows; final SELECT restricts to dates in the last 3 days and applies four qualifying conditions: prior_n ≥ 14, base_sd > 0, date − prev_date ≤ 35, and value exceeding both base_mean + 5 × base_sd and base_mean × 10.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 Hop, hop, through the data stream,
Counting documents — a noisy dream!
Z-scores checked, the baseline set,
Ten-times spikes? Not a chance, I bet.
With 28 days of trailing sight,
This bunny keeps the data right! 🌟

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately and specifically describes the main change: a spike detection test for collaborative document activity using per-user trailing baselines.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/ingestion/dbt/tests/collaboration/assert_collab_document_counts_no_spike.sql`:
- Line 115: In the WHERE clause of the spike detection query in
assert_collab_document_counts_no_spike.sql, replace the filter condition that
uses activity date (the line with AND date >= today() - 3) with a filter on the
collection timestamp instead. Change the filter to use collected_at >= today() -
3 to ensure late-arriving backfill data is properly included in the spike
detection window. If collected_at is not available in the silver schema, use
_version >= today() - 3 as an alternative to capture the ReplacingMergeTree
update timestamp.
- Around line 43-96: The spike detection test is missing `insight_source_id`
from its partitioning grain, which allows activity from different M365 source
instances within the same tenant to share a contaminated baseline. Add
`insight_source_id` to the SELECT clause in each of the three CTEs (recent,
unpivoted, and baselined) and include it in the PARTITION BY clause of the
WINDOW w definition alongside the existing partition keys (tenant_id,
person_key, product, data_source, metric). This ensures each source instance
maintains its own isolated adaptive baseline for accurate spike detection.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 250adc65-335a-4768-959d-0ec51ee3c04e

📥 Commits

Reviewing files that changed from the base of the PR and between c32821d and 09e8474.

📒 Files selected for processing (1)

• src/ingestion/dbt/tests/collaboration/assert_collab_document_counts_no_spike.sql