diff --git a/docs/glossary.mdx b/docs/glossary.mdx index 3c5d6672..5aa23c9b 100644 --- a/docs/glossary.mdx +++ b/docs/glossary.mdx @@ -746,6 +746,20 @@ Lower variance improves power and reduces how long experiments need to run. --- +## W + +### Warehouse-native +A deployment mode where experiment analysis runs directly inside your own data warehouse instead of in the experimentation platform's cloud. +ABsmartly handles experiment assignment, management, statistical analysis and metric governance, while the underlying data — exposures, goals and attributes — stays in your warehouse. + +Warehouse-native experimentation supports a single source of truth between experimentation and BI/finance reporting, helps meet data residency and compliance requirements (GDPR, SOC2, HIPAA) by keeping user-level data within your own infrastructure, and provides auditability through your existing data governance tools. + +ABsmartly supports warehouse-native analysis on BigQuery, Snowflake, ClickHouse, Redshift and Databricks. + +**Example:** Connecting ABsmartly to your Snowflake account, mapping your exposure and event tables to ABsmartly's expected schema, and letting ABsmartly compute experiment results on a refresh schedule — without copying user-level data out of Snowflake. + +--- + ## Z ### Z-score diff --git a/docs/platform-release-notes/2026/05.mdx b/docs/platform-release-notes/2026/05.mdx new file mode 100644 index 00000000..d3d7eb28 --- /dev/null +++ b/docs/platform-release-notes/2026/05.mdx @@ -0,0 +1,90 @@ +import Image from "../../../src/components/Image"; + +# May 2026 + +## Overview + +This release brings a long-awaited **Metric Changelog**, introduces the new **activity_per_period** metric type, +and ships several quality-of-life improvements across the Explore Metrics tab, experiment list, and Events dashboard. + +--- + +## Metrics + +### Metric Changelog + +Every metric now has a full **change log** that captures every modification made over its lifetime, so you always know **what changed, who changed it, and when**. + +Open the change log from any metric to see a chronological timeline of changes, with the most recent at the top. Each entry shows: + +- **Who** made the change, with avatar and name +- **When** it happened (relative time, with the exact timestamp on hover) +- **Which metric version** the change produced +- **A short summary** of the change (e.g. _Changed owners_, _Added metadata_) +- **Field-level before/after diffs**, grouped into collapsible sections such as **Identity**, **Details**, **Definition** **Goal**, with clear add/remove indicators + +This gives teams a complete audit trail for metric governance which is useful for reviews, debugging unexpected results, and understanding the history behind a metric. +The same change log will soon be available for other assets like **experiments** and **features**. + +Metric changelog showing chronological changes with who, when, and field-level diffs + +--- + +### New Metric Type: `activity_per_period` + +We've added a new metric type, **[activity_per_period](/docs/web-console-docs/goals-and-metrics/metrics/metric-types/activity-per-period)**, that measures user activity over a configurable time window (for example, _sessions per week_ or _orders per month_). +It's designed for cases where you care about the **frequency** of an action per user across a rolling period, rather than a single count or ratio. + +See the [Activity per Period documentation](/docs/web-console-docs/goals-and-metrics/metrics/metric-types/activity-per-period) for more information. + +### Improved Metrics Discoverability + +Finding the right metric on the **Explore Metrics** tab is now much easier. +We've improved how metrics surface in the list, making it faster to locate the metrics you care about when investigating an experiment. + +--- + +## Events + +### Keyboard Navigation in the Event Details Dialog + +The **Event Details** dialog now supports keyboard arrow navigation, so you can step through events without reaching for the mouse. A small change that makes browsing through long event lists noticeably faster. + +--- + +## Experiments + +### Custom Assignments (Early Beta) + +We're opening up an **early beta** of **Custom Assignments** — a new way to define who should be **excluded from an experiment** and always served a specific variant instead. + +Custom Assignments (also known as **exclusion rules**) are useful for: + +- **Testing & QA** — force yourself or your team into a specific variant to verify the experience, without being counted as a participant. +- **Internal or beta users** — ensure groups like internal employees or beta testers always see a chosen variant, without polluting experiment data. + +Excluded visitors see the configured variant but **are not tracked as participants**, so the experiment statistics stay clean. + +See the [Custom Assignments documentation](/docs/web-console-docs/experiments/creating-an-experiment#custom-assignments-exclusion-rules) for details on how to configure them. + +:::caution Early Beta — JavaScript SDK Only +Custom Assignments are currently in **early beta** and are **only available for the JavaScript SDK**. The feature is behind a flag and requires an **SDK update** before it can be used. +If you'd like to enable it for your account, please **[reach out to our team](mailto:support@absmartly.com)** and we'll help you get set up. +::: + +### Group Experiment List by Experiment + +You can now **group the experiment list by experiment**, making it easier to navigate large lists +especially when working with experiment families or multiple iterations of the same test. + +--- + +## Bug Fixes + +This release also includes a number of security, stability and reliability improvements based on feedback from users. + +--- + +## Questions or Feedback? + +We're always happy to help, so reach out if you have any questions or want to explore how to make the most of these new capabilities. diff --git a/docs/web-console-docs/experiments/creating-an-experiment.mdx b/docs/web-console-docs/experiments/creating-an-experiment.mdx index 34de9651..19b5f339 100644 --- a/docs/web-console-docs/experiments/creating-an-experiment.mdx +++ b/docs/web-console-docs/experiments/creating-an-experiment.mdx @@ -116,20 +116,20 @@ their data will not be tracked. When audience enforced is **off**, it is the responsibility of the experimenter to ensure that only eligible visitors are shown the experiments. ABsmartly will only warn you when visitors not matching the audience are exposed to your experiment. In this case you would see an **audience mismatch warning** in your experiment. This option makes it easier to full-on when the experiment is completed as the audience filtering is not part of the experiment set up. -## Exclusion Rules +### Custom Assignments (Exclusion Rules) Exclusion rules let you define which visitors or groups of visitors should be **excluded from the experiment** and instead always see a specific variant. Configuring exclusion rules for an experiment -### When to Use Exclusion Rules +#### When to Use Custom Assignments Exclusion rules are useful in two main scenarios: - **Testing & QA** — Force yourself or your team into a specific variant to verify the experience before or during the experiment, without being counted as a participant. - **Internal or beta users** — Ensure that a specific group of users (e.g. internal employees, beta testers) always sees the new variant without polluting the experiment data. -### How It Works +#### How It Works An exclusion rule defines: @@ -142,7 +142,7 @@ Excluded visitors are **not part of the experiment**. They will see the assigned You can add multiple exclusion rules to a single experiment. Rules are evaluated in order — a visitor matching any rule will be excluded. ::: -### Example +#### Example To exclude internal users and always show them Variant B: @@ -152,8 +152,9 @@ To exclude internal users and always show them Variant B: This ensures your team can experience the new variant in production while keeping the experiment data clean. -:::caution -Exclusion rules require an **SDK update**. Make sure your SDKs are up to date before using this feature. +:::caution Early Beta — JavaScript SDK Only +Exclusion rules (also known as **Custom Assignments**) are currently in **early beta** and are **only available for the JavaScript SDK**. The feature is behind a flag and requires an **SDK update** before it can be used. +If you'd like to enable it for your account, please **[reach out to our team](mailto:support@absmartly.com)** and we'll help you get set up. See the [SDK documentation](/docs/APIs-and-SDKs/SDK-Documentation/running-your-first-experiment) for details. ::: diff --git a/docs/web-console-docs/goals-and-metrics/metrics/metric-types/activity-per-period.mdx b/docs/web-console-docs/goals-and-metrics/metrics/metric-types/activity-per-period.mdx new file mode 100644 index 00000000..87ec9d75 --- /dev/null +++ b/docs/web-console-docs/goals-and-metrics/metrics/metric-types/activity-per-period.mdx @@ -0,0 +1,57 @@ +--- +sidebar_position: 8 +--- + +# Activity per Period + +## Overview + +An `Activity per Period` metric measures **how frequently** a user performs a goal action across a configurable time window +— for example, _sessions per week_ or _orders per month_. + +Unlike a simple `Goal Count`, which sums all events per user, `Activity per Period` normalises activity into a **rate per period**, making it easier to compare user engagement over time and across cohorts of different exposure lengths. + +You can configure: + +- the **goal event** to track +- the **period length** (e.g. day, week, month) +- whether the period starts: + - from the user's first exposure to the experiment, or + - from the user's first occurrence of the goal event + +## Examples + +```javascript +context.track("session_start", { + source: "homepage", + device: "mobile" +}); +``` + +Imagine you want to measure **average sessions per week per user** during your experiment. + +You can create a `Sessions per Week` metric by: +- Selecting the `session_start` goal +- Setting the `Period` to 7 days +- Choosing whether periods start from the user's first exposure or first session + +The metric then computes, for each user, the number of goal events per period, and aggregates the result across the experiment population. + +**More examples** + +- `Orders per Month`: +Average number of purchases per user per 30-day period. + +- `Articles Read per Week`: +Average number of articles read per user, per 7-day period. + +- `Active Days per Week`: +Average number of distinct days a user was active in a 7-day window. + +## Good to know + +- Useful when you care about **frequency** of an action rather than total volume or simple conversion. +- More robust than `Goal Count` when users have different exposure lengths — the per-period normalisation makes long-tenured and short-tenured users directly comparable. +- Filters on the goal event apply before the per-period rollup (for example: _purchases per week_ restricted to a specific category). +- Changing the period length alters the meaning of the metric and will require a new version. +- Ideal for experiments aimed at **engagement, habit formation, or retention frequency** — onboarding flows, notifications, recommendation systems, or loyalty mechanics. diff --git a/static/img/release-notes/metric-changelog.png b/static/img/release-notes/metric-changelog.png new file mode 100644 index 00000000..7694456a Binary files /dev/null and b/static/img/release-notes/metric-changelog.png differ