From d2015681950f00a7b4d002f3f8d96ede30b4372e Mon Sep 17 00:00:00 2001 From: Christopher Date: Thu, 2 Jul 2026 00:52:26 +1000 Subject: [PATCH] docs(concepts): define Tags as the promptfoo map; note manual tags removed (av-qsxw) Adds a "Tags" entry (promptfoo-shaped Record map; reserved `experiment` key; Dashboard Tags-tab grouping; the sole tags concept after the manual per-run chips were removed) and notes on the Experiment entry that experiment is expressed as `tags.experiment` with no top-level field. Co-Authored-By: Claude Opus 4.8 (1M context) --- CONCEPTS.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CONCEPTS.md b/CONCEPTS.md index 1f671a3d5..767bb008e 100644 --- a/CONCEPTS.md +++ b/CONCEPTS.md @@ -20,7 +20,9 @@ Shared domain vocabulary for this project — entities, named processes, and sta **Wrapper eval** — Eval YAML whose main job is to import task suites and bind top-level runtime policy such as target selection, repeat count, timeout, budget, and thresholds. Wrapper evals may live under an `experiments/` directory, but that path is an optional user-owned convention and AgentV does not infer behavior from it. A wrapper that imports suites with `type: suite` does not define parent `workspace`; imported suites own task environment. -**Experiment** — A string metadata/run-grouping label such as `baseline`, `candidate`, `with_skills`, or `without_skills`. It is not a runtime-policy object and not a result path namespace. Runtime policy belongs in top-level eval fields or target objects; the experiment label is recorded in `summary.json` and `index.jsonl` for Dashboard grouping and comparison. Lifecycle setup belongs in `workspace.hooks` or `targets[].hooks`, not in a separate experiment artifact. +**Experiment** — A string metadata/run-grouping label such as `baseline`, `candidate`, `with_skills`, or `without_skills`. It is not a runtime-policy object and not a result path namespace. Experiment is expressed as the reserved `tags.experiment` key (see **Tags**); there is no top-level `experiment` field. Runtime policy belongs in top-level eval fields or target objects; the experiment label is recorded in `summary.json` and `index.jsonl` for Dashboard grouping and comparison. Lifecycle setup belongs in `workspace.hooks` or `targets[].hooks`, not in a separate experiment artifact. + +**Tags** — A promptfoo-shaped `Record` map authored on an eval (or project config / `--tag key=value`) that labels a run with structured facets such as `experiment`, `team`, or `env`. The reserved `experiment` key feeds the experiment namespace. The resolved map is recorded in `summary.json` `metadata.tags` and every `index.jsonl` row, and the Dashboard "Tags" tab groups and compares runs by any tag key. This is the only "tags" concept: the earlier free-form manual per-run tag chips have been removed. (Suite-level `tags` may still be authored as a string list, which is a selection construct for `select.tags` / `--tag name` filtering rather than run metadata.) **Workspace** — The task environment an eval prepares for the agent: repositories, templates, fixture files, and lifecycle hooks. It is not prompt input; use `input` for instructions and `workspace.repos[]` for multi-repo workspaces the agent can inspect or modify through tools.