From f3e8bd1868a601f9dc5598664391d47d5f8148d9 Mon Sep 17 00:00:00 2001 From: Kalen Kennedy Date: Thu, 23 Apr 2026 10:53:00 +1000 Subject: [PATCH] Ship: docs: add test mining implementation planning set --- AGENTS.md | 308 +-- README.md | 176 +- docs/README.md | 35 + docs/implementation-roadmap.md | 169 ++ docs/requirements-traceability-matrix.md | 73 + docs/requirements.md | 3100 +++++++++++----------- docs/security-plan.md | 299 +++ docs/technical-specification.md | 471 ++++ docs/testing-plan.md | 329 +++ docs/ui-ux-plan.md | 349 +++ 10 files changed, 3517 insertions(+), 1792 deletions(-) create mode 100644 docs/README.md create mode 100644 docs/implementation-roadmap.md create mode 100644 docs/requirements-traceability-matrix.md create mode 100644 docs/security-plan.md create mode 100644 docs/technical-specification.md create mode 100644 docs/testing-plan.md create mode 100644 docs/ui-ux-plan.md diff --git a/AGENTS.md b/AGENTS.md index 8dab768..895e107 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,154 +1,154 @@ -# AGENTS.md - -This file is the operating contract for any agent — human or AI — making changes in this repository. Read it before making a change. If it conflicts with a user instruction, surface the conflict and ask before proceeding. - -## 1. Mission - -This repository hosts two concerns that must not be conflated: - -1. **Primary product:** the semantic test mining platform specified in [docs/requirements.md](docs/requirements.md), with conceptual background in [docs/concept.md](docs/concept.md). New application work targets this product. -2. **Retained tooling:** Symphony (`SPEC.md`, `IMPLEMENTATION_PLAN.md`, `WORKFLOW.md`, `src/Symphony.*`, `tests/Symphony.*`, `symphony_docs/`). Symphony is preserved because it is used *by* this project; it is not the product described by `docs/requirements.md`. - -Every change must declare, in its pull request description, which of the two concerns it targets. Cross-boundary changes are split into separate pull requests. - -## 2. Sources of Truth - -In order of precedence, for test mining platform work: - -1. `docs/requirements.md` (product specification) -2. `docs/concept.md` (design rationale; informative, not normative) -3. `README.md` (user-facing framing) - -For Symphony work: - -1. `SPEC.md` -2. `IMPLEMENTATION_PLAN.md` -3. `WORKFLOW.md` - -If a spec and a plan disagree, the spec wins and the plan is updated. If requirements and Symphony docs disagree about repository direction, `docs/requirements.md` wins for product work; Symphony docs continue to govern Symphony work. - -## 3. Non-Negotiable Guardrails - -These are hard stops. Do not weaken them without an explicit, documented decision. - -1. **Never persist, log, or emit captured secrets.** This includes `GITHUB_TOKEN`, workflow secrets, target-application credentials, session cookies, Playwright storage states, bearer tokens, and anything classified as `Sensitive`. If you are unsure whether a value is sensitive, treat it as sensitive. -2. **Never launch a Playwright browser context against a target URL absent from the administrator-managed allow-list** (requirements §12.4). The allow-list check is server-side and runs before context creation. -3. **Never encrypt-at-rest bypass.** Storage states, cookies, cached auth material, and sensitive scenario variables are encrypted at rest (requirements §12.5). No code path may write them plaintext. -4. **Never edit generated source files by hand and commit the result.** Generated output is a derived artefact. Regenerate from scenario data and templates (requirements FR-GEN-001). -5. **Never auto-apply healing to a persisted scenario in v1.** Healing produces proposals; humans approve (requirements FR-HEAL-003). -6. **Never let AI be the runtime-critical path.** AI assists; deterministic logic decides (requirements principle 3). -7. **Never mutate Symphony files from a test-mining task, or vice versa.** Scope discipline is enforced at PR review. -8. **Never parse stderr as protocol events, and never parse secrets out of logs.** (Inherited Symphony constraint retained for Symphony work.) - -## 4. Architecture Rules - -1. Clear boundaries. Core domain and scenario contracts must not depend on concrete infrastructure APIs. Infrastructure adapters implement interfaces declared in core. -2. Ports and adapters for the big four external systems: Playwright, persistence, artefact storage, identity provider. Each has a core-defined interface and an isolated adapter project. -3. Persistence: - - EF Core with migrations. - - PostgreSQL is the v1 default provider (requirements §6.2). SQL Server is a deferred option behind the same abstraction. - - No provider-specific SQL in domain or application layers. Isolated, gated, and justified in infrastructure only. - - Optimistic concurrency (`RowVersion`) on mutable aggregates; soft delete on `Scenario`, `RecordingSession`, `GenerationArtifact`, and artefact records (requirements §7.2.0). -4. Recording engine: - - Playwright-native instrumentation hooks (`AddInitScriptAsync` and exposed bindings) are the primary event transport. Console scraping is a bounded fallback only (FR-REC-009). - - Recorder script ships from source control with a pinned `RecorderVersion` surfaced in captures (requirements §22.4). No runtime fetch from third-party origins. -5. Replay engine: - - Isolated browser contexts per run (FR-REP-005). - - Deterministic heuristics first; AI advisory at most. -6. Generation: - - Scenarios are the source of truth. Output is reproducible (FR-GEN-001, FR-GEN-010). - - Snapshot/approval tests for generator output (FR-GEN, §14.4). - -## 5. Coding Standards - -- C# language version must remain compatible with the repository's configured .NET build settings. Do not change SDK expectations without an explicit, reviewed change. -- Nullable reference types enabled. -- Async all the way for I/O; cancellation tokens propagated through polling, subprocess, and HTTP. -- Analyzer warnings treated as errors in core projects. -- Keep classes focused and small. Split large orchestration into feature services. -- Prefer built-in ASP.NET Core and .NET primitives over third-party packages unless a concrete justification is recorded. - -## 6. Security Hygiene - -1. Secret-like strings must not appear in diffs. CI secret-scanning runs on every PR (requirements §22.3). -2. Any new captured data field or outbound network call is called out explicitly in the PR description. -3. Dependency vulnerability checks run before merge. -4. Structured logs include correlation IDs (recording session, scenario version, replay run, user) but never raw event payloads that could contain sensitive values. -5. Masking policy changes are retroactive: changing a classification to `Sensitive` must re-mask prior previews and re-generation (requirements §12.5). - -## 7. Testing Expectations - -Minimum bar for any non-trivial change: - -1. Unit tests for business logic and state transitions. -2. Integration tests for infrastructure boundaries touched. Use real PostgreSQL (Testcontainers) for persistence integration — not an in-memory provider. -3. Conformance tests tagged with the requirement identifier (e.g. `FR-REC-004`) they cover. -4. Generator changes require snapshot tests (FR-GEN, §14.4). -5. Flaky tests are fixed or quarantined within one working day; persistent quarantine is not acceptable. -6. For Symphony work, keep or extend the `SPEC.md` section-17 conformance tests. - -## 8. Configuration and Options - -- Read target URL allow-lists, retention policies, and generation defaults from the configuration surface described in requirements §11. -- Resolve `$ENV_VAR` values in configuration. -- Fail fast on invalid required configuration at startup or on administrative save. -- Keep configurable defaults aligned with requirements §22 (build) and §7.4 (retention). - -## 9. Observability - -- Structured logs with recording session id, scenario id, scenario version id, replay run id, user id, browser context id (requirements §13.1). -- Operational metrics per §13.2 — sufficient to verify the performance targets in §15.2. -- Snapshot/status surfaces derived from orchestrator or scenario engine state, not ad hoc caches. -- Artefact traceability: every screenshot, trace, DOM snapshot, and generated bundle links back to the session, version, or replay run that produced it. - -## 10. Delivery Workflow - -1. Cite the relevant requirement identifier(s) (e.g. `FR-REC-004`) in the PR description. -2. State which of the two concerns from §1 this change targets. -3. Implement the smallest vertical slice that demonstrates the requirement. -4. Add or update tests mapped to the cited requirements. -5. Run `dotnet build` and `dotnet test` locally before handing off. -6. Update `README.md` or `docs/` if user-visible behaviour changed. -7. Keep PRs scoped. Refactors that cross the Symphony/test-mining boundary are split. - -## 11. Suggested Commands - -```powershell -dotnet restore -dotnet build -dotnet test -``` - -Database migrations (platform persistence project, once introduced): - -```powershell -dotnet ef migrations add --project src/TestMining.Platform.Persistence -dotnet ef database update --project src/TestMining.Platform.Persistence -``` - -Symphony migrations (retained tooling): - -```powershell -dotnet ef migrations add --project src/Symphony.Infrastructure/Persistence.Sqlite -dotnet ef database update --project src/Symphony.Infrastructure/Persistence.Sqlite -``` - -## 12. Definition of Done (Per Change) - -1. Behaviour aligns with `docs/requirements.md` (or `SPEC.md` for Symphony work). -2. Tests prove the behaviour or failure mode, tagged with requirement identifiers. -3. Logging and error paths are explicit. No raw sensitive values in logs. -4. No secrets, keys, or tokens in diffs or output. -5. Reviewer can trace the change from requirement clause to implementation to test. -6. Retention, security, or captured-data surface changes are explicitly called out in the PR. -7. Generated code is reproducible from scenario data plus template version; no hand-edits committed. - -## 13. Escalation - -If a requirement is ambiguous, silent on an edge case, or appears to contradict another section: - -1. Stop. Do not guess. -2. Raise the question in the PR description or as a clarifying note. -3. Record the resolution either by updating `docs/requirements.md` under the originating section or by adding an architecture decision record under `docs/adr/` once that directory exists. - -Shortcuts that trade safety, auditability, or reproducibility for speed are not acceptable. When an obstacle appears, identify the root cause rather than bypassing a guardrail. +# AGENTS.md + +This file is the operating contract for any agent — human or AI — making changes in this repository. Read it before making a change. If it conflicts with a user instruction, surface the conflict and ask before proceeding. + +## 1. Mission + +This repository hosts two concerns that must not be conflated: + +1. **Primary product:** the semantic test mining platform specified in [docs/requirements.md](docs/requirements.md), with conceptual background in [docs/concept.md](docs/concept.md). New application work targets this product. +2. **Retained tooling:** Symphony (`SPEC.md`, `IMPLEMENTATION_PLAN.md`, `WORKFLOW.md`, `src/Symphony.*`, `tests/Symphony.*`, `symphony_docs/`). Symphony is preserved because it is used *by* this project; it is not the product described by `docs/requirements.md`. + +Every change must declare, in its pull request description, which of the two concerns it targets. Cross-boundary changes are split into separate pull requests. + +## 2. Sources of Truth + +In order of precedence, for test mining platform work: + +1. `docs/requirements.md` (product specification) +2. `docs/concept.md` (design rationale; informative, not normative) +3. `README.md` (user-facing framing) + +For Symphony work: + +1. `SPEC.md` +2. `IMPLEMENTATION_PLAN.md` +3. `WORKFLOW.md` + +If a spec and a plan disagree, the spec wins and the plan is updated. If requirements and Symphony docs disagree about repository direction, `docs/requirements.md` wins for product work; Symphony docs continue to govern Symphony work. + +## 3. Non-Negotiable Guardrails + +These are hard stops. Do not weaken them without an explicit, documented decision. + +1. **Never persist, log, or emit captured secrets.** This includes `GITHUB_TOKEN`, workflow secrets, target-application credentials, session cookies, Playwright storage states, bearer tokens, and anything classified as `Sensitive`. If you are unsure whether a value is sensitive, treat it as sensitive. +2. **Never launch a Playwright browser context against a target URL absent from the administrator-managed allow-list** (requirements §12.4). The allow-list check is server-side and runs before context creation. +3. **Never encrypt-at-rest bypass.** Storage states, cookies, cached auth material, and sensitive scenario variables are encrypted at rest (requirements §12.5). No code path may write them plaintext. +4. **Never edit generated source files by hand and commit the result.** Generated output is a derived artefact. Regenerate from scenario data and templates (requirements FR-GEN-001). +5. **Never auto-apply healing to a persisted scenario in v1.** Healing produces proposals; humans approve (requirements FR-HEAL-003). +6. **Never let AI be the runtime-critical path.** AI assists; deterministic logic decides (requirements principle 3). +7. **Never mutate Symphony files from a test-mining task, or vice versa.** Scope discipline is enforced at PR review. +8. **Never parse stderr as protocol events, and never parse secrets out of logs.** (Inherited Symphony constraint retained for Symphony work.) + +## 4. Architecture Rules + +1. Clear boundaries. Core domain and scenario contracts must not depend on concrete infrastructure APIs. Infrastructure adapters implement interfaces declared in core. +2. Ports and adapters for the big four external systems: Playwright, persistence, artefact storage, identity provider. Each has a core-defined interface and an isolated adapter project. +3. Persistence: + - EF Core with migrations. + - PostgreSQL is the v1 default provider (requirements §6.2). SQL Server is a deferred option behind the same abstraction. + - No provider-specific SQL in domain or application layers. Isolated, gated, and justified in infrastructure only. + - Optimistic concurrency (`RowVersion`) on mutable aggregates; soft delete on `Scenario`, `RecordingSession`, `GenerationArtifact`, and artefact records (requirements §7.2.0). +4. Recording engine: + - Playwright-native instrumentation hooks (`AddInitScriptAsync` and exposed bindings) are the primary event transport. Console scraping is a bounded fallback only (FR-REC-009). + - Recorder script ships from source control with a pinned `RecorderVersion` surfaced in captures (requirements §22.4). No runtime fetch from third-party origins. +5. Replay engine: + - Isolated browser contexts per run (FR-REP-005). + - Deterministic heuristics first; AI advisory at most. +6. Generation: + - Scenarios are the source of truth. Output is reproducible (FR-GEN-001, FR-GEN-010). + - Snapshot/approval tests for generator output (FR-GEN, §14.4). + +## 5. Coding Standards + +- C# language version must remain compatible with the repository's configured .NET build settings. Do not change SDK expectations without an explicit, reviewed change. +- Nullable reference types enabled. +- Async all the way for I/O; cancellation tokens propagated through polling, subprocess, and HTTP. +- Analyzer warnings treated as errors in core projects. +- Keep classes focused and small. Split large orchestration into feature services. +- Prefer built-in ASP.NET Core and .NET primitives over third-party packages unless a concrete justification is recorded. + +## 6. Security Hygiene + +1. Secret-like strings must not appear in diffs. CI secret-scanning runs on every PR (requirements §22.3). +2. Any new captured data field or outbound network call is called out explicitly in the PR description. +3. Dependency vulnerability checks run before merge. +4. Structured logs include correlation IDs (recording session, scenario version, replay run, user) but never raw event payloads that could contain sensitive values. +5. Masking policy changes are retroactive: changing a classification to `Sensitive` must re-mask prior previews and re-generation (requirements §12.5). + +## 7. Testing Expectations + +Minimum bar for any non-trivial change: + +1. Unit tests for business logic and state transitions. +2. Integration tests for infrastructure boundaries touched. Use real PostgreSQL (Testcontainers) for persistence integration — not an in-memory provider. +3. Conformance tests tagged with the requirement identifier (e.g. `FR-REC-004`) they cover. +4. Generator changes require snapshot tests (FR-GEN, §14.4). +5. Flaky tests are fixed or quarantined within one working day; persistent quarantine is not acceptable. +6. For Symphony work, keep or extend the `SPEC.md` section-17 conformance tests. + +## 8. Configuration and Options + +- Read target URL allow-lists, retention policies, and generation defaults from the configuration surface described in requirements §11. +- Resolve `$ENV_VAR` values in configuration. +- Fail fast on invalid required configuration at startup or on administrative save. +- Keep configurable defaults aligned with requirements §22 (build) and §7.4 (retention). + +## 9. Observability + +- Structured logs with recording session id, scenario id, scenario version id, replay run id, user id, browser context id (requirements §13.1). +- Operational metrics per §13.2 — sufficient to verify the performance targets in §15.2. +- Snapshot/status surfaces derived from orchestrator or scenario engine state, not ad hoc caches. +- Artefact traceability: every screenshot, trace, DOM snapshot, and generated bundle links back to the session, version, or replay run that produced it. + +## 10. Delivery Workflow + +1. Cite the relevant requirement identifier(s) (e.g. `FR-REC-004`) in the PR description. +2. State which of the two concerns from §1 this change targets. +3. Implement the smallest vertical slice that demonstrates the requirement. +4. Add or update tests mapped to the cited requirements. +5. Run `dotnet build` and `dotnet test` locally before handing off. +6. Update `README.md` or `docs/` if user-visible behaviour changed. +7. Keep PRs scoped. Refactors that cross the Symphony/test-mining boundary are split. + +## 11. Suggested Commands + +```powershell +dotnet restore +dotnet build +dotnet test +``` + +Database migrations (platform persistence project, once introduced): + +```powershell +dotnet ef migrations add --project src/TestMining.Platform.Persistence +dotnet ef database update --project src/TestMining.Platform.Persistence +``` + +Symphony migrations (retained tooling): + +```powershell +dotnet ef migrations add --project src/Symphony.Infrastructure/Persistence.Sqlite +dotnet ef database update --project src/Symphony.Infrastructure/Persistence.Sqlite +``` + +## 12. Definition of Done (Per Change) + +1. Behaviour aligns with `docs/requirements.md` (or `SPEC.md` for Symphony work). +2. Tests prove the behaviour or failure mode, tagged with requirement identifiers. +3. Logging and error paths are explicit. No raw sensitive values in logs. +4. No secrets, keys, or tokens in diffs or output. +5. Reviewer can trace the change from requirement clause to implementation to test. +6. Retention, security, or captured-data surface changes are explicitly called out in the PR. +7. Generated code is reproducible from scenario data plus template version; no hand-edits committed. + +## 13. Escalation + +If a requirement is ambiguous, silent on an edge case, or appears to contradict another section: + +1. Stop. Do not guess. +2. Raise the question in the PR description or as a clarifying note. +3. Record the resolution either by updating `docs/requirements.md` under the originating section or by adding an architecture decision record under `docs/adr/` once that directory exists. + +Shortcuts that trade safety, auditability, or reproducibility for speed are not acceptable. When an obstacle appears, identify the root cause rather than bypassing a guardrail. diff --git a/README.md b/README.md index 508d4d9..62ca7a9 100644 --- a/README.md +++ b/README.md @@ -1,88 +1,88 @@ -# Test Mining Platform - -This repository is being used to build a C#-based semantic test mining platform that records browser interactions, converts them into structured scenarios, and generates maintainable Playwright tests with replay and healing support. - -The current source of truth for the planned product is [docs/requirements.md](/mnt/c/Users/kenne/Desktop/ReleasedGroup/2EndSquaredTesting/docs/requirements.md). The concept document that informed it is [docs/concept.md](/mnt/c/Users/kenne/Desktop/ReleasedGroup/2EndSquaredTesting/docs/concept.md). - -## Product Summary - -The application described in this repository is not intended to be a raw click recorder. It is intended to be a semantic test mining and stabilisation platform that: - -- records real user workflows in a browser -- captures DOM, accessibility, navigation, and runtime context around meaningful actions -- transforms raw events into structured, intent-rich scenarios -- infers resilient locators, variable data strategies, and outcome-oriented assertions -- generates readable C# Playwright tests and supporting helpers -- replays generated scenarios and provides deterministic healing when selectors drift - -The core design rule is that structured scenarios are the source of truth. Generated code is a derived artefact and must remain reproducible from scenario data and generation settings. - -## Intended Stack - -The requirements currently define this implementation model: - -- Backend: ASP.NET Core -- Frontend: Blazor Server -- Database: PostgreSQL or SQL Server -- Browser automation: Microsoft.Playwright for .NET -- Code generation: Roslyn or Scriban templates -- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper if needed - -## Planned Capabilities - -The platform is expected to include these major areas: - -- Recording engine for guided browser-session capture -- DOM and accessibility analysis -- Scenario authoring and timeline review -- Locator ranking and assertion suggestion -- C# Playwright code generation -- Replay diagnostics -- Deterministic healing workflows -- Persistent storage for sessions, scenarios, artefacts, and replay history -- Administrative configuration for environments, browser settings, and generation defaults - -## Repository Guidance - -This repository currently contains existing Symphony-related files and scaffolding. Those files are being retained in the repository because Symphony is a tool used within this project context. - -Symphony guidance for users and agents: - -- Treat Symphony-related files as retained tool files, not as the application being described in `docs/requirements.md`. -- Do not rewrite, repurpose, or delete Symphony files unless a task explicitly asks for Symphony/tooling work. -- Keep Symphony files in the repository even while building the new application. -- New product work should align to `docs/requirements.md`, not the legacy Symphony README content that previously occupied this file. - -Examples of retained Symphony-related content include: - -- `SPEC.md` -- `IMPLEMENTATION_PLAN.md` -- `WORKFLOW.md` -- `src/Symphony.*` -- `tests/Symphony.*` -- `symphony_docs/` - -## Expected Solution Direction - -A likely repository shape for the new application is described in the requirements document and includes logical areas such as: - -- host and Blazor Server UI -- core domain and scenario contracts -- recording and Playwright orchestration -- analysis and inference -- code generation -- replay and healing -- persistence and artefact storage -- automated tests - -## Current Status - -The repository is in a planning and transition stage: - -- the requirements specification exists in `docs/requirements.md` -- the concept source exists in `docs/concept.md` -- the new application implementation may coexist with retained Symphony tooling content during development - -## Working Rule - -If there is a conflict between older Symphony-oriented repository documentation and the new product direction, treat `docs/requirements.md` as the authoritative description of the application to be built, while still preserving Symphony-related files as tooling assets unless explicitly instructed otherwise. +# Test Mining Platform + +This repository is being used to build a C#-based semantic test mining platform that records browser interactions, converts them into structured scenarios, and generates maintainable Playwright tests with replay and healing support. + +The current source of truth for the planned product is [docs/requirements.md](/mnt/c/Users/kenne/Desktop/ReleasedGroup/2EndSquaredTesting/docs/requirements.md). The concept document that informed it is [docs/concept.md](/mnt/c/Users/kenne/Desktop/ReleasedGroup/2EndSquaredTesting/docs/concept.md). + +## Product Summary + +The application described in this repository is not intended to be a raw click recorder. It is intended to be a semantic test mining and stabilisation platform that: + +- records real user workflows in a browser +- captures DOM, accessibility, navigation, and runtime context around meaningful actions +- transforms raw events into structured, intent-rich scenarios +- infers resilient locators, variable data strategies, and outcome-oriented assertions +- generates readable C# Playwright tests and supporting helpers +- replays generated scenarios and provides deterministic healing when selectors drift + +The core design rule is that structured scenarios are the source of truth. Generated code is a derived artefact and must remain reproducible from scenario data and generation settings. + +## Intended Stack + +The requirements currently define this implementation model: + +- Backend: ASP.NET Core +- Frontend: Blazor Server +- Database: PostgreSQL or SQL Server +- Browser automation: Microsoft.Playwright for .NET +- Code generation: Roslyn or Scriban templates +- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper if needed + +## Planned Capabilities + +The platform is expected to include these major areas: + +- Recording engine for guided browser-session capture +- DOM and accessibility analysis +- Scenario authoring and timeline review +- Locator ranking and assertion suggestion +- C# Playwright code generation +- Replay diagnostics +- Deterministic healing workflows +- Persistent storage for sessions, scenarios, artefacts, and replay history +- Administrative configuration for environments, browser settings, and generation defaults + +## Repository Guidance + +This repository currently contains existing Symphony-related files and scaffolding. Those files are being retained in the repository because Symphony is a tool used within this project context. + +Symphony guidance for users and agents: + +- Treat Symphony-related files as retained tool files, not as the application being described in `docs/requirements.md`. +- Do not rewrite, repurpose, or delete Symphony files unless a task explicitly asks for Symphony/tooling work. +- Keep Symphony files in the repository even while building the new application. +- New product work should align to `docs/requirements.md`, not the legacy Symphony README content that previously occupied this file. + +Examples of retained Symphony-related content include: + +- `SPEC.md` +- `IMPLEMENTATION_PLAN.md` +- `WORKFLOW.md` +- `src/Symphony.*` +- `tests/Symphony.*` +- `symphony_docs/` + +## Expected Solution Direction + +A likely repository shape for the new application is described in the requirements document and includes logical areas such as: + +- host and Blazor Server UI +- core domain and scenario contracts +- recording and Playwright orchestration +- analysis and inference +- code generation +- replay and healing +- persistence and artefact storage +- automated tests + +## Current Status + +The repository is in a planning and transition stage: + +- the requirements specification exists in `docs/requirements.md` +- the concept source exists in `docs/concept.md` +- the new application implementation may coexist with retained Symphony tooling content during development + +## Working Rule + +If there is a conflict between older Symphony-oriented repository documentation and the new product direction, treat `docs/requirements.md` as the authoritative description of the application to be built, while still preserving Symphony-related files as tooling assets unless explicitly instructed otherwise. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..3c7113a --- /dev/null +++ b/docs/README.md @@ -0,0 +1,35 @@ +# Test Mining Platform Documentation Index + +This folder now contains the planning and specification documents for the new test mining platform described by [requirements.md](./requirements.md). + +## Core Source Documents + +- [Requirements Specification](./requirements.md) +- [Concept Document](./concept.md) + +## Planning Documents + +- [Technical Specification](./technical-specification.md) +- [UI/UX Plan](./ui-ux-plan.md) +- [Security Plan](./security-plan.md) +- [Testing Plan](./testing-plan.md) +- [Implementation Roadmap](./implementation-roadmap.md) +- [Requirements Traceability Matrix](./requirements-traceability-matrix.md) + +## Recommended Reading Order + +1. `requirements.md` +2. `technical-specification.md` +3. `ui-ux-plan.md` +4. `security-plan.md` +5. `testing-plan.md` +6. `implementation-roadmap.md` +7. `requirements-traceability-matrix.md` + +## Usage Guidance + +- Treat `requirements.md` as the source of truth. +- Use the technical specification and roadmap to plan implementation slices. +- Use the UI/UX, security, and testing plans as non-optional design constraints. +- Use the traceability matrix to connect future code, tests, ADRs, and pull requests back to requirements. +- Treat all planning documents as subordinate to `requirements.md`; if a planning document and the requirements differ, update the planning document. diff --git a/docs/implementation-roadmap.md b/docs/implementation-roadmap.md new file mode 100644 index 0000000..273b370 --- /dev/null +++ b/docs/implementation-roadmap.md @@ -0,0 +1,169 @@ +# Test Mining Platform Implementation Roadmap + +## 1. Purpose + +This roadmap converts the delivery phasing in [docs/requirements.md](./requirements.md) Section 18 into implementation slices that fit the repository and reduce delivery risk. + +## 2. Roadmap Principles + +1. Build vertical slices, not disconnected layers. +2. Preserve `Symphony.*` tooling assets untouched unless explicitly requested. +3. Prove the canonical `Scenario` workflow early. +4. Treat security and observability as phase-entry requirements, not polish. +5. Keep each slice traceable to `FR-*` requirements and acceptance criteria. + +## 3. Foundation Track + +This track should start before Phase 1 feature depth expands. + +Deliverables: + +- `TestMining.Platform.*` project scaffolding +- PostgreSQL persistence bootstrap and migrations pipeline +- authentication shell and role model +- internal API versioning convention +- SignalR infrastructure +- artefact storage abstraction +- fixture app test harness +- baseline observability and audit primitives +- ADRs for generator choice, authentication provider, replay execution boundary, and draft persistence shape + +Requirement anchors: + +- Sections 6, 10, 11, 12.1, 13, 14, 16, 17, 22 + +## 4. Phase 1: Core Proof + +Objective: + +Prove the end-to-end path from recording to generated code to replay on a simple fixture application. + +Suggested slices: + +1. Recording session creation plus allow-list validation +2. Playwright browser launch and recorder injection +3. Meaningful event capture for navigation, click, fill, select, and checkbox +4. Incremental recording persistence +5. Initial timeline UI with human-readable steps +6. Locator candidate creation and ranking +7. Draft scenario editing and immutable version creation +8. Deterministic C# generator for one profile +9. Basic replay execution and step-level result reporting + +Phase 1 exit should match Section 18.5 Phase 1 exit criteria. + +Phase 1 is not complete unless URL allow-list enforcement and encryption of stored auth/session material are demonstrably working, because they are part of the stated exit criteria rather than optional hardening. + +## 5. Phase 2: Robustness + +Objective: + +Increase semantic quality and diagnostic usefulness without changing the core source-of-truth model. + +Suggested slices: + +1. assertion suggestion engine +2. variable classification editing +3. fuzzy/resilient assertion strategies +4. scoped locator support +5. replay failure screenshots +6. richer replay diagnostics packaging +7. confidence explanation surfaces + +Requirement anchors: + +- `FR-INF-006` through `FR-INF-009` +- `FR-AUTH-003`, `FR-AUTH-004` +- `FR-REP-002`, `FR-REP-003` + +## 6. Phase 3: Product Hardening + +Objective: + +Harden the system for multi-user and longer-lived operational use. + +Suggested slices: + +1. authentication bootstrap modes +2. export packaging workflow +3. trace viewer integration +4. healing approval workflow and version lineage +5. scenario history and diff views +6. retention cleanup scheduling and admin controls + +Requirement anchors: + +- `FR-ADM-002` +- `FR-HEAL-001` through `FR-HEAL-005` +- Sections 7.4, 12.7, 16.4 + +## 7. Phase 4: AI Assistance + +Objective: + +Add advisory AI features without weakening deterministic execution or approval rules. + +Suggested slices: + +1. AI-assisted naming suggestions +2. AI-assisted assertion suggestion refinement +3. AI-assisted healing proposal augmentation +4. page object refactoring suggestions + +Guardrails: + +- feature flagged +- disabled by default +- advisory only +- deterministic evidence always shown +- no mutation of persisted scenarios without the same approval and traceability rules required for deterministic healing + +## 8. Cross-Cutting Backlog + +These items should run across phases: + +- security hardening and audits +- performance instrumentation +- accessibility improvements in Blazor UI +- documentation updates +- CI pipeline maturation + +## 9. Recommended Work Breakdown Structure + +Suggested implementation order: + +1. scaffold projects and shared contracts +2. establish persistence, auth shell, and audit primitives +3. build recording pipeline +4. build timeline authoring and versioning +5. build generation +6. build replay +7. add diagnostics depth +8. add healing workflow +9. add advanced admin and AI-adjacent features + +## 10. Dependencies and Risks + +Key dependencies: + +- stable fixture applications for test coverage +- Playwright recorder script build strategy +- PostgreSQL dev/test environment availability +- final authentication provider selection + +Key risks: + +- over-investing in raw recording before scenario editing is solid +- leaking sensitive data through early diagnostics +- making generator output non-deterministic +- coupling replay too tightly to generated code instead of scenario model + +## 11. Milestone Definition + +Each roadmap slice should only be closed when it has: + +- linked requirement IDs +- passing automated coverage +- documented artefact/security impact +- structured logs and diagnostics where applicable +- no mutation of retained `Symphony.*` assets diff --git a/docs/requirements-traceability-matrix.md b/docs/requirements-traceability-matrix.md new file mode 100644 index 0000000..e1f09ae --- /dev/null +++ b/docs/requirements-traceability-matrix.md @@ -0,0 +1,73 @@ +# Test Mining Platform Requirements Traceability Matrix + +## 1. Purpose + +This matrix links the requirements in [docs/requirements.md](./requirements.md) to the implementation planning documents added for the repository. + +Use this document to keep future code changes, ADRs, test cases, and pull requests traceable to the source requirements. + +## 2. Planning Document Index + +- [Documentation Index](./README.md) +- [Technical Specification](./technical-specification.md) +- [UI/UX Plan](./ui-ux-plan.md) +- [Security Plan](./security-plan.md) +- [Testing Plan](./testing-plan.md) +- [Implementation Roadmap](./implementation-roadmap.md) + +## 3. Requirement Coverage Matrix + +| Requirement Area | Requirement Source | Primary Planning Docs | +| --- | --- | --- | +| Product intent and canonical scenario principle | Sections 1, 2, 7.1, 24 | Technical Specification, Implementation Roadmap | +| Scope and phase boundaries | Sections 3, 18, 21 | Technical Specification, Implementation Roadmap | +| Roles and primary use cases | Section 4 | UI/UX Plan, Technical Specification | +| Logical architecture and lifecycles | Sections 5, 17 | Technical Specification, Implementation Roadmap | +| Technology stack and provider strategy | Section 6 | Technical Specification | +| Domain entities and artefacts | Sections 7.2 through 7.5 | Technical Specification, Testing Plan | +| Recording requirements | `FR-REC-001` through `FR-REC-010` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan, Implementation Roadmap | +| Inference requirements | `FR-INF-001` through `FR-INF-009` | Technical Specification, UI/UX Plan, Testing Plan | +| Scenario authoring requirements | `FR-AUTH-001` through `FR-AUTH-006` | Technical Specification, UI/UX Plan, Testing Plan | +| Generation requirements | `FR-GEN-001` through `FR-GEN-010` | Technical Specification, UI/UX Plan, Testing Plan, Implementation Roadmap | +| Replay requirements | `FR-REP-001` through `FR-REP-005` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan | +| Healing requirements | `FR-HEAL-001` through `FR-HEAL-005` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan, Implementation Roadmap | +| Administration requirements | `FR-ADM-001` through `FR-ADM-004` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan | +| UI requirements | Section 9 | UI/UX Plan, Technical Specification | +| API and real-time requirements | Section 10 | Technical Specification, UI/UX Plan | +| Configuration requirements | Section 11 | Technical Specification, Security Plan | +| Security and privacy requirements | Section 12 | Security Plan, Testing Plan, Technical Specification | +| Observability and diagnostics | Section 13 | Technical Specification, Security Plan, Testing Plan | +| Verification and testing requirements | Section 14 | Testing Plan | +| Non-functional requirements | Section 15 | Technical Specification, UI/UX Plan, Testing Plan | +| Deployment and operational safety | Section 16 | Technical Specification, Security Plan, Implementation Roadmap | +| Acceptance criteria | Section 19 | Testing Plan, Technical Specification, Implementation Roadmap | +| Risks and constraints | Section 20 | Technical Specification, Implementation Roadmap | +| Build, test, and delivery requirements | Section 22 | Testing Plan, Technical Specification, Security Plan | +| Implementer guardrails | Section 23 | Security Plan, Technical Specification, Implementation Roadmap | + +## 4. Acceptance Criteria Mapping + +| Acceptance Criterion | Requirements Reference | Planned Evidence | +| --- | --- | --- | +| Record a workflow and persist a structured scenario | `FR-REC-001`, `FR-REC-004`, `FR-REC-010` | Technical Specification workflow design, Testing Plan Phase 1 path | +| Review, edit, and approve through UI | `FR-AUTH-001` through `FR-AUTH-006` | UI/UX Plan scenario editor, Testing Plan scenario authoring coverage | +| Generate readable C# Playwright artefacts | `FR-GEN-001` through `FR-GEN-010` | Technical Specification generation engine, Testing Plan generator verification | +| Replay with useful diagnostics | `FR-REP-001`, `FR-REP-002`, `FR-REP-003` | Technical Specification replay engine, UI/UX Plan replay workspace, Testing Plan replay coverage | +| Rank and persist locator candidates | `FR-INF-004`, `FR-INF-005` | Technical Specification analysis engine, Testing Plan inference coverage | +| Mask and encrypt sensitive values | `FR-REC-007`, Sections 12.3, 12.5 | Security Plan controls, Testing Plan security coverage | +| Approve/reject outcome-oriented assertions | `FR-INF-007`, `FR-AUTH-003` | UI/UX Plan inspector design, Testing Plan assertion approval coverage | +| Healing is deterministic, evidenced, and approval-based | `FR-HEAL-001` through `FR-HEAL-005` | Security Plan and Technical Specification healing workflow, Testing Plan healing coverage | +| Stack is ASP.NET Core + Blazor Server + PostgreSQL | Sections 6.1, 6.2 | Technical Specification solution shape | +| Generated output remains reproducible | `FR-GEN-001`, `FR-GEN-010` | Technical Specification generation design, Testing Plan snapshot strategy | +| Allow-list blocks disallowed targets | Section 12.4 | Security Plan allow-list control, Testing Plan allow-list tests | +| Observability supports performance verification | Sections 13.1, 15.2 | Technical Specification observability design, Testing Plan performance instrumentation | + +## 5. Future Use + +When code work begins, this matrix should be extended with: + +- implementation project/file locations +- test case IDs +- pull request references +- ADR references for open decisions +- links to evidence for phase exit criteria diff --git a/docs/requirements.md b/docs/requirements.md index 92fbddb..d561a94 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -1,1550 +1,1550 @@ -# @EndSquareTesting Requirements Specification - -## 1. Executive Summary - -The application defined by this specification is not a raw click recorder. It is a semantic test mining and stabilisation platform. - -The platform shall: - -- Record browser sessions performed by a human user against a target web application. -- Observe DOM state, accessibility metadata, navigation changes, network outcomes, and runtime context around each meaningful interaction. -- Transform raw browser events into a structured, intent-rich scenario model. -- Infer resilient selectors, variable data strategies, and business-level assertions. -- Generate readable C# Playwright test code and supporting assets from the scenario model. -- Replay generated tests and attempt deterministic healing when selectors or data drift. -- Preserve enough artefacts, metadata, and diagnostics to let users review, edit, approve, and regenerate scenarios over time. - -The platform shall be built with the following solution model: - -- Backend: ASP.NET Core -- Frontend: Blazor Server -- Database: PostgreSQL -- Browser automation: Microsoft.Playwright for .NET -- Code generation: Roslyn or Scriban templates -- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper - -The single most important product decision is this: the system shall store structured scenarios as the source of truth, not generated code. Generated code is a derived artefact and must always be reproducible from persisted scenario data plus generation settings. - -## 2. Product Vision - -### 2.1 Problem Statement - -Teams frequently want fast automated coverage of real user workflows, but hand-writing end-to-end tests is expensive and generic recorders produce brittle output. Raw click streams, unstable CSS selectors, exact-text assertions, and timing-based playback produce tests that quickly fail after routine UI changes. - -The platform shall solve this by treating recording as a data capture step and test generation as a compilation step. It shall infer user intent, promote resilient selectors, classify dynamic data, and generate maintainable Playwright tests that align with how users experience the application. - -### 2.2 Product Goal - -Enable product owners, testers, developers, and analysts to record business workflows once, refine them in a guided UI, and produce resilient C# Playwright tests that can be replayed, diagnosed, and healed as applications evolve. - -### 2.3 Product Principles - -The product shall be designed around these principles: - -1. Semantic first. -The system shall prefer user-facing meaning such as roles, labels, accessible names, nearby headings, and business outcomes over raw DOM structure. - -2. Structured source of truth. -The scenario model shall be the canonical artefact from which tests, page objects, diagnostics, and healing recommendations are derived. - -3. Deterministic before intelligent. -The system shall rely on deterministic heuristics and Playwright-native behaviour first. AI-assisted suggestions may augment the system, but must not be the only execution path for runtime-critical flows. - -4. User review is part of the workflow. -The platform shall support manual review and editing before generated code is considered ready. - -5. Diagnostics are mandatory. -Every recording, generation, replay, and healing attempt shall preserve sufficient evidence for supportability and debugging. - -## 3. Scope - -### 3.1 In Scope - -The v1 platform shall support: - -- Guided browser session recording -- DOM and accessibility metadata capture -- Playwright-based environment observation -- Post-recording scenario review and editing -- Locator ranking and locator candidate persistence -- Assertion suggestion and manual approval -- Variable data extraction and classification -- C# Playwright code generation -- Generated helper library emission -- Replay execution and diagnostics -- Deterministic selector healing workflows -- Persistent storage of sessions, scenarios, steps, artefacts, and replay history -- Multi-user authenticated web administration experience through Blazor Server -- Export of generated tests into repository-friendly file structures - -### 3.2 Out of Scope for v1 - -The following are explicitly out of scope unless later prioritised: - -- Full low-code test authoring without any review step -- Support for native mobile app automation -- AI-only locator resolution at execution time -- Canvas-heavy, game-like, or non-DOM-centric application automation as a first-class supported scenario -- Rich collaborative review workflows such as branching, inline comments, or merge conflict handling between multiple editors -- Fine-grained enterprise RBAC beyond foundational role separation -- General-purpose workflow orchestration unrelated to browser test mining -- Electron or .NET MAUI desktop packaging in the initial release - -### 3.3 Deferred but Anticipated Scope - -The architecture shall allow later addition of: - -- Page object refactoring generation -- CI/CD export packs and pipeline templates -- Browser extension-based capture -- AI-assisted scenario summarisation and healing proposals -- Team collaboration and scenario versioning -- Desktop packaging for offline-friendly operator experiences - -## 4. Target Users and Primary Use Cases - -### 4.1 User Roles - -The system shall recognise these primary personas: - -1. Test Author -Records workflows, reviews steps, approves assertions, and generates tests. - -2. Developer -Consumes generated code, integrates it into solution structures, and refines helpers or templates. - -3. QA Engineer -Validates workflow coverage, replay reliability, and failure diagnostics. - -4. Platform Administrator -Configures browser runners, storage, security, code-generation defaults, and environment connections. - -### 4.2 Core Use Cases - -The platform shall support these end-to-end use cases: - -1. Record a new business workflow from a live application. -2. Review and edit the recorded timeline before generation. -3. Mark inputs as fixed, variable, generated, lookup-driven, or sensitive. -4. Accept or reject suggested assertions. -5. Generate a readable Playwright C# test suite and helper files. -6. Replay the generated scenario against the target application. -7. Investigate replay failures using screenshots, traces, DOM context, and locator diagnostics. -8. Accept a healing suggestion and regenerate the derived code. - -## 5. High-Level System Overview - -### 5.1 Product Capabilities - -The application shall consist of these major capability areas: - -1. Recording -Browser session launch, event capture, instrumentation injection, and artefact creation. - -2. Interpretation -DOM intelligence, locator ranking, variable classification, widget recognition, and assertion inference. - -3. Scenario Authoring -Timeline editing, step suppression, annotation, data strategy configuration, and acceptance of inferred outputs. - -4. Generation -Transformation of scenario models into C# Playwright tests, helpers, fixtures, and optional page object scaffolding. - -5. Replay and Healing -Execution, diagnostics, candidate re-resolution, deterministic healing, and controlled regeneration. - -6. Administration -Environment configuration, browser configuration, template settings, project export settings, and user management. - -### 5.2 Logical Architecture - -The solution shall be decomposed into the following logical components: - -1. Blazor Server UI -- Hosts recording workflows, timeline editing, generation preview, replay diagnostics, and admin surfaces. - -2. ASP.NET Core Application Host -- Provides HTTP endpoints, SignalR backplane for live session updates, background workers, authentication, orchestration, and API surfaces for the UI. - -3. Recording Engine -- Launches Playwright browsers and coordinates injected capture scripts, event streams, screenshots, traces, and browser-side metadata collection. - -4. Capture and Analysis Engine -- Normalises raw browser events, enriches them with DOM context, identifies semantic widgets, ranks locators, and produces assertion candidates. - -5. Scenario Engine -- Stores, edits, versions, validates, and materialises structured scenarios. - -6. Code Generation Engine -- Produces C# Playwright test source files, helper classes, optional page objects, and configuration assets using Roslyn or Scriban. - -7. Replay and Healing Engine -- Replays scenarios or generated tests, captures outcomes, diagnoses failures, evaluates fallback locators, and records healing proposals. - -8. Persistence Layer -- Stores structured entities, audit history, artefact references, configuration, and operational logs in PostgreSQL or SQL Server. - -9. Artefact Storage -- Stores screenshots, traces, exported code bundles, DOM snapshots, and replay attachments on a filesystem or object-backed abstraction. - -## 5.3 Operational Lifecycles - -### 5.3.1 Recording Lifecycle - -The recording lifecycle shall follow this logical sequence: - -1. User creates a recording session with target URL and settings. -2. The backend provisions a Playwright browser context and applies configured bootstrap settings. -3. Recorder instrumentation is injected before or at page startup. -4. The user performs actions in the target application. -5. Raw browser events and Playwright observations are streamed to the backend. -6. The backend normalises events and persists session artefacts incrementally. -7. The user stops recording. -8. The backend finalises the session and generates an initial scenario draft for review. - -### 5.3.2 Scenario Authoring Lifecycle - -The authoring lifecycle shall support: - -1. Reviewing the recorded step timeline. -2. Removing or suppressing noise. -3. Promoting meaningful events into semantic actions. -4. Approving or rejecting inferred assertions. -5. Classifying test data and secrets. -6. Choosing generation profile settings. -7. Saving an immutable scenario version ready for generation or replay. - -### 5.3.3 Generation Lifecycle - -The generation lifecycle shall support: - -1. Selecting an approved scenario version. -2. Applying generation profile and template settings. -3. Producing output files and a manifest. -4. Storing a checksum and file inventory. -5. Presenting the generated output for preview and export. - -### 5.3.4 Replay and Healing Lifecycle - -The replay and healing lifecycle shall support: - -1. Selecting a scenario version or generated artefact for execution. -2. Launching an isolated Playwright replay context. -3. Executing steps while recording diagnostics. -4. Classifying failures when they occur. -5. Attempting deterministic healing when drift is suspected. -6. Presenting a proposal with supporting evidence. -7. Applying approved healing changes to a new scenario version or updating approved locator preferences under controlled rules. - -## 6. Mandatory Technology Requirements - -### 6.1 Application Stack - -The implementation shall use: - -- ASP.NET Core for backend hosting and service orchestration -- Blazor Server for the primary web UI -- Microsoft.Playwright for .NET for browser control, observation, replay, trace capture, and browser-context management -- EF Core for relational persistence -- PostgreSQL or SQL Server as the durable database -- Roslyn and/or Scriban for code generation - -### 6.2 Database Provider Strategy - -To make v1 deliverable without doubling persistence cost, the platform shall adopt the following strategy: - -- PostgreSQL shall be the **primary and only required provider for v1**. Local development, shared non-production, and production-like deployments shall all use PostgreSQL unless explicitly reconfigured. -- SQL Server shall be a **deferred-but-supported secondary provider**. The persistence layer shall be designed so that SQL Server can be added later without redesigning the domain or repositories, but v1 acceptance does not require a working SQL Server deployment. -- The persistence layer shall use EF Core abstractions and migrations. -- Provider-specific SQL shall be avoided unless isolated behind clearly bounded infrastructure components and gated by a provider capability check. -- Data types, indexing strategies, JSON storage, and concurrency rules shall be designed to remain portable to SQL Server. Use of PostgreSQL-only features (for example `jsonb` operators, array columns, partial indexes) is permitted in v1 provided the feature can be substituted or approximated on SQL Server later. -- Integration tests shall run against real PostgreSQL (for example Testcontainers) rather than an in-memory provider, because in-memory providers do not exercise migration or JSON behaviour. - -### 6.3 Browser Support - -The initial supported execution browser shall be Chromium through Playwright. The architecture shall not prevent later support for WebKit and Firefox, but those browsers are not required for v1 unless explicitly enabled by configuration. - -## 7. Information Architecture and Domain Model - -### 7.1 Canonical Principle - -The canonical persisted business artefact shall be `Scenario`. Generated tests, replay runs, and healing decisions shall reference the scenario version they were derived from. - -### 7.2 Core Entities - -The persistence model shall include, at minimum, these entities. - -#### 7.2.0 Common Entity Requirements - -Unless explicitly exempted, every persistent entity defined in this section shall carry: - -- A stable primary key (`Id`) that is globally unique (GUID/UUID) and assigned by the application, not the database. -- Audit fields: `CreatedAtUtc`, `CreatedByUserId`, `UpdatedAtUtc`, `UpdatedByUserId` (the last two may be nullable where the entity is immutable by design, such as `ScenarioVersion` and `RecordedStep`). -- A `RowVersion` (optimistic concurrency token) column on entities that are editable by more than one workflow, including `Scenario`, `RecordingSession`, `HealingSuggestion`, and administrative configuration entities. -- A soft-delete marker (`IsDeleted`, `DeletedAtUtc`, `DeletedByUserId`) on `Scenario`, `RecordingSession`, `GenerationArtifact`, and artefact records. Immutable versions (e.g. `ScenarioVersion`, `RecordedStep`) shall not support soft delete; deletion of a scenario shall be a cascade-tombstoning operation documented in the data model. -- Timestamps stored in UTC; the domain shall not depend on database-local clocks for ordering decisions beyond audit. - -Field lists in the subsections below name the entity-specific fields and may omit the common fields above for brevity. - -#### 7.2.1 RecordingSession - -Represents an interactive capture session started by a user. - -Required fields: - -- `Id` -- `Name` -- `TargetBaseUrl` -- `EnvironmentName` -- `Status` -- `StartedByUserId` -- `StartedAtUtc` -- `EndedAtUtc` -- `BrowserKind` -- `BrowserChannel` -- `Headless` -- `AuthenticationMode` -- `RecordAssertionsAutomatically` -- `MaskedFieldPolicyJson` -- `SettingsJson` - -#### 7.2.2 Scenario - -Represents a business workflow compiled from one or more recording sessions and maintained as the product source of truth. - -Required fields: - -- `Id` -- `Name` -- `Description` -- `Status` -- `PrimaryRecordingSessionId` -- `TargetApplicationName` -- `TargetBaseUrl` -- `EnvironmentName` -- `CreatedByUserId` -- `CreatedAtUtc` -- `UpdatedAtUtc` -- `CurrentVersionNumber` -- `DefaultGenerationProfileId` -- `TagsJson` - -#### 7.2.3 ScenarioVersion - -Represents an immutable version of the scenario structure used for generation or replay traceability. - -In-progress edits shall not mutate an immutable `ScenarioVersion`. Any draft editing workflow shall -store draft state separately and create a new immutable scenario version when changes are committed. - -Required fields: - -- `Id` -- `ScenarioId` -- `VersionNumber` -- `ParentScenarioVersionId` -- `CreatedAtUtc` -- `CreatedByUserId` -- `ChangeSummary` -- `StructureJson` -- `InferenceMetadataJson` -- `ApprovedForGeneration` - -#### 7.2.4 RecordedStep - -Represents a normalised action or assertion candidate in sequence order. - -Required fields: - -- `Id` -- `ScenarioVersionId` -- `Sequence` -- `StepType` -- `SemanticActionType` -- `DisplayName` -- `UserNote` -- `Status` -- `UrlBefore` -- `UrlAfter` -- `TimestampUtc` -- `DurationMs` -- `RawValue` -- `NormalisedValue` -- `VariableStrategy` -- `SensitivityClassification` -- `TargetElementSnapshotId` -- `ParentContainerSnapshotId` -- `ExpectedOutcomeJson` -- `GeneratedCodeHintJson` - -#### 7.2.5 ElementSnapshot - -Represents the known properties of an element at record or replay time. - -Required fields: - -- `Id` -- `CapturePhase` -- `RecordingSessionId` (nullable; populated for recording-time capture) -- `ReplayRunId` (nullable; populated for replay-time capture) -- `RecordedStepId` -- `TagName` -- `InputType` -- `Role` -- `AccessibleName` -- `LabelText` -- `Placeholder` -- `TextContent` -- `TitleAttribute` -- `NameAttribute` -- `ValuePreview` -- `TestId` -- `AriaAttributesJson` -- `DataAttributesJson` -- `CssPath` -- `XPath` -- `BoundingBoxJson` -- `VisibilityState` -- `EnabledState` -- `ParentContextJson` -- `SiblingContextJson` -- `WidgetKind` -- `DomHash` - -#### 7.2.6 LocatorCandidate - -Represents one potential locator strategy for a target element. - -Required fields: - -- `Id` -- `RecordedStepId` -- `StrategyType` -- `Expression` -- `ScopeExpression` -- `Priority` -- `Confidence` -- `UniquenessAtRecordTime` -- `VisibilityAtRecordTime` -- `Recommended` -- `FailureReasonIfRejected` - -#### 7.2.7 AssertionCandidate - -Represents a suggested or approved expected outcome. - -Required fields: - -- `Id` -- `RecordedStepId` -- `AssertionType` -- `ExpressionJson` -- `Priority` -- `Confidence` -- `SourceSignal` -- `Approved` -- `RejectedReason` - -#### 7.2.8 ReplayRun - -Represents one execution attempt of a scenario or generated test. - -Required fields: - -- `Id` -- `ScenarioId` -- `ScenarioVersionId` -- `ReplaySourceKind` -- `GenerationArtifactId` (nullable; populated when replay uses a generated artefact bundle) -- `Status` -- `StartedAtUtc` -- `EndedAtUtc` -- `TriggeredByUserId` -- `ExecutionEnvironment` -- `BrowserKind` -- `Headless` -- `FailureCategory` -- `FailureSummary` -- `DiagnosticsJson` - -#### 7.2.9 HealingSuggestion - -Represents a proposed change when replay detects drift. - -Required fields: - -- `Id` -- `ReplayRunId` -- `RecordedStepId` -- `SuggestionType` -- `CurrentFailureEvidenceJson` -- `ProposedLocatorJson` -- `DeterministicConfidence` -- `AiConfidence` -- `ApprovalStatus` -- `AppliedAtUtc` -- `AppliedByUserId` - -#### 7.2.10 UserAccount - -Represents an authenticated principal of the platform. Identity data may be federated from an external provider, but the platform shall maintain a local record to support authorship attribution, audit trails, and role assignment. - -Required fields: - -- `Id` -- `ExternalSubjectId` (nullable; populated when federated) -- `DisplayName` -- `Email` -- `Role` (at minimum one of: `Administrator`, `Author`, `Viewer`) -- `IsDisabled` -- `LastLoginAtUtc` - -All audit references in other entities (for example `StartedByUserId`, `CreatedByUserId`, `TriggeredByUserId`) shall resolve to `UserAccount.Id`. - -#### 7.2.11 GenerationArtifact - -Represents a generated bundle or file set. - -Required fields: - -- `Id` -- `ScenarioVersionId` -- `GeneratorKind` -- `GeneratorVersion` -- `TemplateProfile` -- `TemplateVersion` -- `OutputRoot` -- `ManifestJson` -- `CreatedAtUtc` -- `CreatedByUserId` -- `Checksum` - -### 7.3 Structured Scenario Contract - -The application shall define a first-class structured scenario contract that can represent: - -- Navigation steps -- Click actions -- Fill actions -- Select actions -- Checkbox and toggle interactions -- Keyboard actions -- Dialog interactions -- Grid and list interactions -- Widget-scoped semantic actions -- Assertions -- Control flow metadata such as waits inferred from user-facing events -- Variable definitions -- Data generation strategies -- Replay and healing hints - -The scenario contract shall be expressive enough that: - -- Tests can be regenerated without rerecording. -- Helper libraries can evolve independently of the raw recording. -- Multiple code-generation styles can be supported later. -- Healing decisions can update the scenario model without directly editing generated output. - -## 7.4 Artefact Model - -The application shall treat the following as first-class artefacts: - -- Screenshots -- Playwright traces -- DOM snapshots -- Session logs -- Generated source files -- Replay diagnostics packages - -Each artefact shall have: - -- A stable identifier -- A logical owner such as recording session, scenario version, or replay run -- Content type -- Retention classification -- Storage path or provider-specific handle -- Created timestamp -- Optional checksum -- A size-in-bytes record - -The platform shall enforce default retention and size ceilings so artefact storage does not grow unbounded in the absence of administrative action: - -- Screenshots and DOM snapshots: retained for at least 30 days after the owning session or replay run completes; individual artefact size cap of 10 MB unless an administrator raises it. -- Playwright traces: retained for at least 14 days; individual trace cap of 200 MB. -- Session logs and replay diagnostic bundles: retained for at least 30 days. -- Generated source bundles: retained until the owning scenario is deleted. - -Administrators may extend any retention window or raise any size cap through the configuration surface defined in Section 11. A background cleanup process shall delete artefacts that have exceeded their retention window and record the deletion in the audit trail. - -## 7.5 State Model Expectations - -The system shall define explicit statuses for at least: - -- Recording sessions -- Scenarios -- Scenario versions -- Replay runs -- Healing suggestions -- Generation artefacts - -Status transitions shall be explicit, validated, and auditable. - -## 8. Functional Requirements - -### 8.1 Recording Requirements - -#### FR-REC-001 Guided Recording Creation - -The system shall let a user start a recording by providing: - -- Scenario name -- Target application URL -- Optional environment label -- Browser selection -- Whether automatic assertion suggestions are enabled -- Authentication or session bootstrap mode - -#### FR-REC-002 Browser Session Launch - -The system shall launch a Playwright browser context for recording with configurable: - -- Headless or headed mode -- Viewport -- Storage state -- Trace settings -- Video settings, if enabled -- Network capture settings - -#### FR-REC-003 Early Instrumentation Injection - -The system shall inject recorder logic before application content loads wherever technically possible, using Playwright-supported initialisation hooks such as `AddInitScriptAsync`. - -#### FR-REC-004 Event Capture - -The recorder shall capture, at minimum, these event types when user meaningful: - -- Page navigation -- Click -- Double click when distinguishable -- Input -- Change -- Form submit -- Key press -- Dialog open and close -- File download start -- Popup open -- Checkbox or toggle change -- Select and autocomplete selection - -The recorder shall suppress noise events such as incidental mouse movement unless explicitly enabled for diagnostics. - -#### FR-REC-005 Element Context Capture - -For each meaningful event, the system shall capture contextual element metadata including, where available: - -- HTML tag -- Input type -- Role -- Accessible name -- Label text -- Placeholder -- Visible text -- Title attribute -- Name attribute -- Test id or equivalent stable data attribute -- Bounding box -- Enabled and visible state -- Nearby heading -- Parent form, region, modal, or table context -- Candidate CSS path -- Candidate XPath as a last-resort fallback - -#### FR-REC-006 Observer Correlation - -The recording engine shall correlate user events with surrounding Playwright-observed browser state including: - -- URL changes -- Page load phases -- Network requests and responses -- Console errors and warnings, if enabled -- Dialog lifecycle -- Download lifecycle -- Popup lifecycle -- Screenshots -- Trace segments - -#### FR-REC-007 Sensitive Data Handling - -The user shall be able to mark fields or values as sensitive during or after recording. Sensitive values shall be masked in persisted logs, previews, and generated artefacts unless the user explicitly authorises retention in a secure variable form. - -#### FR-REC-008 Recording Controls - -The UI shall allow the user to: - -- Start -- Pause -- Resume -- Stop -- Cancel -- Mark current step as important -- Mark current step as ignorable -- Add an inline note - -#### FR-REC-009 Recorder Transport - -The recorder shall use a reliable browser-to-host transport mechanism for event delivery. Preferred mechanisms include Playwright-exposed bindings or a similarly explicit channel. Console scraping may be used only as a bounded fallback or diagnostic technique, not as the primary durable transport contract. - -#### FR-REC-010 Incremental Persistence - -Long-running recording sessions shall persist meaningful progress incrementally so that unexpected host failures do not necessarily lose the entire session. - -### 8.2 Post-Processing and Inference Requirements - -#### FR-INF-001 Event Normalisation - -The system shall transform raw captured events into a cleaned event stream by: - -- Removing technical noise -- Merging related input bursts -- Folding navigation waits into surrounding intent -- Linking actions to the most likely observed outcomes - -#### FR-INF-002 Semantic Action Inference - -The system shall infer higher-order action meaning where possible, including but not limited to: - -- Open details page -- Save form -- Submit search -- Open modal -- Confirm dialog -- Select row in grid -- Apply filter -- Choose date -- Dismiss notification - -#### FR-INF-003 Widget Recognition - -The system shall recognise common UI widget patterns where feasible, including: - -- Forms -- Tables and grids -- Modals and dialogs -- Dropdowns and combo boxes -- Tabs -- Toast notifications -- Date pickers -- Autocomplete controls - -The recognised widget type shall be stored as metadata usable by generation and healing logic. - -#### FR-INF-004 Locator Ranking - -For each targetable step, the system shall create a ranked set of locator candidates. The ranking algorithm shall prioritise: - -1. Explicit test id or configured stable data attribute -2. Role plus accessible name -3. Label-based targeting -4. Visible text -5. Placeholder -6. Scoped parent-region plus child-role targeting -7. Stable attribute combinations -8. CSS selector fallback -9. XPath fallback - -#### FR-INF-005 Locator Validation - -Each locator candidate shall be validated at record time when possible for uniqueness, visibility, and scope confidence. - -#### FR-INF-006 Variable Classification - -The system shall support classifying entered or observed values into at least these categories: - -- Fixed test data -- Scenario variable -- Generated data -- Lookup value -- Sensitive value -- Incidental value that should not become an assertion target - -#### FR-INF-007 Assertion Suggestion - -The system shall infer candidate assertions from post-action signals including: - -- Success toast visibility -- URL pattern change -- Heading change -- Dialog closure -- Newly visible row, card, or detail view -- Relevant network success response -- Count changes where meaningful - -#### FR-INF-008 Fuzzy Outcome Strategies - -The system shall support resilience strategies for data-sensitive assertions including: - -- Exact text to contains matching -- Regex matching for identifiers -- Date and time range validation -- Row existence by key column -- Partial heading matching -- Minimum count assertions - -#### FR-INF-009 Confidence Scoring - -The system shall assign confidence scores to inferred locators, assertions, widget classifications, and healing suggestions. Confidence values shall be explainable and based on deterministic evidence where possible. - -### 8.3 Scenario Authoring Requirements - -#### FR-AUTH-001 Timeline Review - -The UI shall present recorded scenarios as an ordered timeline of meaningful steps with human-readable labels. - -#### FR-AUTH-002 Step Editing - -The user shall be able to: - -- Delete a step -- Suppress a step from generation -- Reorder steps where valid -- Convert a step into an assertion -- Change a step to use a different locator candidate -- Add notes and rationale -- Mark a step as variable-driven -- Mark a step as sensitive - -#### FR-AUTH-003 Assertion Approval - -Suggested assertions shall not be treated as final until approved by the user or by a configured auto-approval rule. - -#### FR-AUTH-004 Data Strategy Editing - -The author shall be able to review every captured value and choose whether it becomes: - -- A literal -- A scenario parameter -- A generated value -- A fixture reference -- A secret -- Ignored data - -#### FR-AUTH-005 Scenario Validation - -Before generation, the system shall validate that the scenario is structurally coherent, including: - -- Ordered step sequence -- Supported step types -- Required locator presence -- Required variable definitions -- No unresolved sensitive-data policy violations - -#### FR-AUTH-006 Version Creation - -Saving a material scenario change after review shall create a new immutable scenario version. -If the product supports draft editing, the draft state shall be stored separately from immutable -scenario versions until the user commits the change. The application shall preserve enough history -to identify what changed between versions. - -### 8.4 Code Generation Requirements - -#### FR-GEN-001 Derived Output Principle - -Generated source files shall always be reproducible from scenario data and template settings. Manual edits to generated code shall not silently become the source of truth. - -#### FR-GEN-002 Supported Output - -The v1 system shall generate: - -- C# Playwright test classes -- Shared locator-resolution helpers -- Optional data model or fixture classes -- Supporting test project files or export manifests where needed - -#### FR-GEN-003 Readability - -Generated code shall prioritise readability for human developers. It shall: - -- Use clear method names -- Avoid unreadable deeply nested locator code where helper abstraction is appropriate -- Separate data setup from action steps -- Use Playwright web-first assertions rather than fixed sleeps - -#### FR-GEN-004 LocatorResolver Helper - -The generated runtime helper library shall include a locator-resolution abstraction capable of: - -- Trying candidates in ranked order -- Validating uniqueness before acting -- Optionally scoping search to a parent container -- Emitting useful diagnostics when no candidate resolves uniquely - -#### FR-GEN-005 Assertion Generation - -Generated assertions shall prefer user-facing outcome checks over implementation-detail checks. - -#### FR-GEN-006 Variable and Fixture Generation - -When configured, the generator shall emit strongly typed classes or structures representing scenario variables, generated data, and fixture data. - -#### FR-GEN-007 Generation Profiles - -The system shall support generation profiles so teams can choose between styles such as: - -- Flat test methods -- Test plus helper methods -- Test plus page object scaffolding - -#### FR-GEN-008 Template Technology - -The implementation may use Roslyn, Scriban, or a hybrid approach, but the chosen approach shall support: - -- Deterministic output -- Repeatable formatting -- Testable generation logic -- Easy future extension - -#### FR-GEN-009 Repository-Friendly Output - -The generation system shall support output layouts that are easy to place under source control, including predictable file names, stable folder structure, and manifests that identify which files belong to each generation run. - -#### FR-GEN-010 Runtime Helper Compatibility - -Generated helper libraries shall remain versioned and compatible with the generator profile that emitted them. Breaking helper changes shall be traceable through generation manifests or template version metadata. - -### 8.5 Replay Requirements - -#### FR-REP-001 Scenario Replay - -The system shall replay a scenario using Playwright and the currently approved scenario version or the chosen generated artefact. - -#### FR-REP-002 Replay Diagnostics - -Each replay run shall capture: - -- Start and end timestamps -- Browser and environment settings -- Screenshots at key failure points -- Playwright traces when enabled -- Step-level pass or fail state -- Locator resolution attempts -- Assertion evaluation results -- Failure categorisation - -#### FR-REP-003 Failure Categories - -Replay failures shall be classified into categories including: - -- Locator resolution failure -- Timeout -- Unexpected navigation -- Assertion failure -- Authentication failure -- Environment availability failure -- Script or generation bug - -#### FR-REP-004 Targeted Rerun - -The UI shall allow rerunning: - -- The full scenario -- The current step onward -- A single step for diagnostic purposes where valid - -#### FR-REP-005 Replay Isolation - -Replay runs shall execute in isolated browser contexts and shall not share mutable session state unless explicitly configured for a trusted use case. - -### 8.6 Healing Requirements - -#### FR-HEAL-001 Deterministic Healing First - -When replay fails due to locator drift, the system shall first attempt deterministic recovery using: - -- Alternate persisted locator candidates -- Scoped searches -- Fuzzy accessible-name or text matching -- Parent context anchoring -- Nearby heading anchoring - -#### FR-HEAL-002 Healing Evidence - -Every healing proposal shall include evidence showing: - -- The original locator strategy -- Why it failed -- The proposed replacement -- What contextual signals supported the proposal -- Confidence scores - -#### FR-HEAL-003 Human Approval - -In v1, every healing change that alters the persisted scenario shall require explicit human approval. Automatic application of healing proposals to persisted scenarios shall not ship in v1. - -A controlled auto-apply policy for deterministic high-confidence matches may be introduced in a later phase, subject to all of the following: - -- The feature shall be disabled by default. -- Auto-apply shall be gated behind an administrator-only setting scoped per environment (never globally by default). -- Auto-apply shall be restricted to `SuggestionType` values classified as locator-only, deterministic, and non-destructive. Changes to assertion semantics, step ordering, variable classification, or sensitive-data policy shall never be auto-applied. -- A dry-run mode shall be available that records what would have been applied without mutating the scenario. -- Every auto-applied change shall still create a new immutable `ScenarioVersion` with a traceable `HealingSuggestion` link. - -#### FR-HEAL-004 AI as Adviser - -AI-assisted healing, if enabled later, shall be advisory and reviewable. It shall not bypass deterministic diagnostics or obscure the basis for a proposal. - -#### FR-HEAL-005 Healing Traceability - -When a healing proposal is approved and applied, the system shall retain a clear audit record linking the replay failure, the proposal, the approver, and the resulting scenario or locator change. - -### 8.7 Administration Requirements - -#### FR-ADM-001 Environment Configuration - -Administrators shall be able to configure: - -- Application environments -- Browser execution settings -- Base URLs -- Storage settings -- Database provider and connection -- Artefact retention rules -- Code-generation defaults - -#### FR-ADM-002 Authentication Bootstrap Options - -The system shall support configurable authentication bootstrap approaches such as: - -- Manual login during recording -- Stored Playwright storage state -- Cookie import under controlled administration - -#### FR-ADM-003 Auditability - -Administrative changes shall be auditable with who, when, and what changed. - -#### FR-ADM-004 Retention Policies - -Administrators shall be able to configure retention policies for logs, screenshots, traces, generated artefacts, and stale replay diagnostics. - -## 9. User Interface Requirements - -### 9.1 Overall UI Structure - -The Blazor Server application shall provide at least these primary work areas: - -1. Recording workspace -2. Timeline and scenario editor -3. Generated code preview -4. Replay and diagnostics workspace -5. Administration area - -### 9.2 Recording Workspace - -The recording workspace shall display: - -- Browser session status -- Start, pause, resume, and stop controls -- Current URL -- Live step feed -- Ability to flag importance or ignore noise -- Inline note entry - -### 9.3 Timeline Workspace - -The timeline workspace shall provide: - -- Ordered list of steps -- Step type badges -- Locator confidence indicators -- Assertion suggestion indicators -- Variable classification editing -- Reorder and suppress controls - -### 9.4 Code Preview Workspace - -The code preview workspace shall provide: - -- Generated C# preview -- File list or manifest view -- Warnings for low-confidence locators or assertions -- Regenerate action - -### 9.5 Replay Workspace - -The replay workspace shall provide: - -- Replay status -- Step-by-step result stream -- Failure summary -- Locator diagnostics -- Screenshot and trace links -- Healing proposal review and approval actions - -### 9.6 Responsiveness - -The application shall be usable on standard desktop browser sizes and shall degrade gracefully on smaller widths, but desktop-first layout is acceptable for v1. - -## 10. API and Integration Requirements - -### 10.1 Internal API Surface - -The ASP.NET Core backend shall expose internal or authenticated HTTP APIs sufficient to support: - -- Recording lifecycle control -- Scenario CRUD -- Timeline editing -- Replay triggering -- Healing approval -- Artefact retrieval -- Administrative configuration - -### 10.2 Real-Time Updates - -The application shall provide real-time UI updates for long-running recording and replay operations. SignalR is the preferred approach. - -### 10.3 Artefact Export - -The system shall support export of generated assets into a chosen output directory structure that is suitable for committing to a repository or copying into another solution. - -### 10.4 External Target Application Interaction - -The platform shall treat the target application as an external system under test. It shall not require modifications to the target application, but it shall provide guidance when better accessibility or testability hooks would improve recording quality. - -### 10.5 API Design Expectations - -Backend APIs should: - -- Use explicit resource identifiers -- Return actionable validation errors -- Support long-running operation status retrieval -- Avoid leaking secrets or raw sensitive payloads -- Be versionable from the outset - -## 11. Configuration Requirements - -### 11.1 Configuration Sources - -The application shall support configuration from: - -- App settings files -- Environment variables -- Secure secret providers or equivalent deployment-time injection -- Database-backed administrative settings where appropriate - -### 11.2 Configurable Areas - -The following areas shall be configurable: - -- Database provider and connection string -- Artefact storage root or provider -- Browser launch defaults -- Headless defaults -- Base URL allowlists -- Session timeout and retention policies -- Masking rules -- Default generation profile -- Replay diagnostics level -- Authentication provider settings - -### 11.3 Validation - -Invalid required configuration shall fail fast at application startup or at the point of administrative save, depending on whether the setting is environment-level or runtime-editable. - -## 12. Security and Privacy Requirements - -### 12.1 Authentication and Access Control - -The platform shall require authenticated access for non-public environments. Baseline role separation shall distinguish at least: - -- Administrator -- Author -- Viewer - -### 12.2 Secret Protection - -Secrets shall never be written to plain logs, screenshots metadata, or generated code by default. This includes: - -- Target application credentials -- Session cookies -- Tokens -- Secret variable values - -### 12.3 Sensitive Data Masking - -The system shall allow masking rules by field name, selector metadata, or manual marking. Masked values shall remain masked in: - -- UI previews -- Diagnostic logs -- Stored event payloads -- Generation previews - -### 12.4 Target URL Allow-List Enforcement - -The platform shall enforce an administrator-managed allow-list of target base URLs. A recording or replay session shall fail to start if its requested target URL does not match an entry on the allow-list for the selected environment. The allow-list check shall be performed server-side before any Playwright browser context is launched. The allow-list shall be auditable and changes shall be captured in the audit trail defined in Section 12.7. - -### 12.5 Encryption of Captured Session Material - -The platform captures material that is sensitive by construction: authentication cookies, Playwright storage states, raw event payloads containing user input, and user-authored scenario variables marked sensitive. The following protections shall apply: - -- Playwright storage states, cookies, and any cached authentication material shall be encrypted at rest using platform-managed keys sourced from a secure secret provider. -- Scenario variables classified as `Sensitive` shall be encrypted at rest. Their plaintext shall never be written to generated source, generation previews, logs, or UI previews. -- Raw event payloads shall have sensitive fields masked or encrypted at rest according to the `MaskedFieldPolicyJson` in force at recording time. A field whose classification is changed to sensitive after recording shall be retroactively masked in subsequent previews and re-generation. -- Key material shall not be checked into source control. Key rotation shall be supported at least by re-encrypting affected records during a controlled administrative operation. - -### 12.6 Browser Session Isolation - -Each recording or replay session shall execute in an isolated Playwright browser context unless explicitly configured otherwise by an administrator. - -### 12.7 Audit Logging - -The system shall maintain an audit trail for: - -- Login and administrative access -- Scenario creation and approval -- Healing approval -- Artefact export -- Configuration changes - -## 13. Observability and Diagnostics Requirements - -### 13.1 Structured Logging - -The backend shall emit structured logs with identifiers such as: - -- Recording session id -- Scenario id -- Scenario version id -- Replay run id -- User id -- Browser context id - -### 13.2 Operational Metrics - -The system shall capture metrics useful for support and scaling, including: - -- Active recordings -- Active replays -- Average recording duration -- Replay success rate -- Most common failure categories -- Healing proposal frequency -- Generation duration - -### 13.3 Artefact Traceability - -Screenshots, traces, DOM snapshots, and generated outputs shall remain traceable to the session, scenario version, or replay run that created them. - -## 14. Verification and Testing Requirements - -### 14.1 Product Test Strategy - -The platform implementation shall include automated coverage for: - -- Domain and inference rules -- Scenario validation -- Locator ranking -- Assertion suggestion rules -- Code-generation output shape -- Replay orchestration -- Healing evaluation -- Persistence mappings and migrations - -### 14.2 Test Layers - -At minimum, the repository should include: - -- Unit tests for core business rules and model transformations -- Integration tests for Playwright coordination, persistence, and artefact storage -- End-to-end tests against controlled sample web applications - -### 14.3 Regression Fixtures - -The implementation should maintain representative sample applications or fixture pages that cover: - -- Basic forms -- Grid interactions -- Modal dialogs -- Login flows -- Dynamic identifiers -- Toast-based success notifications - -### 14.4 Generator Verification - -Generated output shall be verified through snapshot-style approval tests or equivalent structural tests so template changes do not unintentionally degrade code readability or behaviour. - -## 15. Non-Functional Requirements - -### 15.1 Reliability - -The platform shall favour deterministic execution. Generated tests and replay flows shall use Playwright waiting behaviour and retryable assertions rather than fixed delays except where no better alternative exists. - -### 15.2 Performance - -The system shall feel responsive for interactive authoring. Initial targets, measured on a reference developer workstation (quad-core modern CPU, SSD, local PostgreSQL), expressed as percentiles over a rolling sample of at least 20 operations of the same type: - -- Time from "Start recording" click to Playwright page navigation ready: p50 ≤ 5 s, p95 ≤ 10 s. -- End-to-end latency from a captured browser event to its corresponding step appearing in the timeline: p50 ≤ 750 ms, p95 ≤ 2 s. -- Code generation for a scenario of up to 100 meaningful steps: p50 ≤ 8 s, p95 ≤ 15 s. -- Replay start-up overhead (time from replay trigger to first step execution) shall not exceed 10 s at p95. - -These are product-quality targets, not contractual SLAs. Observability required under Section 13 shall be sufficient to verify them post-deployment. - -### 15.3 Scalability - -The v1 system shall support multiple concurrent users and background replays on a single server instance. The architecture shall allow future scale-out of non-UI background execution services if demand grows. - -### 15.4 Maintainability - -The implementation shall keep clear boundaries between: - -- UI concerns -- Application services -- Playwright execution -- Scenario/inference logic -- Persistence -- Generation -- Healing - -### 15.5 Testability - -The system itself shall be designed for automated testing using unit, integration, and end-to-end coverage. The product shall not rely on untestable static global state for core flows. - -### 15.6 Accessibility - -Because the product explicitly depends on accessibility metadata from target applications, the platform UI should also model good practice. The Blazor UI should use accessible semantics, keyboard support, and meaningful announcements for long-running operations where practical. - -## 16. Deployment and Operational Requirements - -### 16.1 Host Model - -The initial application shall run as an ASP.NET Core server-hosted web application with Blazor Server integrated into the host process or solution boundary. - -### 16.2 Environment Profiles - -The implementation shall support at least: - -- Local developer execution -- Shared non-production environment -- Production-like server deployment - -### 16.3 Packaging Direction - -Desktop packaging through .NET MAUI Hybrid or Electron is explicitly a future option. The initial architecture shall therefore keep the backend and UI boundaries clean enough that later packaging can host the same application surfaces without re-implementing core recording, generation, or replay services. - -### 16.4 Operational Safety - -Operational processes shall account for: - -- Browser process cleanup -- Expired artefact cleanup -- Failed replay recovery -- Application restarts during long-running operations - -## 17. Suggested Repository Architecture - -The application should be introduced into this repository as a new vertical slice rather than by mutating the existing Symphony specification. A suggested project layout is: - -```text -/src - /TestMining.Platform.Host (ASP.NET Core host, Blazor Server UI, auth, APIs) - /TestMining.Platform.Core (domain models, interfaces, scenario contracts) - /TestMining.Platform.Recording (Playwright launch, injected recorder, capture coordination) - /TestMining.Platform.Analysis (DOM intelligence, inference, locator ranking, assertions) - /TestMining.Platform.Generation (Roslyn/Scriban generation) - /TestMining.Platform.Replay (replay execution and diagnostics) - /TestMining.Platform.Healing (deterministic healing services) - /TestMining.Platform.Persistence (EF Core, PostgreSQL/SQL Server adapters, repositories) - /TestMining.Platform.Artefacts (artefact storage abstraction) -/tests - /TestMining.Platform.Core.Tests - /TestMining.Platform.Integration.Tests - /TestMining.Platform.E2E.Tests -/docs - concept.md - requirements.md -``` - -## 18. Delivery Phasing - -### 18.1 Phase 1: Core Proof - -Phase 1 shall target: - -- Recording of navigation, click, fill, select, checkbox, and simple assertion-relevant actions -- Timeline UI -- Locator ranking -- Basic scenario persistence with immutable scenario version creation -- C# Playwright generation -- Basic replay - -### 18.2 Phase 2: Robustness - -Phase 2 shall add: - -- Assertion inference -- Variable extraction -- Regex and partial matching -- Scoped locators -- Replay diagnostics -- Screenshot capture per important step - -### 18.3 Phase 3: Product Hardening - -Phase 3 shall add: - -- Login and session bootstrap management -- Export packaging -- Trace viewer integration -- Advanced healing review workflow and approval ergonomics -- Scenario history comparison and version-diff views - -### 18.4 Phase 4: AI Assistance - -Phase 4 may add: - -- Scenario naming suggestions -- Assertion suggestion assistance -- Healing proposal assistance -- Page object refactoring suggestions - -### 18.5 Phase Exit Criteria - -A phase shall not be declared complete until the criteria below are demonstrably met. These are conformance gates, not aspirational targets. - -Phase 1 exit: -- A user can authenticate, record a simple CRUD workflow end-to-end against a fixture web application, and see the persisted scenario. -- Locator candidates are ranked and persisted for every targetable step. -- Generated C# Playwright output compiles against the emitted helper library. -- Replay executes the generated scenario against the same fixture and reports pass/fail per step. -- URL allow-list enforcement (Section 12.4) and encryption of storage state (Section 12.5) are enforced. - -Phase 2 exit: -- Assertion inference produces at least one outcome-oriented assertion suggestion for every save/submit/navigate step in the Phase 1 fixture library. -- Variable classification is editable in the UI and round-trips through regeneration without loss. -- Replay diagnostics bundles include screenshots at each failure, locator resolution attempts, and a failure category. - -Phase 3 exit: -- Login bootstrap options (Section FR-ADM-002) are demonstrable end-to-end. -- Healing review workflow can approve a deterministic locator healing proposal and create a new immutable scenario version referencing the originating replay run. -- Artefact retention cleanup (Section 7.4) runs on schedule and is observable. - -Phase 4 exit (if undertaken): -- Any AI-assisted suggestion is always advisory, never auto-applied, and is labelled as AI-sourced in the UI and audit trail. -- Deterministic diagnostics continue to run and are presented alongside any AI suggestion. - -## 19. Acceptance Criteria - -The implementation shall be considered to satisfy this specification only when all of the following are demonstrably true and covered by at least one automated test traceable to the cited requirement identifiers. - -1. A user can record a real browser workflow through the UI and the system persists a structured scenario. [FR-REC-001, FR-REC-004, FR-REC-010] -2. The scenario can be reviewed, edited, and approved without direct database manipulation. [FR-AUTH-001 through FR-AUTH-006] -3. The system generates readable C# Playwright artefacts from the approved scenario. [FR-GEN-001 through FR-GEN-010] -4. The generated or scenario-derived replay executes through Playwright and produces useful diagnostics. [FR-REP-001, FR-REP-002, FR-REP-003] -5. Locator candidates are ranked and persisted rather than reduced to a single opaque selector. [FR-INF-004, FR-INF-005] -6. Sensitive values can be masked and remain masked across previews and logs, and are encrypted at rest where stored. [FR-REC-007, 12.3, 12.5] -7. Assertions are outcome-oriented and can be approved or rejected before generation. [FR-INF-007, FR-AUTH-003] -8. A locator drift failure can produce a deterministic healing proposal with evidence and require human approval before altering a scenario in v1. [FR-HEAL-001 through FR-HEAL-005] -9. The application runs on ASP.NET Core with a Blazor Server frontend and a PostgreSQL-backed persistence layer, with SQL Server pluggability preserved as a deferred option. [6.1, 6.2] -10. Generated output remains reproducible bit-for-bit (modulo timestamps declared as non-deterministic) from scenario data plus generation profile plus template version. [FR-GEN-001, FR-GEN-010] -11. Recording and replay refuse to start against target URLs not present on the administrator-managed allow-list. [12.4] -12. Observability emits the identifiers listed in 13.1 and permits verification of the performance targets in 15.2. - -## 20. Risks and Constraints - -### 20.1 Known Challenging Target Apps - -The product shall acknowledge reduced reliability for: - -- Canvas-heavy applications -- Highly virtualised lists or grids -- Poorly accessible React or SPA interfaces -- UIs with unstable text and random generated IDs -- Drag-and-drop heavy workflows -- Rich text editors - -### 20.2 Product Response to Poor Testability - -When recording quality is degraded by target-application design, the platform should surface actionable recommendations such as: - -- Add stable `data-testid` attributes -- Improve ARIA labels -- Expose predictable success messages -- Add stable row identifiers -- Reduce dependence on random element IDs - -## 21. Open Design Decisions - -The following decisions remain implementation-level choices unless later locked by an approved architecture note: - -- Whether Roslyn, Scriban, or a hybrid is the default generator engine -- Whether artefact storage is local filesystem first or abstracted for object storage from day one -- Whether replay executes in-process or via a dedicated background worker boundary -- Which authentication provider is used for the host application -- Which database provider is used as the primary development default - -## 22. Build, Test, and Delivery Requirements - -### 22.1 Repository Boundaries - -The test mining platform shall be introduced into this repository as a new vertical slice using the `TestMining.Platform.*` naming convention described in Section 17. Existing Symphony projects (`src/Symphony.*`, `tests/Symphony.*`, `symphony_docs/`, and `SPEC.md`) are retained tooling assets and shall not be mutated by work targeting this specification unless a task explicitly requires it. - -### 22.2 Build System - -- The solution shall build with the repository's documented required .NET SDK version. Any SDK upgrade shall be an explicit, documented change. -- `dotnet restore`, `dotnet build`, and `dotnet test` shall succeed from a clean checkout with no unresolved warnings treated as errors in core projects. -- Projects shall enable nullable reference types and treat analyzer warnings as errors where practical. - -### 22.3 Continuous Integration - -The repository shall include a CI pipeline (GitHub Actions is the expected host) that, on every pull request touching the platform: - -1. Restores and builds the full solution. -2. Runs unit and integration tests, including PostgreSQL-backed integration tests using a containerised database. -3. Runs generator snapshot tests (Section 14.4). -4. Publishes test results and code coverage summaries. -5. Fails the pipeline if any new secret-looking string appears in diffs or if retention/policy violations are detected by linting rules once established. - -### 22.4 Recorder Script Delivery - -The browser-side recorder script is a first-class build artefact: - -- It shall live in source control under a clearly named project directory and shall be built deterministically as part of the platform build. -- Its version shall be pinned and surfaced in captured recordings as `RecorderVersion` metadata so replay and healing can detect incompatible captures. -- It shall be injected via Playwright-supported initialisation hooks. Runtime fetching of the recorder from an external origin is prohibited. - -### 22.5 Test Layering and Flake Budget - -- Unit tests shall be the default: deterministic, no network, no browser. -- Integration tests shall exercise real infrastructure (PostgreSQL, Playwright against fixture web applications shipped with the repo). -- End-to-end tests shall be kept small in number and tagged so they can be excluded from fast developer loops. -- Any test that becomes flaky shall be either fixed or quarantined within one working day; persistent quarantine is not acceptable. - -## 23. Implementer Guardrails - -These guardrails exist to prevent the most likely failure modes of agentic or human implementers working on this specification. - -### 23.1 Prohibited Actions - -1. Writing captured credentials, cookies, storage states, tokens, or any value classified as `Sensitive` to logs, screenshots metadata, generation previews, or generated code. -2. Launching a Playwright browser context against a target URL that does not match the configured allow-list. -3. Introducing provider-specific SQL or raw ADO.NET calls into domain or application layers. Provider-specific code shall live in clearly isolated infrastructure components. -4. Editing generated source files by hand and checking the result in. Generated output shall be reproduced from scenario data and templates only. -5. Promoting an AI-assisted suggestion to a runtime-critical path. AI may advise; deterministic logic decides. -6. Deleting or restructuring Symphony-related files while working on test mining platform tasks. -7. Silently relaxing any non-functional requirement in Section 15, security requirement in Section 12, or acceptance criterion in Section 19. - -### 23.2 Required Behaviours - -1. Cite the requirement identifier (e.g. `FR-REC-004`, `FR-HEAL-003`) that motivates a code change in the pull request description. -2. For any behaviour change, update or add a conformance test traceable to the cited requirement. -3. Use feature-flagged rollout for anything that touches healing auto-apply, AI assistance, or cross-environment configuration. -4. When a requirement is ambiguous, raise a clarification rather than guessing. Record the resolution in this document or in an architecture decision record. - -### 23.3 Pull Request Discipline - -1. Pull requests shall be scoped to a single vertical slice. Drive-by refactors, especially across the Symphony/test-mining boundary, shall be split into separate changes. -2. Every PR shall identify new or changed artefact retention characteristics, new captured data fields, or new outbound network calls. -3. Secret-scanning and dependency vulnerability checks shall run before merge. - -## 24. Final Requirement Statement - -This repository’s new application shall be a C#-based semantic test mining platform built on ASP.NET Core, Blazor Server, Microsoft.Playwright for .NET, and PostgreSQL or SQL Server. It shall record browser interactions, infer structured scenarios, rank resilient locators, generate maintainable Playwright C# tests, replay them with strong diagnostics, and support deterministic healing while keeping structured scenarios as the enduring source of truth. +# @EndSquareTesting Requirements Specification + +## 1. Executive Summary + +The application defined by this specification is not a raw click recorder. It is a semantic test mining and stabilisation platform. + +The platform shall: + +- Record browser sessions performed by a human user against a target web application. +- Observe DOM state, accessibility metadata, navigation changes, network outcomes, and runtime context around each meaningful interaction. +- Transform raw browser events into a structured, intent-rich scenario model. +- Infer resilient selectors, variable data strategies, and business-level assertions. +- Generate readable C# Playwright test code and supporting assets from the scenario model. +- Replay generated tests and attempt deterministic healing when selectors or data drift. +- Preserve enough artefacts, metadata, and diagnostics to let users review, edit, approve, and regenerate scenarios over time. + +The platform shall be built with the following solution model: + +- Backend: ASP.NET Core +- Frontend: Blazor Server +- Database: PostgreSQL +- Browser automation: Microsoft.Playwright for .NET +- Code generation: Roslyn or Scriban templates +- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper + +The single most important product decision is this: the system shall store structured scenarios as the source of truth, not generated code. Generated code is a derived artefact and must always be reproducible from persisted scenario data plus generation settings. + +## 2. Product Vision + +### 2.1 Problem Statement + +Teams frequently want fast automated coverage of real user workflows, but hand-writing end-to-end tests is expensive and generic recorders produce brittle output. Raw click streams, unstable CSS selectors, exact-text assertions, and timing-based playback produce tests that quickly fail after routine UI changes. + +The platform shall solve this by treating recording as a data capture step and test generation as a compilation step. It shall infer user intent, promote resilient selectors, classify dynamic data, and generate maintainable Playwright tests that align with how users experience the application. + +### 2.2 Product Goal + +Enable product owners, testers, developers, and analysts to record business workflows once, refine them in a guided UI, and produce resilient C# Playwright tests that can be replayed, diagnosed, and healed as applications evolve. + +### 2.3 Product Principles + +The product shall be designed around these principles: + +1. Semantic first. +The system shall prefer user-facing meaning such as roles, labels, accessible names, nearby headings, and business outcomes over raw DOM structure. + +2. Structured source of truth. +The scenario model shall be the canonical artefact from which tests, page objects, diagnostics, and healing recommendations are derived. + +3. Deterministic before intelligent. +The system shall rely on deterministic heuristics and Playwright-native behaviour first. AI-assisted suggestions may augment the system, but must not be the only execution path for runtime-critical flows. + +4. User review is part of the workflow. +The platform shall support manual review and editing before generated code is considered ready. + +5. Diagnostics are mandatory. +Every recording, generation, replay, and healing attempt shall preserve sufficient evidence for supportability and debugging. + +## 3. Scope + +### 3.1 In Scope + +The v1 platform shall support: + +- Guided browser session recording +- DOM and accessibility metadata capture +- Playwright-based environment observation +- Post-recording scenario review and editing +- Locator ranking and locator candidate persistence +- Assertion suggestion and manual approval +- Variable data extraction and classification +- C# Playwright code generation +- Generated helper library emission +- Replay execution and diagnostics +- Deterministic selector healing workflows +- Persistent storage of sessions, scenarios, steps, artefacts, and replay history +- Multi-user authenticated web administration experience through Blazor Server +- Export of generated tests into repository-friendly file structures + +### 3.2 Out of Scope for v1 + +The following are explicitly out of scope unless later prioritised: + +- Full low-code test authoring without any review step +- Support for native mobile app automation +- AI-only locator resolution at execution time +- Canvas-heavy, game-like, or non-DOM-centric application automation as a first-class supported scenario +- Rich collaborative review workflows such as branching, inline comments, or merge conflict handling between multiple editors +- Fine-grained enterprise RBAC beyond foundational role separation +- General-purpose workflow orchestration unrelated to browser test mining +- Electron or .NET MAUI desktop packaging in the initial release + +### 3.3 Deferred but Anticipated Scope + +The architecture shall allow later addition of: + +- Page object refactoring generation +- CI/CD export packs and pipeline templates +- Browser extension-based capture +- AI-assisted scenario summarisation and healing proposals +- Team collaboration and scenario versioning +- Desktop packaging for offline-friendly operator experiences + +## 4. Target Users and Primary Use Cases + +### 4.1 User Roles + +The system shall recognise these primary personas: + +1. Test Author +Records workflows, reviews steps, approves assertions, and generates tests. + +2. Developer +Consumes generated code, integrates it into solution structures, and refines helpers or templates. + +3. QA Engineer +Validates workflow coverage, replay reliability, and failure diagnostics. + +4. Platform Administrator +Configures browser runners, storage, security, code-generation defaults, and environment connections. + +### 4.2 Core Use Cases + +The platform shall support these end-to-end use cases: + +1. Record a new business workflow from a live application. +2. Review and edit the recorded timeline before generation. +3. Mark inputs as fixed, variable, generated, lookup-driven, or sensitive. +4. Accept or reject suggested assertions. +5. Generate a readable Playwright C# test suite and helper files. +6. Replay the generated scenario against the target application. +7. Investigate replay failures using screenshots, traces, DOM context, and locator diagnostics. +8. Accept a healing suggestion and regenerate the derived code. + +## 5. High-Level System Overview + +### 5.1 Product Capabilities + +The application shall consist of these major capability areas: + +1. Recording +Browser session launch, event capture, instrumentation injection, and artefact creation. + +2. Interpretation +DOM intelligence, locator ranking, variable classification, widget recognition, and assertion inference. + +3. Scenario Authoring +Timeline editing, step suppression, annotation, data strategy configuration, and acceptance of inferred outputs. + +4. Generation +Transformation of scenario models into C# Playwright tests, helpers, fixtures, and optional page object scaffolding. + +5. Replay and Healing +Execution, diagnostics, candidate re-resolution, deterministic healing, and controlled regeneration. + +6. Administration +Environment configuration, browser configuration, template settings, project export settings, and user management. + +### 5.2 Logical Architecture + +The solution shall be decomposed into the following logical components: + +1. Blazor Server UI +- Hosts recording workflows, timeline editing, generation preview, replay diagnostics, and admin surfaces. + +2. ASP.NET Core Application Host +- Provides HTTP endpoints, SignalR backplane for live session updates, background workers, authentication, orchestration, and API surfaces for the UI. + +3. Recording Engine +- Launches Playwright browsers and coordinates injected capture scripts, event streams, screenshots, traces, and browser-side metadata collection. + +4. Capture and Analysis Engine +- Normalises raw browser events, enriches them with DOM context, identifies semantic widgets, ranks locators, and produces assertion candidates. + +5. Scenario Engine +- Stores, edits, versions, validates, and materialises structured scenarios. + +6. Code Generation Engine +- Produces C# Playwright test source files, helper classes, optional page objects, and configuration assets using Roslyn or Scriban. + +7. Replay and Healing Engine +- Replays scenarios or generated tests, captures outcomes, diagnoses failures, evaluates fallback locators, and records healing proposals. + +8. Persistence Layer +- Stores structured entities, audit history, artefact references, configuration, and operational logs in PostgreSQL or SQL Server. + +9. Artefact Storage +- Stores screenshots, traces, exported code bundles, DOM snapshots, and replay attachments on a filesystem or object-backed abstraction. + +## 5.3 Operational Lifecycles + +### 5.3.1 Recording Lifecycle + +The recording lifecycle shall follow this logical sequence: + +1. User creates a recording session with target URL and settings. +2. The backend provisions a Playwright browser context and applies configured bootstrap settings. +3. Recorder instrumentation is injected before or at page startup. +4. The user performs actions in the target application. +5. Raw browser events and Playwright observations are streamed to the backend. +6. The backend normalises events and persists session artefacts incrementally. +7. The user stops recording. +8. The backend finalises the session and generates an initial scenario draft for review. + +### 5.3.2 Scenario Authoring Lifecycle + +The authoring lifecycle shall support: + +1. Reviewing the recorded step timeline. +2. Removing or suppressing noise. +3. Promoting meaningful events into semantic actions. +4. Approving or rejecting inferred assertions. +5. Classifying test data and secrets. +6. Choosing generation profile settings. +7. Saving an immutable scenario version ready for generation or replay. + +### 5.3.3 Generation Lifecycle + +The generation lifecycle shall support: + +1. Selecting an approved scenario version. +2. Applying generation profile and template settings. +3. Producing output files and a manifest. +4. Storing a checksum and file inventory. +5. Presenting the generated output for preview and export. + +### 5.3.4 Replay and Healing Lifecycle + +The replay and healing lifecycle shall support: + +1. Selecting a scenario version or generated artefact for execution. +2. Launching an isolated Playwright replay context. +3. Executing steps while recording diagnostics. +4. Classifying failures when they occur. +5. Attempting deterministic healing when drift is suspected. +6. Presenting a proposal with supporting evidence. +7. Applying approved healing changes to a new scenario version or updating approved locator preferences under controlled rules. + +## 6. Mandatory Technology Requirements + +### 6.1 Application Stack + +The implementation shall use: + +- ASP.NET Core for backend hosting and service orchestration +- Blazor Server for the primary web UI +- Microsoft.Playwright for .NET for browser control, observation, replay, trace capture, and browser-context management +- EF Core for relational persistence +- PostgreSQL or SQL Server as the durable database +- Roslyn and/or Scriban for code generation + +### 6.2 Database Provider Strategy + +To make v1 deliverable without doubling persistence cost, the platform shall adopt the following strategy: + +- PostgreSQL shall be the **primary and only required provider for v1**. Local development, shared non-production, and production-like deployments shall all use PostgreSQL unless explicitly reconfigured. +- SQL Server shall be a **deferred-but-supported secondary provider**. The persistence layer shall be designed so that SQL Server can be added later without redesigning the domain or repositories, but v1 acceptance does not require a working SQL Server deployment. +- The persistence layer shall use EF Core abstractions and migrations. +- Provider-specific SQL shall be avoided unless isolated behind clearly bounded infrastructure components and gated by a provider capability check. +- Data types, indexing strategies, JSON storage, and concurrency rules shall be designed to remain portable to SQL Server. Use of PostgreSQL-only features (for example `jsonb` operators, array columns, partial indexes) is permitted in v1 provided the feature can be substituted or approximated on SQL Server later. +- Integration tests shall run against real PostgreSQL (for example Testcontainers) rather than an in-memory provider, because in-memory providers do not exercise migration or JSON behaviour. + +### 6.3 Browser Support + +The initial supported execution browser shall be Chromium through Playwright. The architecture shall not prevent later support for WebKit and Firefox, but those browsers are not required for v1 unless explicitly enabled by configuration. + +## 7. Information Architecture and Domain Model + +### 7.1 Canonical Principle + +The canonical persisted business artefact shall be `Scenario`. Generated tests, replay runs, and healing decisions shall reference the scenario version they were derived from. + +### 7.2 Core Entities + +The persistence model shall include, at minimum, these entities. + +#### 7.2.0 Common Entity Requirements + +Unless explicitly exempted, every persistent entity defined in this section shall carry: + +- A stable primary key (`Id`) that is globally unique (GUID/UUID) and assigned by the application, not the database. +- Audit fields: `CreatedAtUtc`, `CreatedByUserId`, `UpdatedAtUtc`, `UpdatedByUserId` (the last two may be nullable where the entity is immutable by design, such as `ScenarioVersion` and `RecordedStep`). +- A `RowVersion` (optimistic concurrency token) column on entities that are editable by more than one workflow, including `Scenario`, `RecordingSession`, `HealingSuggestion`, and administrative configuration entities. +- A soft-delete marker (`IsDeleted`, `DeletedAtUtc`, `DeletedByUserId`) on `Scenario`, `RecordingSession`, `GenerationArtifact`, and artefact records. Immutable versions (e.g. `ScenarioVersion`, `RecordedStep`) shall not support soft delete; deletion of a scenario shall be a cascade-tombstoning operation documented in the data model. +- Timestamps stored in UTC; the domain shall not depend on database-local clocks for ordering decisions beyond audit. + +Field lists in the subsections below name the entity-specific fields and may omit the common fields above for brevity. + +#### 7.2.1 RecordingSession + +Represents an interactive capture session started by a user. + +Required fields: + +- `Id` +- `Name` +- `TargetBaseUrl` +- `EnvironmentName` +- `Status` +- `StartedByUserId` +- `StartedAtUtc` +- `EndedAtUtc` +- `BrowserKind` +- `BrowserChannel` +- `Headless` +- `AuthenticationMode` +- `RecordAssertionsAutomatically` +- `MaskedFieldPolicyJson` +- `SettingsJson` + +#### 7.2.2 Scenario + +Represents a business workflow compiled from one or more recording sessions and maintained as the product source of truth. + +Required fields: + +- `Id` +- `Name` +- `Description` +- `Status` +- `PrimaryRecordingSessionId` +- `TargetApplicationName` +- `TargetBaseUrl` +- `EnvironmentName` +- `CreatedByUserId` +- `CreatedAtUtc` +- `UpdatedAtUtc` +- `CurrentVersionNumber` +- `DefaultGenerationProfileId` +- `TagsJson` + +#### 7.2.3 ScenarioVersion + +Represents an immutable version of the scenario structure used for generation or replay traceability. + +In-progress edits shall not mutate an immutable `ScenarioVersion`. Any draft editing workflow shall +store draft state separately and create a new immutable scenario version when changes are committed. + +Required fields: + +- `Id` +- `ScenarioId` +- `VersionNumber` +- `ParentScenarioVersionId` +- `CreatedAtUtc` +- `CreatedByUserId` +- `ChangeSummary` +- `StructureJson` +- `InferenceMetadataJson` +- `ApprovedForGeneration` + +#### 7.2.4 RecordedStep + +Represents a normalised action or assertion candidate in sequence order. + +Required fields: + +- `Id` +- `ScenarioVersionId` +- `Sequence` +- `StepType` +- `SemanticActionType` +- `DisplayName` +- `UserNote` +- `Status` +- `UrlBefore` +- `UrlAfter` +- `TimestampUtc` +- `DurationMs` +- `RawValue` +- `NormalisedValue` +- `VariableStrategy` +- `SensitivityClassification` +- `TargetElementSnapshotId` +- `ParentContainerSnapshotId` +- `ExpectedOutcomeJson` +- `GeneratedCodeHintJson` + +#### 7.2.5 ElementSnapshot + +Represents the known properties of an element at record or replay time. + +Required fields: + +- `Id` +- `CapturePhase` +- `RecordingSessionId` (nullable; populated for recording-time capture) +- `ReplayRunId` (nullable; populated for replay-time capture) +- `RecordedStepId` +- `TagName` +- `InputType` +- `Role` +- `AccessibleName` +- `LabelText` +- `Placeholder` +- `TextContent` +- `TitleAttribute` +- `NameAttribute` +- `ValuePreview` +- `TestId` +- `AriaAttributesJson` +- `DataAttributesJson` +- `CssPath` +- `XPath` +- `BoundingBoxJson` +- `VisibilityState` +- `EnabledState` +- `ParentContextJson` +- `SiblingContextJson` +- `WidgetKind` +- `DomHash` + +#### 7.2.6 LocatorCandidate + +Represents one potential locator strategy for a target element. + +Required fields: + +- `Id` +- `RecordedStepId` +- `StrategyType` +- `Expression` +- `ScopeExpression` +- `Priority` +- `Confidence` +- `UniquenessAtRecordTime` +- `VisibilityAtRecordTime` +- `Recommended` +- `FailureReasonIfRejected` + +#### 7.2.7 AssertionCandidate + +Represents a suggested or approved expected outcome. + +Required fields: + +- `Id` +- `RecordedStepId` +- `AssertionType` +- `ExpressionJson` +- `Priority` +- `Confidence` +- `SourceSignal` +- `Approved` +- `RejectedReason` + +#### 7.2.8 ReplayRun + +Represents one execution attempt of a scenario or generated test. + +Required fields: + +- `Id` +- `ScenarioId` +- `ScenarioVersionId` +- `ReplaySourceKind` +- `GenerationArtifactId` (nullable; populated when replay uses a generated artefact bundle) +- `Status` +- `StartedAtUtc` +- `EndedAtUtc` +- `TriggeredByUserId` +- `ExecutionEnvironment` +- `BrowserKind` +- `Headless` +- `FailureCategory` +- `FailureSummary` +- `DiagnosticsJson` + +#### 7.2.9 HealingSuggestion + +Represents a proposed change when replay detects drift. + +Required fields: + +- `Id` +- `ReplayRunId` +- `RecordedStepId` +- `SuggestionType` +- `CurrentFailureEvidenceJson` +- `ProposedLocatorJson` +- `DeterministicConfidence` +- `AiConfidence` +- `ApprovalStatus` +- `AppliedAtUtc` +- `AppliedByUserId` + +#### 7.2.10 UserAccount + +Represents an authenticated principal of the platform. Identity data may be federated from an external provider, but the platform shall maintain a local record to support authorship attribution, audit trails, and role assignment. + +Required fields: + +- `Id` +- `ExternalSubjectId` (nullable; populated when federated) +- `DisplayName` +- `Email` +- `Role` (at minimum one of: `Administrator`, `Author`, `Viewer`) +- `IsDisabled` +- `LastLoginAtUtc` + +All audit references in other entities (for example `StartedByUserId`, `CreatedByUserId`, `TriggeredByUserId`) shall resolve to `UserAccount.Id`. + +#### 7.2.11 GenerationArtifact + +Represents a generated bundle or file set. + +Required fields: + +- `Id` +- `ScenarioVersionId` +- `GeneratorKind` +- `GeneratorVersion` +- `TemplateProfile` +- `TemplateVersion` +- `OutputRoot` +- `ManifestJson` +- `CreatedAtUtc` +- `CreatedByUserId` +- `Checksum` + +### 7.3 Structured Scenario Contract + +The application shall define a first-class structured scenario contract that can represent: + +- Navigation steps +- Click actions +- Fill actions +- Select actions +- Checkbox and toggle interactions +- Keyboard actions +- Dialog interactions +- Grid and list interactions +- Widget-scoped semantic actions +- Assertions +- Control flow metadata such as waits inferred from user-facing events +- Variable definitions +- Data generation strategies +- Replay and healing hints + +The scenario contract shall be expressive enough that: + +- Tests can be regenerated without rerecording. +- Helper libraries can evolve independently of the raw recording. +- Multiple code-generation styles can be supported later. +- Healing decisions can update the scenario model without directly editing generated output. + +## 7.4 Artefact Model + +The application shall treat the following as first-class artefacts: + +- Screenshots +- Playwright traces +- DOM snapshots +- Session logs +- Generated source files +- Replay diagnostics packages + +Each artefact shall have: + +- A stable identifier +- A logical owner such as recording session, scenario version, or replay run +- Content type +- Retention classification +- Storage path or provider-specific handle +- Created timestamp +- Optional checksum +- A size-in-bytes record + +The platform shall enforce default retention and size ceilings so artefact storage does not grow unbounded in the absence of administrative action: + +- Screenshots and DOM snapshots: retained for at least 30 days after the owning session or replay run completes; individual artefact size cap of 10 MB unless an administrator raises it. +- Playwright traces: retained for at least 14 days; individual trace cap of 200 MB. +- Session logs and replay diagnostic bundles: retained for at least 30 days. +- Generated source bundles: retained until the owning scenario is deleted. + +Administrators may extend any retention window or raise any size cap through the configuration surface defined in Section 11. A background cleanup process shall delete artefacts that have exceeded their retention window and record the deletion in the audit trail. + +## 7.5 State Model Expectations + +The system shall define explicit statuses for at least: + +- Recording sessions +- Scenarios +- Scenario versions +- Replay runs +- Healing suggestions +- Generation artefacts + +Status transitions shall be explicit, validated, and auditable. + +## 8. Functional Requirements + +### 8.1 Recording Requirements + +#### FR-REC-001 Guided Recording Creation + +The system shall let a user start a recording by providing: + +- Scenario name +- Target application URL +- Optional environment label +- Browser selection +- Whether automatic assertion suggestions are enabled +- Authentication or session bootstrap mode + +#### FR-REC-002 Browser Session Launch + +The system shall launch a Playwright browser context for recording with configurable: + +- Headless or headed mode +- Viewport +- Storage state +- Trace settings +- Video settings, if enabled +- Network capture settings + +#### FR-REC-003 Early Instrumentation Injection + +The system shall inject recorder logic before application content loads wherever technically possible, using Playwright-supported initialisation hooks such as `AddInitScriptAsync`. + +#### FR-REC-004 Event Capture + +The recorder shall capture, at minimum, these event types when user meaningful: + +- Page navigation +- Click +- Double click when distinguishable +- Input +- Change +- Form submit +- Key press +- Dialog open and close +- File download start +- Popup open +- Checkbox or toggle change +- Select and autocomplete selection + +The recorder shall suppress noise events such as incidental mouse movement unless explicitly enabled for diagnostics. + +#### FR-REC-005 Element Context Capture + +For each meaningful event, the system shall capture contextual element metadata including, where available: + +- HTML tag +- Input type +- Role +- Accessible name +- Label text +- Placeholder +- Visible text +- Title attribute +- Name attribute +- Test id or equivalent stable data attribute +- Bounding box +- Enabled and visible state +- Nearby heading +- Parent form, region, modal, or table context +- Candidate CSS path +- Candidate XPath as a last-resort fallback + +#### FR-REC-006 Observer Correlation + +The recording engine shall correlate user events with surrounding Playwright-observed browser state including: + +- URL changes +- Page load phases +- Network requests and responses +- Console errors and warnings, if enabled +- Dialog lifecycle +- Download lifecycle +- Popup lifecycle +- Screenshots +- Trace segments + +#### FR-REC-007 Sensitive Data Handling + +The user shall be able to mark fields or values as sensitive during or after recording. Sensitive values shall be masked in persisted logs, previews, and generated artefacts unless the user explicitly authorises retention in a secure variable form. + +#### FR-REC-008 Recording Controls + +The UI shall allow the user to: + +- Start +- Pause +- Resume +- Stop +- Cancel +- Mark current step as important +- Mark current step as ignorable +- Add an inline note + +#### FR-REC-009 Recorder Transport + +The recorder shall use a reliable browser-to-host transport mechanism for event delivery. Preferred mechanisms include Playwright-exposed bindings or a similarly explicit channel. Console scraping may be used only as a bounded fallback or diagnostic technique, not as the primary durable transport contract. + +#### FR-REC-010 Incremental Persistence + +Long-running recording sessions shall persist meaningful progress incrementally so that unexpected host failures do not necessarily lose the entire session. + +### 8.2 Post-Processing and Inference Requirements + +#### FR-INF-001 Event Normalisation + +The system shall transform raw captured events into a cleaned event stream by: + +- Removing technical noise +- Merging related input bursts +- Folding navigation waits into surrounding intent +- Linking actions to the most likely observed outcomes + +#### FR-INF-002 Semantic Action Inference + +The system shall infer higher-order action meaning where possible, including but not limited to: + +- Open details page +- Save form +- Submit search +- Open modal +- Confirm dialog +- Select row in grid +- Apply filter +- Choose date +- Dismiss notification + +#### FR-INF-003 Widget Recognition + +The system shall recognise common UI widget patterns where feasible, including: + +- Forms +- Tables and grids +- Modals and dialogs +- Dropdowns and combo boxes +- Tabs +- Toast notifications +- Date pickers +- Autocomplete controls + +The recognised widget type shall be stored as metadata usable by generation and healing logic. + +#### FR-INF-004 Locator Ranking + +For each targetable step, the system shall create a ranked set of locator candidates. The ranking algorithm shall prioritise: + +1. Explicit test id or configured stable data attribute +2. Role plus accessible name +3. Label-based targeting +4. Visible text +5. Placeholder +6. Scoped parent-region plus child-role targeting +7. Stable attribute combinations +8. CSS selector fallback +9. XPath fallback + +#### FR-INF-005 Locator Validation + +Each locator candidate shall be validated at record time when possible for uniqueness, visibility, and scope confidence. + +#### FR-INF-006 Variable Classification + +The system shall support classifying entered or observed values into at least these categories: + +- Fixed test data +- Scenario variable +- Generated data +- Lookup value +- Sensitive value +- Incidental value that should not become an assertion target + +#### FR-INF-007 Assertion Suggestion + +The system shall infer candidate assertions from post-action signals including: + +- Success toast visibility +- URL pattern change +- Heading change +- Dialog closure +- Newly visible row, card, or detail view +- Relevant network success response +- Count changes where meaningful + +#### FR-INF-008 Fuzzy Outcome Strategies + +The system shall support resilience strategies for data-sensitive assertions including: + +- Exact text to contains matching +- Regex matching for identifiers +- Date and time range validation +- Row existence by key column +- Partial heading matching +- Minimum count assertions + +#### FR-INF-009 Confidence Scoring + +The system shall assign confidence scores to inferred locators, assertions, widget classifications, and healing suggestions. Confidence values shall be explainable and based on deterministic evidence where possible. + +### 8.3 Scenario Authoring Requirements + +#### FR-AUTH-001 Timeline Review + +The UI shall present recorded scenarios as an ordered timeline of meaningful steps with human-readable labels. + +#### FR-AUTH-002 Step Editing + +The user shall be able to: + +- Delete a step +- Suppress a step from generation +- Reorder steps where valid +- Convert a step into an assertion +- Change a step to use a different locator candidate +- Add notes and rationale +- Mark a step as variable-driven +- Mark a step as sensitive + +#### FR-AUTH-003 Assertion Approval + +Suggested assertions shall not be treated as final until approved by the user or by a configured auto-approval rule. + +#### FR-AUTH-004 Data Strategy Editing + +The author shall be able to review every captured value and choose whether it becomes: + +- A literal +- A scenario parameter +- A generated value +- A fixture reference +- A secret +- Ignored data + +#### FR-AUTH-005 Scenario Validation + +Before generation, the system shall validate that the scenario is structurally coherent, including: + +- Ordered step sequence +- Supported step types +- Required locator presence +- Required variable definitions +- No unresolved sensitive-data policy violations + +#### FR-AUTH-006 Version Creation + +Saving a material scenario change after review shall create a new immutable scenario version. +If the product supports draft editing, the draft state shall be stored separately from immutable +scenario versions until the user commits the change. The application shall preserve enough history +to identify what changed between versions. + +### 8.4 Code Generation Requirements + +#### FR-GEN-001 Derived Output Principle + +Generated source files shall always be reproducible from scenario data and template settings. Manual edits to generated code shall not silently become the source of truth. + +#### FR-GEN-002 Supported Output + +The v1 system shall generate: + +- C# Playwright test classes +- Shared locator-resolution helpers +- Optional data model or fixture classes +- Supporting test project files or export manifests where needed + +#### FR-GEN-003 Readability + +Generated code shall prioritise readability for human developers. It shall: + +- Use clear method names +- Avoid unreadable deeply nested locator code where helper abstraction is appropriate +- Separate data setup from action steps +- Use Playwright web-first assertions rather than fixed sleeps + +#### FR-GEN-004 LocatorResolver Helper + +The generated runtime helper library shall include a locator-resolution abstraction capable of: + +- Trying candidates in ranked order +- Validating uniqueness before acting +- Optionally scoping search to a parent container +- Emitting useful diagnostics when no candidate resolves uniquely + +#### FR-GEN-005 Assertion Generation + +Generated assertions shall prefer user-facing outcome checks over implementation-detail checks. + +#### FR-GEN-006 Variable and Fixture Generation + +When configured, the generator shall emit strongly typed classes or structures representing scenario variables, generated data, and fixture data. + +#### FR-GEN-007 Generation Profiles + +The system shall support generation profiles so teams can choose between styles such as: + +- Flat test methods +- Test plus helper methods +- Test plus page object scaffolding + +#### FR-GEN-008 Template Technology + +The implementation may use Roslyn, Scriban, or a hybrid approach, but the chosen approach shall support: + +- Deterministic output +- Repeatable formatting +- Testable generation logic +- Easy future extension + +#### FR-GEN-009 Repository-Friendly Output + +The generation system shall support output layouts that are easy to place under source control, including predictable file names, stable folder structure, and manifests that identify which files belong to each generation run. + +#### FR-GEN-010 Runtime Helper Compatibility + +Generated helper libraries shall remain versioned and compatible with the generator profile that emitted them. Breaking helper changes shall be traceable through generation manifests or template version metadata. + +### 8.5 Replay Requirements + +#### FR-REP-001 Scenario Replay + +The system shall replay a scenario using Playwright and the currently approved scenario version or the chosen generated artefact. + +#### FR-REP-002 Replay Diagnostics + +Each replay run shall capture: + +- Start and end timestamps +- Browser and environment settings +- Screenshots at key failure points +- Playwright traces when enabled +- Step-level pass or fail state +- Locator resolution attempts +- Assertion evaluation results +- Failure categorisation + +#### FR-REP-003 Failure Categories + +Replay failures shall be classified into categories including: + +- Locator resolution failure +- Timeout +- Unexpected navigation +- Assertion failure +- Authentication failure +- Environment availability failure +- Script or generation bug + +#### FR-REP-004 Targeted Rerun + +The UI shall allow rerunning: + +- The full scenario +- The current step onward +- A single step for diagnostic purposes where valid + +#### FR-REP-005 Replay Isolation + +Replay runs shall execute in isolated browser contexts and shall not share mutable session state unless explicitly configured for a trusted use case. + +### 8.6 Healing Requirements + +#### FR-HEAL-001 Deterministic Healing First + +When replay fails due to locator drift, the system shall first attempt deterministic recovery using: + +- Alternate persisted locator candidates +- Scoped searches +- Fuzzy accessible-name or text matching +- Parent context anchoring +- Nearby heading anchoring + +#### FR-HEAL-002 Healing Evidence + +Every healing proposal shall include evidence showing: + +- The original locator strategy +- Why it failed +- The proposed replacement +- What contextual signals supported the proposal +- Confidence scores + +#### FR-HEAL-003 Human Approval + +In v1, every healing change that alters the persisted scenario shall require explicit human approval. Automatic application of healing proposals to persisted scenarios shall not ship in v1. + +A controlled auto-apply policy for deterministic high-confidence matches may be introduced in a later phase, subject to all of the following: + +- The feature shall be disabled by default. +- Auto-apply shall be gated behind an administrator-only setting scoped per environment (never globally by default). +- Auto-apply shall be restricted to `SuggestionType` values classified as locator-only, deterministic, and non-destructive. Changes to assertion semantics, step ordering, variable classification, or sensitive-data policy shall never be auto-applied. +- A dry-run mode shall be available that records what would have been applied without mutating the scenario. +- Every auto-applied change shall still create a new immutable `ScenarioVersion` with a traceable `HealingSuggestion` link. + +#### FR-HEAL-004 AI as Adviser + +AI-assisted healing, if enabled later, shall be advisory and reviewable. It shall not bypass deterministic diagnostics or obscure the basis for a proposal. + +#### FR-HEAL-005 Healing Traceability + +When a healing proposal is approved and applied, the system shall retain a clear audit record linking the replay failure, the proposal, the approver, and the resulting scenario or locator change. + +### 8.7 Administration Requirements + +#### FR-ADM-001 Environment Configuration + +Administrators shall be able to configure: + +- Application environments +- Browser execution settings +- Base URLs +- Storage settings +- Database provider and connection +- Artefact retention rules +- Code-generation defaults + +#### FR-ADM-002 Authentication Bootstrap Options + +The system shall support configurable authentication bootstrap approaches such as: + +- Manual login during recording +- Stored Playwright storage state +- Cookie import under controlled administration + +#### FR-ADM-003 Auditability + +Administrative changes shall be auditable with who, when, and what changed. + +#### FR-ADM-004 Retention Policies + +Administrators shall be able to configure retention policies for logs, screenshots, traces, generated artefacts, and stale replay diagnostics. + +## 9. User Interface Requirements + +### 9.1 Overall UI Structure + +The Blazor Server application shall provide at least these primary work areas: + +1. Recording workspace +2. Timeline and scenario editor +3. Generated code preview +4. Replay and diagnostics workspace +5. Administration area + +### 9.2 Recording Workspace + +The recording workspace shall display: + +- Browser session status +- Start, pause, resume, and stop controls +- Current URL +- Live step feed +- Ability to flag importance or ignore noise +- Inline note entry + +### 9.3 Timeline Workspace + +The timeline workspace shall provide: + +- Ordered list of steps +- Step type badges +- Locator confidence indicators +- Assertion suggestion indicators +- Variable classification editing +- Reorder and suppress controls + +### 9.4 Code Preview Workspace + +The code preview workspace shall provide: + +- Generated C# preview +- File list or manifest view +- Warnings for low-confidence locators or assertions +- Regenerate action + +### 9.5 Replay Workspace + +The replay workspace shall provide: + +- Replay status +- Step-by-step result stream +- Failure summary +- Locator diagnostics +- Screenshot and trace links +- Healing proposal review and approval actions + +### 9.6 Responsiveness + +The application shall be usable on standard desktop browser sizes and shall degrade gracefully on smaller widths, but desktop-first layout is acceptable for v1. + +## 10. API and Integration Requirements + +### 10.1 Internal API Surface + +The ASP.NET Core backend shall expose internal or authenticated HTTP APIs sufficient to support: + +- Recording lifecycle control +- Scenario CRUD +- Timeline editing +- Replay triggering +- Healing approval +- Artefact retrieval +- Administrative configuration + +### 10.2 Real-Time Updates + +The application shall provide real-time UI updates for long-running recording and replay operations. SignalR is the preferred approach. + +### 10.3 Artefact Export + +The system shall support export of generated assets into a chosen output directory structure that is suitable for committing to a repository or copying into another solution. + +### 10.4 External Target Application Interaction + +The platform shall treat the target application as an external system under test. It shall not require modifications to the target application, but it shall provide guidance when better accessibility or testability hooks would improve recording quality. + +### 10.5 API Design Expectations + +Backend APIs should: + +- Use explicit resource identifiers +- Return actionable validation errors +- Support long-running operation status retrieval +- Avoid leaking secrets or raw sensitive payloads +- Be versionable from the outset + +## 11. Configuration Requirements + +### 11.1 Configuration Sources + +The application shall support configuration from: + +- App settings files +- Environment variables +- Secure secret providers or equivalent deployment-time injection +- Database-backed administrative settings where appropriate + +### 11.2 Configurable Areas + +The following areas shall be configurable: + +- Database provider and connection string +- Artefact storage root or provider +- Browser launch defaults +- Headless defaults +- Base URL allowlists +- Session timeout and retention policies +- Masking rules +- Default generation profile +- Replay diagnostics level +- Authentication provider settings + +### 11.3 Validation + +Invalid required configuration shall fail fast at application startup or at the point of administrative save, depending on whether the setting is environment-level or runtime-editable. + +## 12. Security and Privacy Requirements + +### 12.1 Authentication and Access Control + +The platform shall require authenticated access for non-public environments. Baseline role separation shall distinguish at least: + +- Administrator +- Author +- Viewer + +### 12.2 Secret Protection + +Secrets shall never be written to plain logs, screenshots metadata, or generated code by default. This includes: + +- Target application credentials +- Session cookies +- Tokens +- Secret variable values + +### 12.3 Sensitive Data Masking + +The system shall allow masking rules by field name, selector metadata, or manual marking. Masked values shall remain masked in: + +- UI previews +- Diagnostic logs +- Stored event payloads +- Generation previews + +### 12.4 Target URL Allow-List Enforcement + +The platform shall enforce an administrator-managed allow-list of target base URLs. A recording or replay session shall fail to start if its requested target URL does not match an entry on the allow-list for the selected environment. The allow-list check shall be performed server-side before any Playwright browser context is launched. The allow-list shall be auditable and changes shall be captured in the audit trail defined in Section 12.7. + +### 12.5 Encryption of Captured Session Material + +The platform captures material that is sensitive by construction: authentication cookies, Playwright storage states, raw event payloads containing user input, and user-authored scenario variables marked sensitive. The following protections shall apply: + +- Playwright storage states, cookies, and any cached authentication material shall be encrypted at rest using platform-managed keys sourced from a secure secret provider. +- Scenario variables classified as `Sensitive` shall be encrypted at rest. Their plaintext shall never be written to generated source, generation previews, logs, or UI previews. +- Raw event payloads shall have sensitive fields masked or encrypted at rest according to the `MaskedFieldPolicyJson` in force at recording time. A field whose classification is changed to sensitive after recording shall be retroactively masked in subsequent previews and re-generation. +- Key material shall not be checked into source control. Key rotation shall be supported at least by re-encrypting affected records during a controlled administrative operation. + +### 12.6 Browser Session Isolation + +Each recording or replay session shall execute in an isolated Playwright browser context unless explicitly configured otherwise by an administrator. + +### 12.7 Audit Logging + +The system shall maintain an audit trail for: + +- Login and administrative access +- Scenario creation and approval +- Healing approval +- Artefact export +- Configuration changes + +## 13. Observability and Diagnostics Requirements + +### 13.1 Structured Logging + +The backend shall emit structured logs with identifiers such as: + +- Recording session id +- Scenario id +- Scenario version id +- Replay run id +- User id +- Browser context id + +### 13.2 Operational Metrics + +The system shall capture metrics useful for support and scaling, including: + +- Active recordings +- Active replays +- Average recording duration +- Replay success rate +- Most common failure categories +- Healing proposal frequency +- Generation duration + +### 13.3 Artefact Traceability + +Screenshots, traces, DOM snapshots, and generated outputs shall remain traceable to the session, scenario version, or replay run that created them. + +## 14. Verification and Testing Requirements + +### 14.1 Product Test Strategy + +The platform implementation shall include automated coverage for: + +- Domain and inference rules +- Scenario validation +- Locator ranking +- Assertion suggestion rules +- Code-generation output shape +- Replay orchestration +- Healing evaluation +- Persistence mappings and migrations + +### 14.2 Test Layers + +At minimum, the repository should include: + +- Unit tests for core business rules and model transformations +- Integration tests for Playwright coordination, persistence, and artefact storage +- End-to-end tests against controlled sample web applications + +### 14.3 Regression Fixtures + +The implementation should maintain representative sample applications or fixture pages that cover: + +- Basic forms +- Grid interactions +- Modal dialogs +- Login flows +- Dynamic identifiers +- Toast-based success notifications + +### 14.4 Generator Verification + +Generated output shall be verified through snapshot-style approval tests or equivalent structural tests so template changes do not unintentionally degrade code readability or behaviour. + +## 15. Non-Functional Requirements + +### 15.1 Reliability + +The platform shall favour deterministic execution. Generated tests and replay flows shall use Playwright waiting behaviour and retryable assertions rather than fixed delays except where no better alternative exists. + +### 15.2 Performance + +The system shall feel responsive for interactive authoring. Initial targets, measured on a reference developer workstation (quad-core modern CPU, SSD, local PostgreSQL), expressed as percentiles over a rolling sample of at least 20 operations of the same type: + +- Time from "Start recording" click to Playwright page navigation ready: p50 ≤ 5 s, p95 ≤ 10 s. +- End-to-end latency from a captured browser event to its corresponding step appearing in the timeline: p50 ≤ 750 ms, p95 ≤ 2 s. +- Code generation for a scenario of up to 100 meaningful steps: p50 ≤ 8 s, p95 ≤ 15 s. +- Replay start-up overhead (time from replay trigger to first step execution) shall not exceed 10 s at p95. + +These are product-quality targets, not contractual SLAs. Observability required under Section 13 shall be sufficient to verify them post-deployment. + +### 15.3 Scalability + +The v1 system shall support multiple concurrent users and background replays on a single server instance. The architecture shall allow future scale-out of non-UI background execution services if demand grows. + +### 15.4 Maintainability + +The implementation shall keep clear boundaries between: + +- UI concerns +- Application services +- Playwright execution +- Scenario/inference logic +- Persistence +- Generation +- Healing + +### 15.5 Testability + +The system itself shall be designed for automated testing using unit, integration, and end-to-end coverage. The product shall not rely on untestable static global state for core flows. + +### 15.6 Accessibility + +Because the product explicitly depends on accessibility metadata from target applications, the platform UI should also model good practice. The Blazor UI should use accessible semantics, keyboard support, and meaningful announcements for long-running operations where practical. + +## 16. Deployment and Operational Requirements + +### 16.1 Host Model + +The initial application shall run as an ASP.NET Core server-hosted web application with Blazor Server integrated into the host process or solution boundary. + +### 16.2 Environment Profiles + +The implementation shall support at least: + +- Local developer execution +- Shared non-production environment +- Production-like server deployment + +### 16.3 Packaging Direction + +Desktop packaging through .NET MAUI Hybrid or Electron is explicitly a future option. The initial architecture shall therefore keep the backend and UI boundaries clean enough that later packaging can host the same application surfaces without re-implementing core recording, generation, or replay services. + +### 16.4 Operational Safety + +Operational processes shall account for: + +- Browser process cleanup +- Expired artefact cleanup +- Failed replay recovery +- Application restarts during long-running operations + +## 17. Suggested Repository Architecture + +The application should be introduced into this repository as a new vertical slice rather than by mutating the existing Symphony specification. A suggested project layout is: + +```text +/src + /TestMining.Platform.Host (ASP.NET Core host, Blazor Server UI, auth, APIs) + /TestMining.Platform.Core (domain models, interfaces, scenario contracts) + /TestMining.Platform.Recording (Playwright launch, injected recorder, capture coordination) + /TestMining.Platform.Analysis (DOM intelligence, inference, locator ranking, assertions) + /TestMining.Platform.Generation (Roslyn/Scriban generation) + /TestMining.Platform.Replay (replay execution and diagnostics) + /TestMining.Platform.Healing (deterministic healing services) + /TestMining.Platform.Persistence (EF Core, PostgreSQL/SQL Server adapters, repositories) + /TestMining.Platform.Artefacts (artefact storage abstraction) +/tests + /TestMining.Platform.Core.Tests + /TestMining.Platform.Integration.Tests + /TestMining.Platform.E2E.Tests +/docs + concept.md + requirements.md +``` + +## 18. Delivery Phasing + +### 18.1 Phase 1: Core Proof + +Phase 1 shall target: + +- Recording of navigation, click, fill, select, checkbox, and simple assertion-relevant actions +- Timeline UI +- Locator ranking +- Basic scenario persistence with immutable scenario version creation +- C# Playwright generation +- Basic replay + +### 18.2 Phase 2: Robustness + +Phase 2 shall add: + +- Assertion inference +- Variable extraction +- Regex and partial matching +- Scoped locators +- Replay diagnostics +- Screenshot capture per important step + +### 18.3 Phase 3: Product Hardening + +Phase 3 shall add: + +- Login and session bootstrap management +- Export packaging +- Trace viewer integration +- Advanced healing review workflow and approval ergonomics +- Scenario history comparison and version-diff views + +### 18.4 Phase 4: AI Assistance + +Phase 4 may add: + +- Scenario naming suggestions +- Assertion suggestion assistance +- Healing proposal assistance +- Page object refactoring suggestions + +### 18.5 Phase Exit Criteria + +A phase shall not be declared complete until the criteria below are demonstrably met. These are conformance gates, not aspirational targets. + +Phase 1 exit: +- A user can authenticate, record a simple CRUD workflow end-to-end against a fixture web application, and see the persisted scenario. +- Locator candidates are ranked and persisted for every targetable step. +- Generated C# Playwright output compiles against the emitted helper library. +- Replay executes the generated scenario against the same fixture and reports pass/fail per step. +- URL allow-list enforcement (Section 12.4) and encryption of storage state (Section 12.5) are enforced. + +Phase 2 exit: +- Assertion inference produces at least one outcome-oriented assertion suggestion for every save/submit/navigate step in the Phase 1 fixture library. +- Variable classification is editable in the UI and round-trips through regeneration without loss. +- Replay diagnostics bundles include screenshots at each failure, locator resolution attempts, and a failure category. + +Phase 3 exit: +- Login bootstrap options (Section FR-ADM-002) are demonstrable end-to-end. +- Healing review workflow can approve a deterministic locator healing proposal and create a new immutable scenario version referencing the originating replay run. +- Artefact retention cleanup (Section 7.4) runs on schedule and is observable. + +Phase 4 exit (if undertaken): +- Any AI-assisted suggestion is always advisory, never auto-applied, and is labelled as AI-sourced in the UI and audit trail. +- Deterministic diagnostics continue to run and are presented alongside any AI suggestion. + +## 19. Acceptance Criteria + +The implementation shall be considered to satisfy this specification only when all of the following are demonstrably true and covered by at least one automated test traceable to the cited requirement identifiers. + +1. A user can record a real browser workflow through the UI and the system persists a structured scenario. [FR-REC-001, FR-REC-004, FR-REC-010] +2. The scenario can be reviewed, edited, and approved without direct database manipulation. [FR-AUTH-001 through FR-AUTH-006] +3. The system generates readable C# Playwright artefacts from the approved scenario. [FR-GEN-001 through FR-GEN-010] +4. The generated or scenario-derived replay executes through Playwright and produces useful diagnostics. [FR-REP-001, FR-REP-002, FR-REP-003] +5. Locator candidates are ranked and persisted rather than reduced to a single opaque selector. [FR-INF-004, FR-INF-005] +6. Sensitive values can be masked and remain masked across previews and logs, and are encrypted at rest where stored. [FR-REC-007, 12.3, 12.5] +7. Assertions are outcome-oriented and can be approved or rejected before generation. [FR-INF-007, FR-AUTH-003] +8. A locator drift failure can produce a deterministic healing proposal with evidence and require human approval before altering a scenario in v1. [FR-HEAL-001 through FR-HEAL-005] +9. The application runs on ASP.NET Core with a Blazor Server frontend and a PostgreSQL-backed persistence layer, with SQL Server pluggability preserved as a deferred option. [6.1, 6.2] +10. Generated output remains reproducible bit-for-bit (modulo timestamps declared as non-deterministic) from scenario data plus generation profile plus template version. [FR-GEN-001, FR-GEN-010] +11. Recording and replay refuse to start against target URLs not present on the administrator-managed allow-list. [12.4] +12. Observability emits the identifiers listed in 13.1 and permits verification of the performance targets in 15.2. + +## 20. Risks and Constraints + +### 20.1 Known Challenging Target Apps + +The product shall acknowledge reduced reliability for: + +- Canvas-heavy applications +- Highly virtualised lists or grids +- Poorly accessible React or SPA interfaces +- UIs with unstable text and random generated IDs +- Drag-and-drop heavy workflows +- Rich text editors + +### 20.2 Product Response to Poor Testability + +When recording quality is degraded by target-application design, the platform should surface actionable recommendations such as: + +- Add stable `data-testid` attributes +- Improve ARIA labels +- Expose predictable success messages +- Add stable row identifiers +- Reduce dependence on random element IDs + +## 21. Open Design Decisions + +The following decisions remain implementation-level choices unless later locked by an approved architecture note: + +- Whether Roslyn, Scriban, or a hybrid is the default generator engine +- Whether artefact storage is local filesystem first or abstracted for object storage from day one +- Whether replay executes in-process or via a dedicated background worker boundary +- Which authentication provider is used for the host application +- Which database provider is used as the primary development default + +## 22. Build, Test, and Delivery Requirements + +### 22.1 Repository Boundaries + +The test mining platform shall be introduced into this repository as a new vertical slice using the `TestMining.Platform.*` naming convention described in Section 17. Existing Symphony projects (`src/Symphony.*`, `tests/Symphony.*`, `symphony_docs/`, and `SPEC.md`) are retained tooling assets and shall not be mutated by work targeting this specification unless a task explicitly requires it. + +### 22.2 Build System + +- The solution shall build with the repository's documented required .NET SDK version. Any SDK upgrade shall be an explicit, documented change. +- `dotnet restore`, `dotnet build`, and `dotnet test` shall succeed from a clean checkout with no unresolved warnings treated as errors in core projects. +- Projects shall enable nullable reference types and treat analyzer warnings as errors where practical. + +### 22.3 Continuous Integration + +The repository shall include a CI pipeline (GitHub Actions is the expected host) that, on every pull request touching the platform: + +1. Restores and builds the full solution. +2. Runs unit and integration tests, including PostgreSQL-backed integration tests using a containerised database. +3. Runs generator snapshot tests (Section 14.4). +4. Publishes test results and code coverage summaries. +5. Fails the pipeline if any new secret-looking string appears in diffs or if retention/policy violations are detected by linting rules once established. + +### 22.4 Recorder Script Delivery + +The browser-side recorder script is a first-class build artefact: + +- It shall live in source control under a clearly named project directory and shall be built deterministically as part of the platform build. +- Its version shall be pinned and surfaced in captured recordings as `RecorderVersion` metadata so replay and healing can detect incompatible captures. +- It shall be injected via Playwright-supported initialisation hooks. Runtime fetching of the recorder from an external origin is prohibited. + +### 22.5 Test Layering and Flake Budget + +- Unit tests shall be the default: deterministic, no network, no browser. +- Integration tests shall exercise real infrastructure (PostgreSQL, Playwright against fixture web applications shipped with the repo). +- End-to-end tests shall be kept small in number and tagged so they can be excluded from fast developer loops. +- Any test that becomes flaky shall be either fixed or quarantined within one working day; persistent quarantine is not acceptable. + +## 23. Implementer Guardrails + +These guardrails exist to prevent the most likely failure modes of agentic or human implementers working on this specification. + +### 23.1 Prohibited Actions + +1. Writing captured credentials, cookies, storage states, tokens, or any value classified as `Sensitive` to logs, screenshots metadata, generation previews, or generated code. +2. Launching a Playwright browser context against a target URL that does not match the configured allow-list. +3. Introducing provider-specific SQL or raw ADO.NET calls into domain or application layers. Provider-specific code shall live in clearly isolated infrastructure components. +4. Editing generated source files by hand and checking the result in. Generated output shall be reproduced from scenario data and templates only. +5. Promoting an AI-assisted suggestion to a runtime-critical path. AI may advise; deterministic logic decides. +6. Deleting or restructuring Symphony-related files while working on test mining platform tasks. +7. Silently relaxing any non-functional requirement in Section 15, security requirement in Section 12, or acceptance criterion in Section 19. + +### 23.2 Required Behaviours + +1. Cite the requirement identifier (e.g. `FR-REC-004`, `FR-HEAL-003`) that motivates a code change in the pull request description. +2. For any behaviour change, update or add a conformance test traceable to the cited requirement. +3. Use feature-flagged rollout for anything that touches healing auto-apply, AI assistance, or cross-environment configuration. +4. When a requirement is ambiguous, raise a clarification rather than guessing. Record the resolution in this document or in an architecture decision record. + +### 23.3 Pull Request Discipline + +1. Pull requests shall be scoped to a single vertical slice. Drive-by refactors, especially across the Symphony/test-mining boundary, shall be split into separate changes. +2. Every PR shall identify new or changed artefact retention characteristics, new captured data fields, or new outbound network calls. +3. Secret-scanning and dependency vulnerability checks shall run before merge. + +## 24. Final Requirement Statement + +This repository’s new application shall be a C#-based semantic test mining platform built on ASP.NET Core, Blazor Server, Microsoft.Playwright for .NET, and PostgreSQL or SQL Server. It shall record browser interactions, infer structured scenarios, rank resilient locators, generate maintainable Playwright C# tests, replay them with strong diagnostics, and support deterministic healing while keeping structured scenarios as the enduring source of truth. diff --git a/docs/security-plan.md b/docs/security-plan.md new file mode 100644 index 0000000..213901c --- /dev/null +++ b/docs/security-plan.md @@ -0,0 +1,299 @@ +# Test Mining Platform Security Plan + +## 1. Purpose + +This document defines the implementation security plan for the platform described in [docs/requirements.md](./requirements.md), with emphasis on Sections 12, 16.4, 19, 22.3, and 23. + +This application handles sensitive material by construction, including cookies, storage state, captured inputs, target URLs, generated artefacts, and operator identities. Security controls must therefore be built into the default architecture rather than added later. + +## 2. Security Objectives + +1. Prevent unauthorised access to the platform and its artefacts. +2. Prevent secret leakage to logs, previews, diagnostics, and generated code. +3. Constrain browser automation to approved target URLs and isolated sessions. +4. Preserve auditable traceability for privileged and source-of-truth-changing actions. +5. Keep deterministic product features available even when optional AI or convenience features are disabled. + +## 3. Requirement Mapping + +Primary source requirements: + +- Section 11 configuration validation +- Section 12.1 through 12.7 +- Section 13 observability +- Section 16.4 operational safety +- Section 19 acceptance criteria 6 and 11 +- Section 22.3 CI secret scanning +- Section 23.1 prohibited actions + +## 4. Trust Boundaries + +Primary trust boundaries in v1: + +1. Browser under test vs platform host +2. Browser recorder script vs server-side recording engine +3. Blazor UI connection vs authenticated backend APIs +4. Application services vs persistence and artefact storage +5. Platform operators vs administrator-only configuration surfaces +6. Generated output vs canonical scenario storage + +Each cross-boundary interaction should be explicit, authenticated where applicable, validated, and logged without leaking sensitive payloads. + +## 5. Authentication and Authorization + +### 5.1 Authentication + +The platform shall require authenticated access outside public local development cases, consistent with Section 12.1. + +Recommended v1 approach: + +- local development: developer-friendly local auth toggle +- shared environments: external identity provider via ASP.NET Core authentication +- production-like environments: enforced external identity provider and secure cookie/session configuration + +### 5.2 Authorization + +Minimum role model: + +- `Administrator` +- `Author` +- `Viewer` + +Control expectations: + +- `Viewer` cannot start recording, edit scenarios, generate, replay, approve healing, or export sensitive artefacts. +- `Author` can perform workflow actions but cannot change environment security policy. +- `Administrator` can manage allow-lists, masking rules, retention, auth bootstrap settings, and key rotation workflows. + +## 6. Sensitive Data Classification + +Data should be classified at minimum as: + +- Public operational metadata +- Internal non-sensitive workflow data +- Sensitive captured data +- Secret authentication material + +Examples that must be treated as sensitive or secret: + +- credentials +- cookies +- Playwright storage state +- tokens +- manually marked sensitive input values +- scenario variables classified as `Sensitive` + +## 7. Core Security Controls + +### 7.1 Target URL Allow-List Enforcement + +Required by Section 12.4. + +Control plan: + +- store allow-list entries per environment +- validate requested target URL server-side before browser launch +- reject launch if URL does not match the selected environment policy +- audit every allow-list change + +Implementation note: + +This must happen before any Playwright browser context is created. + +### 7.2 Browser Session Isolation + +Required by Section 12.6 and `FR-REP-005`. + +Control plan: + +- create isolated browser contexts for recording and replay by default +- never share mutable session state across runs unless explicitly configured +- clean up contexts, temp files, traces, and storage state after completion or failure + +### 7.3 Secret Protection and Masking + +Required by Sections 12.2, 12.3, 12.5 and `FR-REC-007`. + +Control plan: + +- apply masking rules before persistence for previewable/raw event fields +- never log sensitive raw values +- never render secret values in UI previews +- never emit sensitive plaintext into generated source +- replace secret-backed values with secure variable references in generator output +- if a field is reclassified as sensitive after recording, future previews and regenerated output must use the updated masking rule + +### 7.4 Encryption at Rest + +Required by Section 12.5. + +Encrypt at rest: + +- Playwright storage state +- cookies +- stored authentication material +- sensitive scenario variables +- any captured raw payload fields configured for encryption + +Key management expectations: + +- keys loaded from secure deployment-time secret provider +- keys never committed to source control +- support controlled key rotation with re-encryption job +- re-encryption operations must be auditable and scoped so failed runs can be resumed safely + +### 7.5 Audit Logging + +Required by Section 12.7 and `FR-ADM-003`. + +Audit at minimum: + +- logins and access failures +- recording start/stop/cancel +- scenario version creation +- assertion approval/rejection where material +- generation export +- replay trigger +- healing approval/rejection +- configuration changes +- retention cleanup deletions +- allow-list changes +- authentication bootstrap configuration changes + +## 8. Secure Design by Capability + +### 8.1 Recording + +Threats: + +- recording against disallowed targets +- secret capture in raw payloads +- recorder transport spoofing or malformed payloads + +Controls: + +- allow-list gate +- strict payload validation +- masking before persistence +- authenticated session ownership checks + +### 8.2 Scenario Authoring + +Threats: + +- exposure of masked values in editing surfaces +- unauthorized source-of-truth changes + +Controls: + +- field-level masking in inspector and previews +- optimistic concurrency +- role-based edit permissions +- audit trail for version creation + +### 8.3 Generation + +Threats: + +- secret leakage to generated code +- export of unauthorised artefacts + +Controls: + +- generator redaction rules +- manifest review before export +- export authorization checks +- secret scanning in CI for generated fixtures/templates + +### 8.4 Replay and Healing + +Threats: + +- reusing mutable session state unsafely +- healing proposals altering source of truth without approval +- AI suggestions obscuring deterministic evidence + +Controls: + +- isolated replay context +- approval gate for persisted healing +- explicit provenance and evidence display +- AI disabled or advisory only by policy + +## 9. Logging and Telemetry Restrictions + +Structured logs must include correlation identifiers from Section 13.1, but must not contain: + +- tokens +- cookies +- storage state blobs +- secret values +- full sensitive captured field payloads + +Recommended practice: + +- structured properties for IDs and categories only +- masked previews when troubleshooting requires representative value shape +- explicit log sanitization tests +- do not place secret-bearing values in exception messages or validation summaries that may later be logged + +## 10. Secure Configuration Plan + +Configuration controls: + +- fail startup on missing required connection strings, auth config, or key references +- validate admin-managed policies before save +- separate secret configuration from ordinary app settings +- prevent unauthorised edits to allow-lists, masking policies, or retention policy +- distinguish environment-level immutable configuration from runtime-editable administrative settings + +## 11. CI/CD Security Controls + +Required by Section 22.3 and 23.3. + +Pipeline controls: + +- dependency vulnerability scanning +- secret scanning on diffs +- tests for masking and encryption rules +- policy/lint checks for retention and security-sensitive configuration + +Recommended additions: + +- SBOM generation +- container image scanning if container deployment is adopted +- restricted deployment secrets by environment + +## 12. Security Testing Plan Summary + +Security-specific tests should cover: + +- allow-list rejection +- sensitive-field masking at record and preview time +- encrypted persistence of storage state and sensitive variables +- authorization failures for admin-only actions +- audit event creation for privileged actions +- replay isolation guarantees +- healing approval enforcement + +Detailed testing ownership is defined in [docs/testing-plan.md](./testing-plan.md). + +## 13. Incident and Operational Response + +Operational readiness should include: + +- ability to revoke or rotate encryption keys +- ability to disable recording or replay by environment +- audit review of recent exports and configuration changes +- retention cleanup monitoring +- procedure for purging incorrectly retained sensitive artefacts + +## 14. Non-Negotiable Security Rules + +The following must never be relaxed in v1: + +1. No secret values in logs, previews, or generated code. +2. No recording or replay outside the configured allow-list. +3. No persisted healing changes without explicit approval. +4. No runtime-critical dependency on AI suggestions. +5. No key material in source control. diff --git a/docs/technical-specification.md b/docs/technical-specification.md new file mode 100644 index 0000000..e34fb7a --- /dev/null +++ b/docs/technical-specification.md @@ -0,0 +1,471 @@ +# Test Mining Platform Technical Specification + +## 1. Purpose + +This document translates [docs/requirements.md](./requirements.md) into an implementation-oriented technical specification for the new test mining platform described in the repository [README.md](../README.md). + +It is subordinate to `docs/requirements.md`. If this document conflicts with the requirements, the requirements win. + +## 2. Source Inputs + +- Product source of truth: `docs/requirements.md` +- Repository framing: `README.md` +- Repository boundary guidance: Requirements Sections 17 and 22.1 + +## 2.1 Decision Status Legend + +To remove ambiguity, this document uses the following terms consistently: + +- `Locked`: directly required by `docs/requirements.md` for v1 +- `Recommended`: strong implementation guidance that fits the requirements and should be followed unless an ADR records a different choice +- `Deferred`: intentionally left open by the requirements and should be resolved by ADR before implementation hardens + +## 3. Product Intent + +The system is a semantic test mining and stabilisation platform, not a raw click recorder. Its durable source of truth is the structured `Scenario` model, and generated Playwright code is a reproducible derived artefact. + +This technical plan primarily supports: + +- recording guided browser workflows +- enriching captured events with DOM, accessibility, and runtime context +- compiling recordings into immutable scenario versions +- generating readable C# Playwright output +- replaying scenarios with diagnostics +- producing deterministic healing suggestions under human approval + +Relevant requirements: + +- Sections 1, 2, 5, 7, 8, 18, 19, 24 +- `FR-REC-*`, `FR-INF-*`, `FR-AUTH-*`, `FR-GEN-*`, `FR-REP-*`, `FR-HEAL-*`, `FR-ADM-*` + +## 4. Solution Shape + +The implementation should introduce a new vertical slice using the `TestMining.Platform.*` naming convention from Section 17 and must not repurpose retained `Symphony.*` assets. + +Proposed solution structure: + +```text +/src + /TestMining.Platform.Host + /TestMining.Platform.Core + /TestMining.Platform.Application + /TestMining.Platform.Recording + /TestMining.Platform.Analysis + /TestMining.Platform.Generation + /TestMining.Platform.Replay + /TestMining.Platform.Healing + /TestMining.Platform.Persistence + /TestMining.Platform.Artefacts + /TestMining.Platform.Contracts + /TestMining.Platform.Recorder +/tests + /TestMining.Platform.Core.Tests + /TestMining.Platform.Application.Tests + /TestMining.Platform.Integration.Tests + /TestMining.Platform.Generator.Tests + /TestMining.Platform.E2E.Tests + /TestMining.Platform.Fixtures +``` + +Notes: + +- `Host` owns ASP.NET Core hosting, Blazor Server UI, DI, authentication, SignalR, HTTP endpoints, and background services. +- `Core` owns domain entities, enums, invariants, scenario contracts, and interfaces. +- `Application` owns use-case orchestration, commands, queries, validation, and DTO mapping. +- `Recording`, `Analysis`, `Generation`, `Replay`, and `Healing` own the product capability implementations. +- `Persistence` owns EF Core, PostgreSQL implementation, migrations, repositories, and provider abstractions. +- `Artefacts` owns storage abstraction and retention cleanup. +- `Contracts` owns shared API and SignalR contracts to keep host/UI boundaries explicit. +- `Recorder` owns the browser-side recorder script and deterministic build output required by Section 22.4. + +## 5. Technology Decisions + +Locked from requirements: + +- Backend: ASP.NET Core +- UI: Blazor Server +- Browser automation: Playwright for .NET +- ORM: EF Core +- Primary database for v1: PostgreSQL +- Future secondary provider: SQL Server +- Code generation: Roslyn, Scriban, or hybrid + +Recommended defaults for v1: + +- ASP.NET Core server app with integrated Blazor Server host +- EF Core migrations targeting PostgreSQL first +- Scriban templates for first-pass generation readability and snapshot testing +- Roslyn reserved for later structural generation/refactoring needs +- Filesystem artefact storage behind an abstraction so object storage can be added later + +Deferred by requirements and requiring ADRs before hardening: + +- exact authentication provider +- exact generator implementation split between Scriban and Roslyn +- whether replay remains in-process in v1 or moves behind a worker boundary +- how much draft scenario state is normalized versus document-shaped + +## 6. Architecture Principles + +Derived from requirements Sections 5, 6, 15, 16, 22, and 23: + +1. `Scenario` and immutable `ScenarioVersion` are the canonical business artefacts. +2. Generated code is never edited as the source of truth. +3. Browser orchestration, inference, generation, replay, healing, persistence, and UI remain separate modules. +4. Deterministic logic is mandatory on runtime-critical paths. +5. Sensitive data is masked or encrypted before persistence and before display. +6. Long-running operations are observable, resumable where reasonable, and auditable. +7. Every significant workflow is traceable to user, scenario version, replay run, and artefacts. + +## 7. Logical Components + +### 7.1 Host and UI + +Responsibilities: + +- route handling +- Blazor Server pages/components +- authentication and authorization +- internal HTTP APIs +- SignalR hubs for recording/replay progress +- background cleanup and replay workers + +Key requirement links: + +- Sections 5.2, 9, 10, 11, 12.1, 13, 16 + +### 7.2 Recording Engine + +Responsibilities: + +- start browser contexts with validated settings +- inject recorder script via Playwright init hooks +- capture meaningful events and correlated Playwright observations +- incrementally persist recording progress + +Key requirement links: + +- `FR-REC-001` through `FR-REC-010` +- Sections 5.3.1, 10.4, 12.4, 12.6 + +### 7.3 Analysis Engine + +Responsibilities: + +- clean and normalise raw event streams +- infer semantic actions and widget types +- create, rank, and validate locator candidates +- propose assertion candidates and data strategies +- assign explainable confidence scores + +Key requirement links: + +- `FR-INF-001` through `FR-INF-009` + +### 7.4 Scenario Engine + +Responsibilities: + +- manage draft edits and immutable version commits +- enforce structural validation before generation +- preserve change summaries and version lineage + +Key requirement links: + +- Sections 7.1 through 7.5 +- `FR-AUTH-001` through `FR-AUTH-006` + +### 7.5 Generation Engine + +Responsibilities: + +- compile scenario version plus generation profile into a deterministic file set +- emit test code, runtime helpers, fixture/data types, and manifest +- persist manifest, checksum, template version, and helper compatibility metadata + +Key requirement links: + +- `FR-GEN-001` through `FR-GEN-010` +- Sections 5.3.3, 14.4, 19 + +### 7.6 Replay Engine + +Responsibilities: + +- execute scenario or generated artefact in isolated browser context +- stream step results and diagnostics +- classify failures +- support full, onward, and targeted reruns where valid + +Key requirement links: + +- `FR-REP-001` through `FR-REP-005` + +### 7.7 Healing Engine + +Responsibilities: + +- evaluate replay failures for deterministic recovery options +- produce evidence-backed healing suggestions +- require approval before persisted scenario changes +- create linked immutable scenario versions when healing is applied + +Key requirement links: + +- `FR-HEAL-001` through `FR-HEAL-005` + +### 7.8 Persistence and Artefacts + +Responsibilities: + +- EF Core entity mapping and repository access +- migrations and provider portability discipline +- encrypted data storage for sensitive material +- artefact storage, retention, and cleanup jobs +- audit trail persistence + +Key requirement links: + +- Sections 7, 11, 12.5, 12.7, 13.3, 16.4, 22.3 + +## 8. Domain Model Summary + +The data model should directly implement the entities in requirements Section 7.2: + +- `RecordingSession` +- `Scenario` +- `ScenarioVersion` +- `RecordedStep` +- `ElementSnapshot` +- `LocatorCandidate` +- `AssertionCandidate` +- `ReplayRun` +- `HealingSuggestion` +- `UserAccount` +- `GenerationArtifact` + +Additional supporting entities recommended for v1: + +- `ScenarioDraft` +- `ScenarioVariableDefinition` +- `ArtefactRecord` +- `AuditEvent` +- `EnvironmentConfiguration` +- `GenerationProfile` +- `MaskedFieldPolicy` +- `RetentionPolicy` +- `ReplayStepResult` + +Design rules: + +- Use application-assigned GUID/UUID keys. +- Use optimistic concurrency tokens for editable entities. +- Store immutable scenario content as versioned JSON plus normalized child tables where query value exists. +- Preserve audit metadata and UTC timestamps everywhere required by Section 7.2.0. +- Keep `ScenarioVersion` immutable after commit; any editable state lives in draft-specific storage only. +- Treat generated files, replay runs, and healing decisions as downstream artefacts that always reference a scenario version, never replace it. + +### 8.1 Retention and Cleanup Defaults + +The implementation should encode the Section 7.4 defaults directly in configuration and migrations-facing seed data where practical: + +- screenshots and DOM snapshots: at least 30 days retention, 10 MB default item cap +- Playwright traces: at least 14 days retention, 200 MB default item cap +- session logs and replay diagnostic bundles: at least 30 days retention +- generated source bundles: retained until the owning scenario is deleted + +Administrative overrides must remain auditable and server-enforced. + +## 9. Main Workflows + +### 9.1 Recording Workflow + +1. User creates recording request with scenario name, environment, target URL, browser, and auth bootstrap mode. +2. Server validates user permissions and allow-list before launching Playwright. +3. Recording engine creates isolated browser context and injects recorder. +4. Browser events stream to host through explicit transport. +5. Engine correlates events with navigation, network, dialog, popup, trace, and screenshot signals. +6. Incremental persistence stores session progress, steps, snapshots, and artefact references. +7. Stop action finalizes the recording and triggers initial scenario draft creation. + +### 9.2 Authoring Workflow + +1. User opens timeline workspace for a draft scenario. +2. UI shows step list, locator confidence, assertion suggestions, and variable strategy metadata. +3. User edits, suppresses, reorders, approves assertions, and marks sensitive or variable values. +4. Scenario validator enforces coherent order, locator presence, variable definitions, and masking rules. +5. Commit action creates a new immutable `ScenarioVersion`. + +### 9.3 Generation Workflow + +1. User selects approved scenario version and generation profile. +2. Generator compiles scenario structure into deterministic output. +3. Host stores manifest, checksum, generator version, template version, and helper version. +4. UI previews files and warnings before export. + +### 9.4 Replay and Healing Workflow + +1. User launches replay from scenario version or generation artefact. +2. Engine executes steps in isolated context and streams step states. +3. Failure categorizer assigns a replay failure class. +4. Healing engine evaluates persisted candidates and contextual anchors. +5. UI shows evidence-backed proposal. +6. Approval creates a linked healing record and new immutable scenario version. + +## 10. Internal API Surface + +The internal authenticated API should be versioned from day one, for example `/api/v1/...`. + +Suggested resource groups: + +- `/recordings` +- `/recordings/{id}/events` +- `/recordings/{id}/controls` +- `/scenarios` +- `/scenarios/{id}/draft` +- `/scenario-versions/{id}` +- `/scenario-versions/{id}/generate` +- `/replays` +- `/replays/{id}` +- `/healing-suggestions/{id}` +- `/artefacts/{id}` +- `/admin/environments` +- `/admin/generation-profiles` +- `/admin/retention-policies` + +API rules: + +- every mutating endpoint must use explicit resource identifiers +- validation failures must return actionable error details +- long-running operations must expose status retrieval and correlation IDs +- API contracts must avoid returning sensitive raw payloads even to authenticated clients by default + +SignalR channels should cover: + +- recording session state +- live step feed +- replay step stream +- generation status +- background cleanup notifications where useful + +## 11. Background Processing + +Recommended hosted services: + +- recording session supervisor +- replay execution queue/dispatcher +- artefact retention cleanup job +- encryption key rotation job runner +- diagnostics aggregation job + +Recommended v1 execution boundary: + +- keep replay execution in-process as a hosted service for v1 simplicity +- isolate the execution behind an application interface so it can move to a dedicated worker later without rewriting domain or UI layers + +These services should operate on explicit persisted state rather than in-memory-only state. + +## 12. Configuration Model + +Configuration sources must support Section 11: + +- `appsettings.*.json` +- environment variables +- secure secret providers +- database-backed admin settings + +Configuration areas: + +- connection strings +- artefact storage root/provider +- allowed target base URLs per environment +- browser defaults +- trace/video/network capture defaults +- masking rules +- retention windows and size limits +- generation defaults +- authentication provider configuration + +Validation rules: + +- environment-level invalid configuration fails startup +- runtime-editable invalid configuration fails save with actionable validation + +## 13. Security-Critical Technical Controls + +This technical specification depends on the detailed controls in [docs/security-plan.md](./security-plan.md). At minimum: + +- server-side allow-list check before browser launch +- isolated browser contexts by default +- encrypted storage state, cookies, and sensitive variables at rest +- masking before persistence for previewable fields +- no secret leakage to logs, previews, or generated code +- audit trail for security-sensitive actions + +## 14. Observability + +The host must emit structured logs and metrics around: + +- recording start/stop/failure +- generation start/complete/failure +- replay start/step/failure/complete +- healing proposal creation/approval/rejection +- artefact cleanup and retention deletions + +Every event should include the identifiers required by Section 13.1 where applicable. + +Metrics should be sufficient to verify: + +- recording startup latency +- event-to-timeline publication latency +- generation duration +- replay startup latency +- replay pass/fail rate by failure category +- healing proposal rate and approval rate + +## 15. Performance and Scaling Targets + +Implementation should be designed so Section 15.2 is measurable: + +- record launch readiness +- event-to-timeline latency +- generation duration +- replay startup overhead + +Recommended approach: + +- capture timings in application services and background workers +- expose metrics through OpenTelemetry-compatible instrumentation +- measure per environment and per browser kind + +## 16. Open Technical Decisions + +The following remain deferred and should be resolved in ADRs before implementation hardens: + +1. Generator default: Scriban, Roslyn, or hybrid +2. Draft editing persistence shape: JSON document, normalized tables, or mixed +3. Replay execution boundary: in-process hosted service or dedicated worker process +4. Authentication provider choice for host application +5. Artefact storage abstraction depth in v1 + +## 17. Recommended First Vertical Slices + +To align with Phase 1 in Section 18: + +1. Host shell plus authenticated Blazor layout and PostgreSQL persistence bootstrap +2. Recording session creation plus allow-list validation +3. Recorder transport and incremental persistence for navigation/click/fill/select +4. Timeline review UI with draft editing +5. Locator ranking and scenario version creation +6. Deterministic generation preview and export +7. Basic replay with pass/fail diagnostics + +## 18. Definition of Ready for Implementation + +Implementation should not start until these are agreed: + +- project layout and naming +- PostgreSQL as the active v1 provider +- generator approach for Phase 1 +- fixture web applications for automated testing +- authentication approach for non-local environments diff --git a/docs/testing-plan.md b/docs/testing-plan.md new file mode 100644 index 0000000..90083fe --- /dev/null +++ b/docs/testing-plan.md @@ -0,0 +1,329 @@ +# Test Mining Platform Testing Plan + +## 1. Purpose + +This document defines the verification strategy for the platform described in [docs/requirements.md](./requirements.md). It expands on Sections 14, 15.5, 18.5, 19, and 22.3 through 22.5. + +## 2. Testing Goals + +The test strategy must prove that: + +1. structured scenarios are persisted and versioned correctly +2. recorder and analysis pipelines transform browser interactions into meaningful scenario steps +3. generated Playwright output is deterministic and readable +4. replay diagnostics and healing workflows are trustworthy +5. security and masking rules are enforced continuously + +## 3. Test Principles + +Derived from requirements: + +- unit tests are the default +- integration tests use real infrastructure where it matters +- end-to-end tests are limited but representative +- flaky tests must be fixed or quarantined quickly +- every acceptance criterion must trace to automated coverage +- the PostgreSQL-backed path is the required v1 provider path in CI and local integration testing + +## 4. Repository Test Layout + +Recommended test projects: + +```text +/tests + /TestMining.Platform.Core.Tests + /TestMining.Platform.Application.Tests + /TestMining.Platform.Integration.Tests + /TestMining.Platform.Generator.Tests + /TestMining.Platform.E2E.Tests + /TestMining.Platform.Fixtures +``` + +Purpose: + +- `Core.Tests`: entities, invariants, domain services, deterministic ranking/scoring rules +- `Application.Tests`: command/query handlers, validation, workflow orchestration with mocked boundaries +- `Integration.Tests`: PostgreSQL, EF Core mappings, artefact storage, Playwright coordination, encryption/masking integration +- `Generator.Tests`: snapshot/approval verification of generated output +- `E2E.Tests`: thin end-to-end flows against shipped fixture apps +- `Fixtures`: sample web apps/pages covering required target patterns + +## 5. Coverage by Requirement Area + +### 5.1 Recording + +Coverage target: + +- `FR-REC-001` through `FR-REC-010` + +Test themes: + +- session creation validation +- allow-list enforcement before launch +- recorder init script injection +- event capture for meaningful event types +- sensitive field masking during capture +- pause/resume/stop behaviour +- incremental persistence recovery + +### 5.2 Analysis and Inference + +Coverage target: + +- `FR-INF-001` through `FR-INF-009` + +Test themes: + +- event stream normalisation +- semantic action inference +- widget recognition +- locator ranking order +- locator uniqueness validation +- variable classification rules +- assertion suggestion and fuzzy matching strategies +- confidence scoring explainability + +### 5.3 Scenario Authoring + +Coverage target: + +- `FR-AUTH-001` through `FR-AUTH-006` + +Test themes: + +- step editing rules +- locator candidate switching +- assertion approval workflows +- sensitive-data policy validation +- immutable scenario version creation +- draft vs committed version behaviour + +### 5.4 Generation + +Coverage target: + +- `FR-GEN-001` through `FR-GEN-010` + +Test themes: + +- deterministic file output +- readable code structure assertions +- helper compatibility metadata +- variable/fixture generation +- repository-friendly layout and manifest generation +- generator warnings for low-confidence decisions + +### 5.5 Replay and Healing + +Coverage target: + +- `FR-REP-001` through `FR-REP-005` +- `FR-HEAL-001` through `FR-HEAL-005` + +Test themes: + +- scenario replay pass/fail execution +- failure categorisation +- targeted rerun rules +- isolated replay contexts +- deterministic healing proposal creation +- human approval gate enforcement +- traceability from replay failure to approved healing version + +### 5.6 Administration and Security + +Coverage target: + +- `FR-ADM-001` through `FR-ADM-004` +- Sections 11, 12, 13, 16.4 + +Test themes: + +- environment configuration validation +- retention policy persistence +- authentication bootstrap configuration +- audit event creation +- encryption and masking enforcement +- log sanitization +- retention cleanup behaviour and artefact traceability + +## 6. Test Layers + +### 6.1 Unit Tests + +Scope: + +- no browser +- no database +- deterministic logic only + +Examples: + +- locator ranking priority +- scenario validation rules +- assertion inference based on synthetic signals +- code generation model transformations before rendering + +### 6.2 Integration Tests + +Scope: + +- real PostgreSQL +- EF Core migrations +- real artefact storage abstraction against local filesystem +- Playwright coordination against fixture pages where appropriate + +Examples: + +- migration application from clean database +- persistence of scenario versions and related steps +- encryption-at-rest round trip +- recorder event flow into persistent draft scenario +- replay diagnostics package creation +- retention cleanup deleting expired artefacts while preserving audit evidence + +### 6.3 End-to-End Tests + +Scope: + +- run the host app +- exercise core user journeys through the UI or internal APIs +- use representative fixture applications + +Required Phase 1 path: + +1. authenticate +2. record simple CRUD workflow +3. review timeline +4. commit scenario version +5. generate C# output +6. replay and inspect result + +## 7. Fixture Application Plan + +Based on Section 14.3, fixtures should cover: + +- simple forms +- editable grids +- modal dialogs +- login flows +- unstable/dynamic IDs +- toast notifications + +Recommended implementation: + +- small ASP.NET Core fixture apps or static pages served from test host +- deterministic seed data +- stable routes and reset hooks for test runs + +## 8. Generator Verification Strategy + +Required by Section 14.4. + +Recommended approach: + +- snapshot or approval tests for generated files +- normalize line endings and timestamp placeholders before comparison +- assert both full-file snapshots and structural invariants + +Structural invariants should include: + +- file names +- namespaces +- helper references +- absence of fixed sleeps by default +- redaction of sensitive values + +## 9. Performance and Reliability Testing + +Support Sections 15.1 through 15.3. + +Measure: + +- recording startup readiness +- event-to-timeline latency +- generation duration for 100-step scenarios +- replay startup overhead + +Recommended practice: + +- capture metrics during integration suites +- run periodic benchmark job outside fast PR loop +- keep thresholds visible but avoid fragile benchmark tests in ordinary CI + +CI should still fail when basic instrumentation needed to measure Section 15.2 is removed or broken. + +## 10. Security Testing + +Must include: + +- allow-list rejection tests +- authorization tests for admin-only settings +- secret redaction and masking tests +- encryption persistence tests +- no-secret-in-generated-output tests +- audit trail creation tests + +## 11. Traceability and Conformance + +Every requirement-backed feature should include: + +- test case IDs linked to `FR-*` requirements or section numbers +- location of automated coverage +- acceptance-criteria mapping + +The repository should maintain this mapping in [docs/requirements-traceability-matrix.md](./requirements-traceability-matrix.md). + +## 12. CI Pipeline Expectations + +Pull request CI should: + +1. restore packages +2. build the solution +3. run unit and integration tests +4. run generator snapshot tests +5. publish test results and coverage +6. run secret scanning and policy checks +7. run PostgreSQL-backed integration tests using a containerized database, per Section 22.3 + +Suggested command groups: + +- `dotnet restore` +- `dotnet build` +- `dotnet test` + +## 13. Exit Criteria by Phase + +### Phase 1 + +- record-to-scenario flow proven against fixture CRUD app +- locator candidates persisted +- generated output compiles +- replay reports per-step pass/fail +- allow-list and encryption rules tested + +### Phase 2 + +- assertion inference and variable classification round-trip tested +- replay diagnostics bundle tested +- screenshot-on-failure behaviour tested + +### Phase 3 + +- authentication bootstrap tested end to end +- healing approval flow creates new immutable version +- retention cleanup job tested and observable + +## 14. Conformance Matrix Maintenance + +When implementation begins, every new feature PR should add or update: + +- the linked `FR-*` identifiers +- the automated tests proving the requirement +- any changed acceptance-criteria evidence +- any new retained artefact or sensitive-data handling impact + +### Phase 4 + +- AI-assisted suggestions verified as advisory only +- deterministic diagnostics still present alongside AI output diff --git a/docs/ui-ux-plan.md b/docs/ui-ux-plan.md new file mode 100644 index 0000000..71e0072 --- /dev/null +++ b/docs/ui-ux-plan.md @@ -0,0 +1,349 @@ +# Test Mining Platform UI/UX Plan + +## 1. Purpose + +This document turns the UI-related requirements in [docs/requirements.md](./requirements.md) into a delivery plan for the Blazor Server experience. + +Primary requirement anchors: + +- Section 4 user roles and use cases +- Section 8.1 through 8.7 +- Section 9 user interface requirements +- Section 10.2 real-time updates +- Section 12.1 access control +- Section 15.6 accessibility + +## 2. UX Principles + +The UI should reflect the product principles from requirements Section 2.3: + +1. Semantic over raw technical detail +2. Structured scenario over generated code +3. Deterministic evidence before automation or AI suggestions +4. Review before approval +5. Diagnostics always available + +Practical interpretation: + +- show business-meaningful labels first, raw technical payload second +- surface confidence, evidence, and warnings wherever the system infers behaviour +- keep every destructive or source-of-truth-changing action explicit +- preserve continuity between recording, editing, generation, and replay +- keep the scenario editor as the primary authoring surface; generated code preview is a downstream review surface, not the editing source of truth + +## 3. User Roles and UX Focus + +- `Administrator`: environment configuration, policy management, audit visibility +- `Author`: recording, editing, approval, generation, replay, healing review +- `Viewer`: read-only review of scenarios, replay runs, and generated artefact metadata + +Permissions should shape visible actions, not just backend responses. + +Role visibility rules: + +- hide or disable actions the current role cannot complete +- explain why an action is unavailable when doing so helps workflow comprehension +- never show approval or export affordances that the current user is not authorized to invoke + +## 4. Primary Navigation Model + +Recommended top-level navigation: + +- Dashboard +- Recordings +- Scenarios +- Replays +- Artefacts +- Administration + +Recommended dashboard widgets: + +- recent recordings +- draft scenarios awaiting review +- recent replay failures +- healing proposals awaiting approval +- retention or configuration warnings + +## 5. Information Architecture + +### 5.1 Recording Workspace + +Supports `FR-REC-001` through `FR-REC-010` and Section 9.2. + +Core regions: + +- session setup panel +- browser status and connection state +- live step feed +- inline note and importance controls +- event detail drawer + +Critical states: + +- ready to start +- launching browser +- recording active +- paused +- stopping/finalizing +- failed with recovery guidance + +### 5.2 Timeline and Scenario Editor + +Supports `FR-AUTH-001` through `FR-AUTH-006` and Section 9.3. + +Core regions: + +- ordered step timeline +- step detail inspector +- locator candidates panel +- assertion suggestions panel +- variable classification panel +- validation summary banner + +Critical states: + +- draft dirty +- validation warning +- blocking validation error +- approval pending for suggested assertions +- ready to version +- version committed + +### 5.3 Code Preview Workspace + +Supports `FR-GEN-001` through `FR-GEN-010` and Section 9.4. + +Core regions: + +- file tree / manifest list +- generated code viewer +- warning panel for low-confidence decisions +- generation profile selector +- export summary + +Critical states: + +- no approved version selected +- generating +- generated successfully +- generation warning +- generation failed + +### 5.4 Replay Workspace + +Supports `FR-REP-001` through `FR-REP-005`, `FR-HEAL-001` through `FR-HEAL-005`, and Section 9.5. + +Core regions: + +- replay launch panel +- live step execution stream +- diagnostics sidebar +- failure evidence tabs +- healing proposal review card + +Critical states: + +- queued/starting +- running +- passed +- failed with diagnostics +- healing available +- healing approved/rejected + +### 5.5 Administration Area + +Supports `FR-ADM-001` through `FR-ADM-004`, Sections 11 and 12.7. + +Core regions: + +- environments and base URL allow-lists +- browser and replay defaults +- masking and sensitive-data rules +- retention policies +- authentication bootstrap options +- audit activity feed + +## 6. Screen-Level Plan + +### 6.1 Dashboard + +Purpose: + +- orient users quickly +- highlight work in progress and required action + +Recommended modules: + +- active recordings +- scenarios needing approval +- recent generations +- replay health summary +- outstanding healing reviews + +### 6.2 New Recording Flow + +User journey: + +1. enter scenario name +2. choose environment +3. enter target URL +4. choose browser and recording options +5. choose authentication bootstrap mode +6. pass server-side allow-list validation +7. start recording + +Validation expectations: + +- invalid URL format +- URL not in environment allow-list +- missing required environment settings +- unsupported authentication bootstrap selection + +### 6.3 Live Recording Screen + +Priority interactions: + +- pause/resume/stop without losing progress +- mark important or ignorable steps +- add inline notes at time of capture +- inspect current URL and recorder health + +Design note: + +The live feed should focus on meaningful user actions and collapse noise by default. + +### 6.4 Scenario Editor + +The editor should make system inference reviewable rather than magical. + +Recommended step row content: + +- sequence number +- semantic label +- step type badge +- locator confidence indicator +- assertion indicator +- variable/sensitive badges +- suppression state + +Recommended inspector tabs: + +- Overview +- Locator Candidates +- Assertions +- Data Strategy +- Context Evidence +- Notes and Audit + +Approval checkpoints in this screen: + +- suggested assertions remain visually distinct from approved assertions +- locator candidate changes should show previous and newly selected strategy +- sensitive-value classifications should update every preview surface immediately after save + +### 6.5 Generation Preview + +Users should see: + +- which scenario version is being generated +- which generation profile and template version were used +- file manifest with warnings +- source preview +- export destination summary + +Warnings should clearly separate: + +- low-confidence locator use +- unapproved suggestions excluded from output +- sensitive values replaced with secure references + +### 6.6 Replay and Healing + +Replay diagnostics should be layered: + +1. human-readable failure summary first +2. step-level timeline second +3. raw diagnostics and traces third + +Healing review should always show: + +- original locator +- proposed locator +- evidence for proposal +- confidence and deterministic basis +- explicit approval or rejection action + +Healing review must never imply automatic application in v1. The UI language should explicitly state that approval creates a new immutable scenario version. + +## 7. UX Rules for Long-Running Operations + +Because recording, generation, replay, export, and cleanup are asynchronous, the UI should: + +- use SignalR updates for live progress +- persist operation status so page reload does not lose state +- show elapsed time and current stage +- provide retry guidance when failures occur +- avoid blocking the entire app shell for operation-specific failures + +## 8. Accessibility Plan + +Required to support Section 15.6: + +- keyboard access for all recording, editing, approval, and replay actions +- semantic landmarks and headings +- live region announcements for recording/replay status changes +- sufficient contrast for confidence states and warnings +- non-color indicators for pass/warn/fail/confidence levels +- focus management when drawers, dialogs, or review panels open + +## 9. Responsive Behaviour + +The UI is desktop-first for v1, but should degrade gracefully. + +Recommended breakpoints: + +- desktop: multi-panel workflow surfaces +- tablet/small laptop: collapsible side panels +- narrow widths: stacked panels, reduced preview width, preserved key actions + +Do not hide critical validation, approval, or security warnings on smaller layouts. + +## 10. Design System Guidance + +Suggested component set: + +- status badge +- confidence pill +- timeline row +- evidence drawer +- split-pane code/diagnostic viewer +- validation summary banner +- audit event list + +Suggested state taxonomy: + +- neutral +- in-progress +- success +- warning +- blocking +- sensitive + +## 11. Source-of-Truth UX Rules + +To stay aligned with Sections 7.1 and 8.4: + +1. Users edit scenarios, not generated code. +2. Generated code views must clearly show the source scenario version and generation profile. +3. Replay and healing screens must always show which scenario version they are derived from. +4. Any action that changes persisted scenario behaviour must route through scenario versioning, not ad hoc direct mutation. + +## 12. UX Acceptance Checks + +The following checks should be true before UI slices are considered complete: + +1. An author can complete the Phase 1 recording-to-replay path without leaving the web UI. +2. Every inferred locator or assertion can be inspected with evidence before approval. +3. Sensitive values never appear in clear text in preview surfaces. +4. Long-running operations recover gracefully from refresh or reconnect. +5. Role-restricted actions are hidden or disabled with clear rationale.