From 1f27ecf448abdd5801a79642c388a024d7c77d22 Mon Sep 17 00:00:00 2001 From: Kalen Kennedy Date: Thu, 23 Apr 2026 11:23:27 +1000 Subject: [PATCH] Ship: docs and local validation planning updates --- README.md | 174 +- WORKFLOW.md | 2 +- docs/README.md | 38 + docs/implementation-roadmap.md | 174 + docs/requirements-traceability-matrix.md | 76 + docs/requirements.md | 3011 +++++++++-------- docs/security-plan.md | 301 ++ docs/sprint-plan.md | 304 ++ docs/technical-specification.md | 486 +++ docs/testing-plan.md | 351 ++ docs/ui-ux-plan.md | 365 ++ ...y.Infrastructure.Persistence.Sqlite.csproj | 23 +- 12 files changed, 3710 insertions(+), 1595 deletions(-) create mode 100644 docs/README.md create mode 100644 docs/implementation-roadmap.md create mode 100644 docs/requirements-traceability-matrix.md create mode 100644 docs/security-plan.md create mode 100644 docs/sprint-plan.md create mode 100644 docs/technical-specification.md create mode 100644 docs/testing-plan.md create mode 100644 docs/ui-ux-plan.md diff --git a/README.md b/README.md index 508d4d9..bc30766 100644 --- a/README.md +++ b/README.md @@ -1,88 +1,90 @@ -# Test Mining Platform - -This repository is being used to build a C#-based semantic test mining platform that records browser interactions, converts them into structured scenarios, and generates maintainable Playwright tests with replay and healing support. - +# Test Mining Platform + +This repository is being used to build a C#-based semantic test mining platform that records browser interactions, converts them into structured scenarios, and generates maintainable Playwright tests with replay and healing support. + The current source of truth for the planned product is [docs/requirements.md](/mnt/c/Users/kenne/Desktop/ReleasedGroup/2EndSquaredTesting/docs/requirements.md). The concept document that informed it is [docs/concept.md](/mnt/c/Users/kenne/Desktop/ReleasedGroup/2EndSquaredTesting/docs/concept.md). -## Product Summary - -The application described in this repository is not intended to be a raw click recorder. It is intended to be a semantic test mining and stabilisation platform that: - -- records real user workflows in a browser -- captures DOM, accessibility, navigation, and runtime context around meaningful actions -- transforms raw events into structured, intent-rich scenarios -- infers resilient locators, variable data strategies, and outcome-oriented assertions -- generates readable C# Playwright tests and supporting helpers -- replays generated scenarios and provides deterministic healing when selectors drift - -The core design rule is that structured scenarios are the source of truth. Generated code is a derived artefact and must remain reproducible from scenario data and generation settings. - -## Intended Stack - -The requirements currently define this implementation model: - -- Backend: ASP.NET Core -- Frontend: Blazor Server -- Database: PostgreSQL or SQL Server -- Browser automation: Microsoft.Playwright for .NET -- Code generation: Roslyn or Scriban templates -- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper if needed - -## Planned Capabilities - -The platform is expected to include these major areas: - -- Recording engine for guided browser-session capture -- DOM and accessibility analysis -- Scenario authoring and timeline review -- Locator ranking and assertion suggestion -- C# Playwright code generation -- Replay diagnostics -- Deterministic healing workflows -- Persistent storage for sessions, scenarios, artefacts, and replay history -- Administrative configuration for environments, browser settings, and generation defaults - -## Repository Guidance - -This repository currently contains existing Symphony-related files and scaffolding. Those files are being retained in the repository because Symphony is a tool used within this project context. - -Symphony guidance for users and agents: - -- Treat Symphony-related files as retained tool files, not as the application being described in `docs/requirements.md`. -- Do not rewrite, repurpose, or delete Symphony files unless a task explicitly asks for Symphony/tooling work. -- Keep Symphony files in the repository even while building the new application. -- New product work should align to `docs/requirements.md`, not the legacy Symphony README content that previously occupied this file. - -Examples of retained Symphony-related content include: - -- `SPEC.md` -- `IMPLEMENTATION_PLAN.md` -- `WORKFLOW.md` -- `src/Symphony.*` -- `tests/Symphony.*` -- `symphony_docs/` - -## Expected Solution Direction - -A likely repository shape for the new application is described in the requirements document and includes logical areas such as: - -- host and Blazor Server UI -- core domain and scenario contracts -- recording and Playwright orchestration -- analysis and inference -- code generation -- replay and healing -- persistence and artefact storage -- automated tests - -## Current Status - -The repository is in a planning and transition stage: - -- the requirements specification exists in `docs/requirements.md` -- the concept source exists in `docs/concept.md` -- the new application implementation may coexist with retained Symphony tooling content during development - -## Working Rule - -If there is a conflict between older Symphony-oriented repository documentation and the new product direction, treat `docs/requirements.md` as the authoritative description of the application to be built, while still preserving Symphony-related files as tooling assets unless explicitly instructed otherwise. +In addition to being production-capable, the application is expected to support a local, production-like developer environment so changes can be exercised safely before they are pushed. The Blazor Server UI is also expected to be built early enough that major workflows can be visually tested during development rather than only through backend or CLI flows. + +## Product Summary + +The application described in this repository is not intended to be a raw click recorder. It is intended to be a semantic test mining and stabilisation platform that: + +- records real user workflows in a browser +- captures DOM, accessibility, navigation, and runtime context around meaningful actions +- transforms raw events into structured, intent-rich scenarios +- infers resilient locators, variable data strategies, and outcome-oriented assertions +- generates readable C# Playwright tests and supporting helpers +- replays generated scenarios and provides deterministic healing when selectors drift + +The core design rule is that structured scenarios are the source of truth. Generated code is a derived artefact and must remain reproducible from scenario data and generation settings. + +## Intended Stack + +The requirements currently define this implementation model: + +- Backend: ASP.NET Core +- Frontend: Blazor Server +- Database: PostgreSQL or SQL Server +- Browser automation: Microsoft.Playwright for .NET +- Code generation: Roslyn or Scriban templates +- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper if needed + +## Planned Capabilities + +The platform is expected to include these major areas: + +- Recording engine for guided browser-session capture +- DOM and accessibility analysis +- Scenario authoring and timeline review +- Locator ranking and assertion suggestion +- C# Playwright code generation +- Replay diagnostics +- Deterministic healing workflows +- Persistent storage for sessions, scenarios, artefacts, and replay history +- Administrative configuration for environments, browser settings, and generation defaults + +## Repository Guidance + +This repository currently contains existing Symphony-related files and scaffolding. Those files are being retained in the repository because Symphony is a tool used within this project context. + +Symphony guidance for users and agents: + +- Treat Symphony-related files as retained tool files, not as the application being described in `docs/requirements.md`. +- Do not rewrite, repurpose, or delete Symphony files unless a task explicitly asks for Symphony/tooling work. +- Keep Symphony files in the repository even while building the new application. +- New product work should align to `docs/requirements.md`, not the legacy Symphony README content that previously occupied this file. + +Examples of retained Symphony-related content include: + +- `SPEC.md` +- `IMPLEMENTATION_PLAN.md` +- `WORKFLOW.md` +- `src/Symphony.*` +- `tests/Symphony.*` +- `symphony_docs/` + +## Expected Solution Direction + +A likely repository shape for the new application is described in the requirements document and includes logical areas such as: + +- host and Blazor Server UI +- core domain and scenario contracts +- recording and Playwright orchestration +- analysis and inference +- code generation +- replay and healing +- persistence and artefact storage +- automated tests + +## Current Status + +The repository is in a planning and transition stage: + +- the requirements specification exists in `docs/requirements.md` +- the concept source exists in `docs/concept.md` +- the new application implementation may coexist with retained Symphony tooling content during development + +## Working Rule + +If there is a conflict between older Symphony-oriented repository documentation and the new product direction, treat `docs/requirements.md` as the authoritative description of the application to be built, while still preserving Symphony-related files as tooling assets unless explicitly instructed otherwise. diff --git a/WORKFLOW.md b/WORKFLOW.md index cd91faf..2418b15 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -4,7 +4,7 @@ tracker: endpoint: https://api.github.com/graphql api_key: $GITHUB_TOKEN owner: releasedgroup - repo: nextmedia-manager-copilot + repo: 2EndSquaredTesting milestone: null include_pull_requests: true labels: [] diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..56b8f8e --- /dev/null +++ b/docs/README.md @@ -0,0 +1,38 @@ +# Test Mining Platform Documentation Index + +This folder now contains the planning and specification documents for the new test mining platform described by [requirements.md](./requirements.md). + +## Core Source Documents + +- [Requirements Specification](./requirements.md) +- [Concept Document](./concept.md) + +## Planning Documents + +- [Technical Specification](./technical-specification.md) +- [UI/UX Plan](./ui-ux-plan.md) +- [Security Plan](./security-plan.md) +- [Testing Plan](./testing-plan.md) +- [Implementation Roadmap](./implementation-roadmap.md) +- [Sprint Plan](./sprint-plan.md) +- [Requirements Traceability Matrix](./requirements-traceability-matrix.md) + +## Recommended Reading Order + +1. `requirements.md` +2. `technical-specification.md` +3. `ui-ux-plan.md` +4. `security-plan.md` +5. `testing-plan.md` +6. `implementation-roadmap.md` +7. `sprint-plan.md` +8. `requirements-traceability-matrix.md` + +## Usage Guidance + +- Treat `requirements.md` as the source of truth. +- Use the technical specification and roadmap to plan implementation slices. +- Use the UI/UX, security, and testing plans as non-optional design constraints. +- Use the traceability matrix to connect future code, tests, ADRs, and pull requests back to requirements. +- Treat all planning documents as subordinate to `requirements.md`; if a planning document and the requirements differ, update the planning document. +- Treat the local production-like developer environment and early UI delivery as non-optional delivery constraints, not optional polish. diff --git a/docs/implementation-roadmap.md b/docs/implementation-roadmap.md new file mode 100644 index 0000000..e2e23ed --- /dev/null +++ b/docs/implementation-roadmap.md @@ -0,0 +1,174 @@ +# Test Mining Platform Implementation Roadmap + +## 1. Purpose + +This roadmap converts the delivery phasing in [docs/requirements.md](./requirements.md) Section 18 into implementation slices that fit the repository and reduce delivery risk. + +## 2. Roadmap Principles + +1. Build vertical slices, not disconnected layers. +2. Preserve `Symphony.*` tooling assets untouched unless explicitly requested. +3. Prove the canonical `Scenario` workflow early. +4. Treat security and observability as phase-entry requirements, not polish. +5. Keep each slice traceable to `FR-*` requirements and acceptance criteria. + +## 3. Foundation Track + +This track should start before Phase 1 feature depth expands. + +Deliverables: + +- `TestMining.Platform.*` project scaffolding +- PostgreSQL persistence bootstrap and migrations pipeline +- authentication shell and role model +- internal API versioning convention +- SignalR infrastructure +- artefact storage abstraction +- local production-like developer environment bootstrap +- fixture app test harness +- baseline observability and audit primitives +- ADRs for generator choice, authentication provider, replay execution boundary, and draft persistence shape + +Requirement anchors: + +- Sections 6, 10, 11, 12.1, 13, 14, 16, 17, 22 + +## 4. Phase 1: Core Proof + +Objective: + +Prove the end-to-end path from recording to generated code to replay on a simple fixture application. + +Suggested slices: + +1. Local production-like developer startup workflow with Blazor UI and PostgreSQL +2. Application shell and navigation suitable for visual testing +3. Recording session creation plus allow-list validation +4. Playwright browser launch and recorder injection +5. Meaningful event capture for navigation, click, fill, select, and checkbox +6. Incremental recording persistence +7. Initial timeline UI with human-readable steps +8. Locator candidate creation and ranking +9. Draft scenario editing and immutable version creation +10. Deterministic C# generator for one profile +11. Basic replay execution and step-level result reporting + +Phase 1 exit should match Section 18.5 Phase 1 exit criteria. + +Phase 1 is not complete unless URL allow-list enforcement and encryption of stored auth/session material are demonstrably working, because they are part of the stated exit criteria rather than optional hardening. + +Phase 1 is also not complete unless a developer can run the real UI locally against a production-like stack shape and visually test the core workflow before push. + +## 5. Phase 2: Robustness + +Objective: + +Increase semantic quality and diagnostic usefulness without changing the core source-of-truth model. + +Suggested slices: + +1. assertion suggestion engine +2. variable classification editing +3. fuzzy/resilient assertion strategies +4. scoped locator support +5. replay failure screenshots +6. richer replay diagnostics packaging +7. confidence explanation surfaces + +Requirement anchors: + +- `FR-INF-006` through `FR-INF-009` +- `FR-AUTH-003`, `FR-AUTH-004` +- `FR-REP-002`, `FR-REP-003` + +## 6. Phase 3: Product Hardening + +Objective: + +Harden the system for multi-user and longer-lived operational use. + +Suggested slices: + +1. authentication bootstrap modes +2. export packaging workflow +3. trace viewer integration +4. healing approval workflow and version lineage +5. scenario history and diff views +6. retention cleanup scheduling and admin controls + +Requirement anchors: + +- `FR-ADM-002` +- `FR-HEAL-001` through `FR-HEAL-005` +- Sections 7.4, 12.7, 16.4 + +## 7. Phase 4: AI Assistance + +Objective: + +Add advisory AI features without weakening deterministic execution or approval rules. + +Suggested slices: + +1. AI-assisted naming suggestions +2. AI-assisted assertion suggestion refinement +3. AI-assisted healing proposal augmentation +4. page object refactoring suggestions + +Guardrails: + +- feature flagged +- disabled by default +- advisory only +- deterministic evidence always shown +- no mutation of persisted scenarios without the same approval and traceability rules required for deterministic healing + +## 8. Cross-Cutting Backlog + +These items should run across phases: + +- security hardening and audits +- performance instrumentation +- accessibility improvements in Blazor UI +- documentation updates +- CI pipeline maturation + +## 9. Recommended Work Breakdown Structure + +Suggested implementation order: + +1. scaffold projects and shared contracts +2. establish persistence, auth shell, and audit primitives +3. build recording pipeline +4. build timeline authoring and versioning +5. build generation +6. build replay +7. add diagnostics depth +8. add healing workflow +9. add advanced admin and AI-adjacent features + +## 10. Dependencies and Risks + +Key dependencies: + +- stable fixture applications for test coverage +- Playwright recorder script build strategy +- PostgreSQL dev/test environment availability +- final authentication provider selection + +Key risks: + +- over-investing in raw recording before scenario editing is solid +- leaking sensitive data through early diagnostics +- making generator output non-deterministic +- coupling replay too tightly to generated code instead of scenario model + +## 11. Milestone Definition + +Each roadmap slice should only be closed when it has: + +- linked requirement IDs +- passing automated coverage +- documented artefact/security impact +- structured logs and diagnostics where applicable +- no mutation of retained `Symphony.*` assets diff --git a/docs/requirements-traceability-matrix.md b/docs/requirements-traceability-matrix.md new file mode 100644 index 0000000..35bd931 --- /dev/null +++ b/docs/requirements-traceability-matrix.md @@ -0,0 +1,76 @@ +# Test Mining Platform Requirements Traceability Matrix + +## 1. Purpose + +This matrix links the requirements in [docs/requirements.md](./requirements.md) to the implementation planning documents added for the repository. + +Use this document to keep future code changes, ADRs, test cases, and pull requests traceable to the source requirements. + +## 2. Planning Document Index + +- [Documentation Index](./README.md) +- [Technical Specification](./technical-specification.md) +- [UI/UX Plan](./ui-ux-plan.md) +- [Security Plan](./security-plan.md) +- [Testing Plan](./testing-plan.md) +- [Implementation Roadmap](./implementation-roadmap.md) +- [Sprint Plan](./sprint-plan.md) + +## 3. Requirement Coverage Matrix + +| Requirement Area | Requirement Source | Primary Planning Docs | +| --- | --- | --- | +| Product intent and canonical scenario principle | Sections 1, 2, 7.1, 24 | Technical Specification, Implementation Roadmap | +| Scope and phase boundaries | Sections 3, 18, 21 | Technical Specification, Implementation Roadmap, Sprint Plan | +| Roles and primary use cases | Section 4 | UI/UX Plan, Technical Specification | +| Logical architecture and lifecycles | Sections 5, 17 | Technical Specification, Implementation Roadmap, Sprint Plan | +| Technology stack and provider strategy | Section 6 | Technical Specification | +| Domain entities and artefacts | Sections 7.2 through 7.5 | Technical Specification, Testing Plan | +| Recording requirements | `FR-REC-001` through `FR-REC-010` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan, Implementation Roadmap, Sprint Plan | +| Inference requirements | `FR-INF-001` through `FR-INF-009` | Technical Specification, UI/UX Plan, Testing Plan, Sprint Plan | +| Scenario authoring requirements | `FR-AUTH-001` through `FR-AUTH-006` | Technical Specification, UI/UX Plan, Testing Plan, Sprint Plan | +| Generation requirements | `FR-GEN-001` through `FR-GEN-010` | Technical Specification, UI/UX Plan, Testing Plan, Implementation Roadmap, Sprint Plan | +| Replay requirements | `FR-REP-001` through `FR-REP-005` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan, Sprint Plan | +| Healing requirements | `FR-HEAL-001` through `FR-HEAL-005` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan, Implementation Roadmap, Sprint Plan | +| Administration requirements | `FR-ADM-001` through `FR-ADM-004` | Technical Specification, UI/UX Plan, Security Plan, Testing Plan, Sprint Plan | +| UI requirements | Section 9 | UI/UX Plan, Technical Specification | +| API and real-time requirements | Section 10 | Technical Specification, UI/UX Plan | +| Configuration requirements | Section 11 | Technical Specification, Security Plan | +| Security and privacy requirements | Section 12 | Security Plan, Testing Plan, Technical Specification | +| Observability and diagnostics | Section 13 | Technical Specification, Security Plan, Testing Plan | +| Verification and testing requirements | Section 14 | Testing Plan | +| Non-functional requirements | Section 15 | Technical Specification, UI/UX Plan, Testing Plan | +| Deployment and operational safety | Section 16 | Technical Specification, Security Plan, Implementation Roadmap | +| Local production-like developer environment and visual workflow validation | Sections 9.1, 14.1, 16.2, 18.1, 19 | Technical Specification, UI/UX Plan, Testing Plan, Implementation Roadmap, Sprint Plan | +| Acceptance criteria | Section 19 | Testing Plan, Technical Specification, Implementation Roadmap, Sprint Plan | +| Risks and constraints | Section 20 | Technical Specification, Implementation Roadmap | +| Build, test, and delivery requirements | Section 22 | Testing Plan, Technical Specification, Security Plan | +| Implementer guardrails | Section 23 | Security Plan, Technical Specification, Implementation Roadmap | + +## 4. Acceptance Criteria Mapping + +| Acceptance Criterion | Requirements Reference | Planned Evidence | +| --- | --- | --- | +| Record a workflow and persist a structured scenario | `FR-REC-001`, `FR-REC-004`, `FR-REC-010` | Technical Specification workflow design, Testing Plan Phase 1 path | +| Review, edit, and approve through UI | `FR-AUTH-001` through `FR-AUTH-006` | UI/UX Plan scenario editor, Testing Plan scenario authoring coverage | +| Generate readable C# Playwright artefacts | `FR-GEN-001` through `FR-GEN-010` | Technical Specification generation engine, Testing Plan generator verification | +| Replay with useful diagnostics | `FR-REP-001`, `FR-REP-002`, `FR-REP-003` | Technical Specification replay engine, UI/UX Plan replay workspace, Testing Plan replay coverage | +| Rank and persist locator candidates | `FR-INF-004`, `FR-INF-005` | Technical Specification analysis engine, Testing Plan inference coverage | +| Mask and encrypt sensitive values | `FR-REC-007`, Sections 12.3, 12.5 | Security Plan controls, Testing Plan security coverage | +| Approve/reject outcome-oriented assertions | `FR-INF-007`, `FR-AUTH-003` | UI/UX Plan inspector design, Testing Plan assertion approval coverage | +| Healing is deterministic, evidenced, and approval-based | `FR-HEAL-001` through `FR-HEAL-005` | Security Plan and Technical Specification healing workflow, Testing Plan healing coverage | +| Stack is ASP.NET Core + Blazor Server + PostgreSQL | Sections 6.1, 6.2 | Technical Specification solution shape | +| Generated output remains reproducible | `FR-GEN-001`, `FR-GEN-010` | Technical Specification generation design, Testing Plan snapshot strategy | +| Allow-list blocks disallowed targets | Section 12.4 | Security Plan allow-list control, Testing Plan allow-list tests | +| Observability supports performance verification | Sections 13.1, 15.2 | Technical Specification observability design, Testing Plan performance instrumentation | +| Developers can validate the real UI locally before push | Sections 9.1, 14.1, 16.2, 19 | UI/UX Plan local visual testing expectations, Testing Plan local validation environment, Sprint Plan sprint exit criteria | + +## 5. Future Use + +When code work begins, this matrix should be extended with: + +- implementation project/file locations +- test case IDs +- pull request references +- ADR references for open decisions +- links to evidence for phase exit criteria diff --git a/docs/requirements.md b/docs/requirements.md index 92fbddb..1eccbb3 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -1,1323 +1,1330 @@ -# @EndSquareTesting Requirements Specification - -## 1. Executive Summary - -The application defined by this specification is not a raw click recorder. It is a semantic test mining and stabilisation platform. - -The platform shall: - -- Record browser sessions performed by a human user against a target web application. -- Observe DOM state, accessibility metadata, navigation changes, network outcomes, and runtime context around each meaningful interaction. -- Transform raw browser events into a structured, intent-rich scenario model. +# @EndSquareTesting Requirements Specification + +## 1. Executive Summary + +The application defined by this specification is not a raw click recorder. It is a semantic test mining and stabilisation platform. + +The platform shall: + +- Record browser sessions performed by a human user against a target web application. +- Observe DOM state, accessibility metadata, navigation changes, network outcomes, and runtime context around each meaningful interaction. +- Transform raw browser events into a structured, intent-rich scenario model. - Infer resilient selectors, variable data strategies, and business-level assertions. - Generate readable C# Playwright test code and supporting assets from the scenario model. - Replay generated tests and attempt deterministic healing when selectors or data drift. - Preserve enough artefacts, metadata, and diagnostics to let users review, edit, approve, and regenerate scenarios over time. - -The platform shall be built with the following solution model: - -- Backend: ASP.NET Core -- Frontend: Blazor Server -- Database: PostgreSQL -- Browser automation: Microsoft.Playwright for .NET -- Code generation: Roslyn or Scriban templates -- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper - -The single most important product decision is this: the system shall store structured scenarios as the source of truth, not generated code. Generated code is a derived artefact and must always be reproducible from persisted scenario data plus generation settings. - -## 2. Product Vision - -### 2.1 Problem Statement - -Teams frequently want fast automated coverage of real user workflows, but hand-writing end-to-end tests is expensive and generic recorders produce brittle output. Raw click streams, unstable CSS selectors, exact-text assertions, and timing-based playback produce tests that quickly fail after routine UI changes. - -The platform shall solve this by treating recording as a data capture step and test generation as a compilation step. It shall infer user intent, promote resilient selectors, classify dynamic data, and generate maintainable Playwright tests that align with how users experience the application. - -### 2.2 Product Goal - -Enable product owners, testers, developers, and analysts to record business workflows once, refine them in a guided UI, and produce resilient C# Playwright tests that can be replayed, diagnosed, and healed as applications evolve. - -### 2.3 Product Principles - -The product shall be designed around these principles: - -1. Semantic first. -The system shall prefer user-facing meaning such as roles, labels, accessible names, nearby headings, and business outcomes over raw DOM structure. - -2. Structured source of truth. -The scenario model shall be the canonical artefact from which tests, page objects, diagnostics, and healing recommendations are derived. - -3. Deterministic before intelligent. -The system shall rely on deterministic heuristics and Playwright-native behaviour first. AI-assisted suggestions may augment the system, but must not be the only execution path for runtime-critical flows. - -4. User review is part of the workflow. -The platform shall support manual review and editing before generated code is considered ready. - -5. Diagnostics are mandatory. -Every recording, generation, replay, and healing attempt shall preserve sufficient evidence for supportability and debugging. - -## 3. Scope - -### 3.1 In Scope - -The v1 platform shall support: - -- Guided browser session recording -- DOM and accessibility metadata capture -- Playwright-based environment observation -- Post-recording scenario review and editing -- Locator ranking and locator candidate persistence -- Assertion suggestion and manual approval -- Variable data extraction and classification -- C# Playwright code generation -- Generated helper library emission -- Replay execution and diagnostics +- Support a local, production-like developer environment so the full application can be exercised before changes are pushed or deployed. +- Prioritise delivery of a usable Blazor Server UI early enough that major workflows can be visually tested throughout development, not only after backend completion. + +The platform shall be built with the following solution model: + +- Backend: ASP.NET Core +- Frontend: Blazor Server +- Database: PostgreSQL +- Browser automation: Microsoft.Playwright for .NET +- Code generation: Roslyn or Scriban templates +- Desktop packaging later: .NET MAUI Hybrid or Electron wrapper + +The single most important product decision is this: the system shall store structured scenarios as the source of truth, not generated code. Generated code is a derived artefact and must always be reproducible from persisted scenario data plus generation settings. + +## 2. Product Vision + +### 2.1 Problem Statement + +Teams frequently want fast automated coverage of real user workflows, but hand-writing end-to-end tests is expensive and generic recorders produce brittle output. Raw click streams, unstable CSS selectors, exact-text assertions, and timing-based playback produce tests that quickly fail after routine UI changes. + +The platform shall solve this by treating recording as a data capture step and test generation as a compilation step. It shall infer user intent, promote resilient selectors, classify dynamic data, and generate maintainable Playwright tests that align with how users experience the application. + +### 2.2 Product Goal + +Enable product owners, testers, developers, and analysts to record business workflows once, refine them in a guided UI, and produce resilient C# Playwright tests that can be replayed, diagnosed, and healed as applications evolve. + +### 2.3 Product Principles + +The product shall be designed around these principles: + +1. Semantic first. +The system shall prefer user-facing meaning such as roles, labels, accessible names, nearby headings, and business outcomes over raw DOM structure. + +2. Structured source of truth. +The scenario model shall be the canonical artefact from which tests, page objects, diagnostics, and healing recommendations are derived. + +3. Deterministic before intelligent. +The system shall rely on deterministic heuristics and Playwright-native behaviour first. AI-assisted suggestions may augment the system, but must not be the only execution path for runtime-critical flows. + +4. User review is part of the workflow. +The platform shall support manual review and editing before generated code is considered ready. + +5. Diagnostics are mandatory. +Every recording, generation, replay, and healing attempt shall preserve sufficient evidence for supportability and debugging. + +## 3. Scope + +### 3.1 In Scope + +The v1 platform shall support: + +- Guided browser session recording +- DOM and accessibility metadata capture +- Playwright-based environment observation +- Post-recording scenario review and editing +- Locator ranking and locator candidate persistence +- Assertion suggestion and manual approval +- Variable data extraction and classification +- C# Playwright code generation +- Generated helper library emission +- Replay execution and diagnostics - Deterministic selector healing workflows - Persistent storage of sessions, scenarios, steps, artefacts, and replay history - Multi-user authenticated web administration experience through Blazor Server +- Local developer execution in a production-like stack shape suitable for pre-push and pre-deployment validation - Export of generated tests into repository-friendly file structures - -### 3.2 Out of Scope for v1 - -The following are explicitly out of scope unless later prioritised: - -- Full low-code test authoring without any review step -- Support for native mobile app automation -- AI-only locator resolution at execution time -- Canvas-heavy, game-like, or non-DOM-centric application automation as a first-class supported scenario -- Rich collaborative review workflows such as branching, inline comments, or merge conflict handling between multiple editors -- Fine-grained enterprise RBAC beyond foundational role separation -- General-purpose workflow orchestration unrelated to browser test mining -- Electron or .NET MAUI desktop packaging in the initial release - -### 3.3 Deferred but Anticipated Scope - -The architecture shall allow later addition of: - -- Page object refactoring generation -- CI/CD export packs and pipeline templates -- Browser extension-based capture -- AI-assisted scenario summarisation and healing proposals -- Team collaboration and scenario versioning -- Desktop packaging for offline-friendly operator experiences - -## 4. Target Users and Primary Use Cases - -### 4.1 User Roles - -The system shall recognise these primary personas: - -1. Test Author -Records workflows, reviews steps, approves assertions, and generates tests. - -2. Developer -Consumes generated code, integrates it into solution structures, and refines helpers or templates. - -3. QA Engineer -Validates workflow coverage, replay reliability, and failure diagnostics. - -4. Platform Administrator -Configures browser runners, storage, security, code-generation defaults, and environment connections. - -### 4.2 Core Use Cases - -The platform shall support these end-to-end use cases: - -1. Record a new business workflow from a live application. -2. Review and edit the recorded timeline before generation. -3. Mark inputs as fixed, variable, generated, lookup-driven, or sensitive. -4. Accept or reject suggested assertions. -5. Generate a readable Playwright C# test suite and helper files. -6. Replay the generated scenario against the target application. -7. Investigate replay failures using screenshots, traces, DOM context, and locator diagnostics. -8. Accept a healing suggestion and regenerate the derived code. - -## 5. High-Level System Overview - -### 5.1 Product Capabilities - -The application shall consist of these major capability areas: - -1. Recording -Browser session launch, event capture, instrumentation injection, and artefact creation. - -2. Interpretation -DOM intelligence, locator ranking, variable classification, widget recognition, and assertion inference. - -3. Scenario Authoring -Timeline editing, step suppression, annotation, data strategy configuration, and acceptance of inferred outputs. - -4. Generation -Transformation of scenario models into C# Playwright tests, helpers, fixtures, and optional page object scaffolding. - -5. Replay and Healing -Execution, diagnostics, candidate re-resolution, deterministic healing, and controlled regeneration. - -6. Administration -Environment configuration, browser configuration, template settings, project export settings, and user management. - -### 5.2 Logical Architecture - -The solution shall be decomposed into the following logical components: - -1. Blazor Server UI -- Hosts recording workflows, timeline editing, generation preview, replay diagnostics, and admin surfaces. - -2. ASP.NET Core Application Host -- Provides HTTP endpoints, SignalR backplane for live session updates, background workers, authentication, orchestration, and API surfaces for the UI. - -3. Recording Engine -- Launches Playwright browsers and coordinates injected capture scripts, event streams, screenshots, traces, and browser-side metadata collection. - -4. Capture and Analysis Engine -- Normalises raw browser events, enriches them with DOM context, identifies semantic widgets, ranks locators, and produces assertion candidates. - -5. Scenario Engine -- Stores, edits, versions, validates, and materialises structured scenarios. - -6. Code Generation Engine -- Produces C# Playwright test source files, helper classes, optional page objects, and configuration assets using Roslyn or Scriban. - -7. Replay and Healing Engine -- Replays scenarios or generated tests, captures outcomes, diagnoses failures, evaluates fallback locators, and records healing proposals. - -8. Persistence Layer -- Stores structured entities, audit history, artefact references, configuration, and operational logs in PostgreSQL or SQL Server. - -9. Artefact Storage -- Stores screenshots, traces, exported code bundles, DOM snapshots, and replay attachments on a filesystem or object-backed abstraction. - -## 5.3 Operational Lifecycles - -### 5.3.1 Recording Lifecycle - -The recording lifecycle shall follow this logical sequence: - -1. User creates a recording session with target URL and settings. -2. The backend provisions a Playwright browser context and applies configured bootstrap settings. -3. Recorder instrumentation is injected before or at page startup. -4. The user performs actions in the target application. -5. Raw browser events and Playwright observations are streamed to the backend. -6. The backend normalises events and persists session artefacts incrementally. -7. The user stops recording. -8. The backend finalises the session and generates an initial scenario draft for review. - -### 5.3.2 Scenario Authoring Lifecycle - -The authoring lifecycle shall support: - -1. Reviewing the recorded step timeline. -2. Removing or suppressing noise. -3. Promoting meaningful events into semantic actions. -4. Approving or rejecting inferred assertions. -5. Classifying test data and secrets. -6. Choosing generation profile settings. -7. Saving an immutable scenario version ready for generation or replay. - -### 5.3.3 Generation Lifecycle - -The generation lifecycle shall support: - -1. Selecting an approved scenario version. -2. Applying generation profile and template settings. -3. Producing output files and a manifest. -4. Storing a checksum and file inventory. -5. Presenting the generated output for preview and export. - -### 5.3.4 Replay and Healing Lifecycle - -The replay and healing lifecycle shall support: - -1. Selecting a scenario version or generated artefact for execution. -2. Launching an isolated Playwright replay context. -3. Executing steps while recording diagnostics. -4. Classifying failures when they occur. -5. Attempting deterministic healing when drift is suspected. -6. Presenting a proposal with supporting evidence. -7. Applying approved healing changes to a new scenario version or updating approved locator preferences under controlled rules. - -## 6. Mandatory Technology Requirements - -### 6.1 Application Stack - -The implementation shall use: - -- ASP.NET Core for backend hosting and service orchestration -- Blazor Server for the primary web UI -- Microsoft.Playwright for .NET for browser control, observation, replay, trace capture, and browser-context management -- EF Core for relational persistence -- PostgreSQL or SQL Server as the durable database -- Roslyn and/or Scriban for code generation - -### 6.2 Database Provider Strategy - -To make v1 deliverable without doubling persistence cost, the platform shall adopt the following strategy: - -- PostgreSQL shall be the **primary and only required provider for v1**. Local development, shared non-production, and production-like deployments shall all use PostgreSQL unless explicitly reconfigured. -- SQL Server shall be a **deferred-but-supported secondary provider**. The persistence layer shall be designed so that SQL Server can be added later without redesigning the domain or repositories, but v1 acceptance does not require a working SQL Server deployment. -- The persistence layer shall use EF Core abstractions and migrations. -- Provider-specific SQL shall be avoided unless isolated behind clearly bounded infrastructure components and gated by a provider capability check. -- Data types, indexing strategies, JSON storage, and concurrency rules shall be designed to remain portable to SQL Server. Use of PostgreSQL-only features (for example `jsonb` operators, array columns, partial indexes) is permitted in v1 provided the feature can be substituted or approximated on SQL Server later. -- Integration tests shall run against real PostgreSQL (for example Testcontainers) rather than an in-memory provider, because in-memory providers do not exercise migration or JSON behaviour. - -### 6.3 Browser Support - -The initial supported execution browser shall be Chromium through Playwright. The architecture shall not prevent later support for WebKit and Firefox, but those browsers are not required for v1 unless explicitly enabled by configuration. - -## 7. Information Architecture and Domain Model - -### 7.1 Canonical Principle - -The canonical persisted business artefact shall be `Scenario`. Generated tests, replay runs, and healing decisions shall reference the scenario version they were derived from. - -### 7.2 Core Entities - -The persistence model shall include, at minimum, these entities. - -#### 7.2.0 Common Entity Requirements - -Unless explicitly exempted, every persistent entity defined in this section shall carry: - -- A stable primary key (`Id`) that is globally unique (GUID/UUID) and assigned by the application, not the database. -- Audit fields: `CreatedAtUtc`, `CreatedByUserId`, `UpdatedAtUtc`, `UpdatedByUserId` (the last two may be nullable where the entity is immutable by design, such as `ScenarioVersion` and `RecordedStep`). -- A `RowVersion` (optimistic concurrency token) column on entities that are editable by more than one workflow, including `Scenario`, `RecordingSession`, `HealingSuggestion`, and administrative configuration entities. -- A soft-delete marker (`IsDeleted`, `DeletedAtUtc`, `DeletedByUserId`) on `Scenario`, `RecordingSession`, `GenerationArtifact`, and artefact records. Immutable versions (e.g. `ScenarioVersion`, `RecordedStep`) shall not support soft delete; deletion of a scenario shall be a cascade-tombstoning operation documented in the data model. -- Timestamps stored in UTC; the domain shall not depend on database-local clocks for ordering decisions beyond audit. - -Field lists in the subsections below name the entity-specific fields and may omit the common fields above for brevity. - -#### 7.2.1 RecordingSession - -Represents an interactive capture session started by a user. - -Required fields: - -- `Id` -- `Name` -- `TargetBaseUrl` -- `EnvironmentName` -- `Status` -- `StartedByUserId` -- `StartedAtUtc` -- `EndedAtUtc` -- `BrowserKind` -- `BrowserChannel` -- `Headless` -- `AuthenticationMode` -- `RecordAssertionsAutomatically` -- `MaskedFieldPolicyJson` -- `SettingsJson` - -#### 7.2.2 Scenario - -Represents a business workflow compiled from one or more recording sessions and maintained as the product source of truth. - -Required fields: - -- `Id` -- `Name` -- `Description` -- `Status` -- `PrimaryRecordingSessionId` -- `TargetApplicationName` -- `TargetBaseUrl` -- `EnvironmentName` -- `CreatedByUserId` -- `CreatedAtUtc` -- `UpdatedAtUtc` -- `CurrentVersionNumber` -- `DefaultGenerationProfileId` -- `TagsJson` - -#### 7.2.3 ScenarioVersion - -Represents an immutable version of the scenario structure used for generation or replay traceability. - -In-progress edits shall not mutate an immutable `ScenarioVersion`. Any draft editing workflow shall -store draft state separately and create a new immutable scenario version when changes are committed. - -Required fields: - -- `Id` -- `ScenarioId` -- `VersionNumber` -- `ParentScenarioVersionId` -- `CreatedAtUtc` -- `CreatedByUserId` -- `ChangeSummary` -- `StructureJson` -- `InferenceMetadataJson` -- `ApprovedForGeneration` - -#### 7.2.4 RecordedStep - -Represents a normalised action or assertion candidate in sequence order. - -Required fields: - -- `Id` -- `ScenarioVersionId` -- `Sequence` -- `StepType` -- `SemanticActionType` -- `DisplayName` -- `UserNote` -- `Status` -- `UrlBefore` -- `UrlAfter` -- `TimestampUtc` -- `DurationMs` -- `RawValue` -- `NormalisedValue` -- `VariableStrategy` -- `SensitivityClassification` -- `TargetElementSnapshotId` -- `ParentContainerSnapshotId` -- `ExpectedOutcomeJson` -- `GeneratedCodeHintJson` - -#### 7.2.5 ElementSnapshot - -Represents the known properties of an element at record or replay time. - -Required fields: - -- `Id` -- `CapturePhase` -- `RecordingSessionId` (nullable; populated for recording-time capture) -- `ReplayRunId` (nullable; populated for replay-time capture) -- `RecordedStepId` -- `TagName` -- `InputType` -- `Role` -- `AccessibleName` -- `LabelText` -- `Placeholder` -- `TextContent` -- `TitleAttribute` -- `NameAttribute` -- `ValuePreview` -- `TestId` -- `AriaAttributesJson` -- `DataAttributesJson` -- `CssPath` -- `XPath` -- `BoundingBoxJson` -- `VisibilityState` -- `EnabledState` -- `ParentContextJson` -- `SiblingContextJson` -- `WidgetKind` -- `DomHash` - -#### 7.2.6 LocatorCandidate - -Represents one potential locator strategy for a target element. - -Required fields: - -- `Id` -- `RecordedStepId` -- `StrategyType` -- `Expression` -- `ScopeExpression` -- `Priority` -- `Confidence` -- `UniquenessAtRecordTime` -- `VisibilityAtRecordTime` -- `Recommended` -- `FailureReasonIfRejected` - -#### 7.2.7 AssertionCandidate - -Represents a suggested or approved expected outcome. - -Required fields: - -- `Id` -- `RecordedStepId` -- `AssertionType` -- `ExpressionJson` -- `Priority` -- `Confidence` -- `SourceSignal` -- `Approved` -- `RejectedReason` - -#### 7.2.8 ReplayRun - -Represents one execution attempt of a scenario or generated test. - -Required fields: - -- `Id` -- `ScenarioId` -- `ScenarioVersionId` -- `ReplaySourceKind` -- `GenerationArtifactId` (nullable; populated when replay uses a generated artefact bundle) -- `Status` -- `StartedAtUtc` -- `EndedAtUtc` -- `TriggeredByUserId` -- `ExecutionEnvironment` -- `BrowserKind` -- `Headless` -- `FailureCategory` -- `FailureSummary` -- `DiagnosticsJson` - -#### 7.2.9 HealingSuggestion - -Represents a proposed change when replay detects drift. - -Required fields: - -- `Id` -- `ReplayRunId` -- `RecordedStepId` -- `SuggestionType` -- `CurrentFailureEvidenceJson` -- `ProposedLocatorJson` -- `DeterministicConfidence` -- `AiConfidence` -- `ApprovalStatus` -- `AppliedAtUtc` -- `AppliedByUserId` - -#### 7.2.10 UserAccount - -Represents an authenticated principal of the platform. Identity data may be federated from an external provider, but the platform shall maintain a local record to support authorship attribution, audit trails, and role assignment. - -Required fields: - -- `Id` -- `ExternalSubjectId` (nullable; populated when federated) -- `DisplayName` -- `Email` -- `Role` (at minimum one of: `Administrator`, `Author`, `Viewer`) -- `IsDisabled` -- `LastLoginAtUtc` - -All audit references in other entities (for example `StartedByUserId`, `CreatedByUserId`, `TriggeredByUserId`) shall resolve to `UserAccount.Id`. - -#### 7.2.11 GenerationArtifact - -Represents a generated bundle or file set. - -Required fields: - -- `Id` -- `ScenarioVersionId` -- `GeneratorKind` -- `GeneratorVersion` -- `TemplateProfile` -- `TemplateVersion` -- `OutputRoot` -- `ManifestJson` -- `CreatedAtUtc` -- `CreatedByUserId` -- `Checksum` - -### 7.3 Structured Scenario Contract - -The application shall define a first-class structured scenario contract that can represent: - -- Navigation steps -- Click actions -- Fill actions -- Select actions -- Checkbox and toggle interactions -- Keyboard actions -- Dialog interactions -- Grid and list interactions -- Widget-scoped semantic actions -- Assertions -- Control flow metadata such as waits inferred from user-facing events -- Variable definitions -- Data generation strategies -- Replay and healing hints - -The scenario contract shall be expressive enough that: - -- Tests can be regenerated without rerecording. -- Helper libraries can evolve independently of the raw recording. -- Multiple code-generation styles can be supported later. -- Healing decisions can update the scenario model without directly editing generated output. - -## 7.4 Artefact Model - -The application shall treat the following as first-class artefacts: - -- Screenshots -- Playwright traces -- DOM snapshots -- Session logs -- Generated source files -- Replay diagnostics packages - -Each artefact shall have: - -- A stable identifier -- A logical owner such as recording session, scenario version, or replay run -- Content type -- Retention classification -- Storage path or provider-specific handle -- Created timestamp -- Optional checksum -- A size-in-bytes record - -The platform shall enforce default retention and size ceilings so artefact storage does not grow unbounded in the absence of administrative action: - -- Screenshots and DOM snapshots: retained for at least 30 days after the owning session or replay run completes; individual artefact size cap of 10 MB unless an administrator raises it. -- Playwright traces: retained for at least 14 days; individual trace cap of 200 MB. -- Session logs and replay diagnostic bundles: retained for at least 30 days. -- Generated source bundles: retained until the owning scenario is deleted. - -Administrators may extend any retention window or raise any size cap through the configuration surface defined in Section 11. A background cleanup process shall delete artefacts that have exceeded their retention window and record the deletion in the audit trail. - -## 7.5 State Model Expectations - -The system shall define explicit statuses for at least: - -- Recording sessions -- Scenarios -- Scenario versions -- Replay runs -- Healing suggestions -- Generation artefacts - -Status transitions shall be explicit, validated, and auditable. - -## 8. Functional Requirements - -### 8.1 Recording Requirements - -#### FR-REC-001 Guided Recording Creation - -The system shall let a user start a recording by providing: - -- Scenario name -- Target application URL -- Optional environment label -- Browser selection -- Whether automatic assertion suggestions are enabled -- Authentication or session bootstrap mode - -#### FR-REC-002 Browser Session Launch - -The system shall launch a Playwright browser context for recording with configurable: - -- Headless or headed mode -- Viewport -- Storage state -- Trace settings -- Video settings, if enabled -- Network capture settings - -#### FR-REC-003 Early Instrumentation Injection - -The system shall inject recorder logic before application content loads wherever technically possible, using Playwright-supported initialisation hooks such as `AddInitScriptAsync`. - -#### FR-REC-004 Event Capture - -The recorder shall capture, at minimum, these event types when user meaningful: - -- Page navigation -- Click -- Double click when distinguishable -- Input -- Change -- Form submit -- Key press -- Dialog open and close -- File download start -- Popup open -- Checkbox or toggle change -- Select and autocomplete selection - -The recorder shall suppress noise events such as incidental mouse movement unless explicitly enabled for diagnostics. - -#### FR-REC-005 Element Context Capture - -For each meaningful event, the system shall capture contextual element metadata including, where available: - -- HTML tag -- Input type -- Role -- Accessible name -- Label text -- Placeholder -- Visible text -- Title attribute -- Name attribute -- Test id or equivalent stable data attribute -- Bounding box -- Enabled and visible state -- Nearby heading -- Parent form, region, modal, or table context -- Candidate CSS path -- Candidate XPath as a last-resort fallback - -#### FR-REC-006 Observer Correlation - -The recording engine shall correlate user events with surrounding Playwright-observed browser state including: - -- URL changes -- Page load phases -- Network requests and responses -- Console errors and warnings, if enabled -- Dialog lifecycle -- Download lifecycle -- Popup lifecycle -- Screenshots -- Trace segments - -#### FR-REC-007 Sensitive Data Handling - -The user shall be able to mark fields or values as sensitive during or after recording. Sensitive values shall be masked in persisted logs, previews, and generated artefacts unless the user explicitly authorises retention in a secure variable form. - -#### FR-REC-008 Recording Controls - -The UI shall allow the user to: - -- Start -- Pause -- Resume -- Stop -- Cancel -- Mark current step as important -- Mark current step as ignorable -- Add an inline note - -#### FR-REC-009 Recorder Transport - -The recorder shall use a reliable browser-to-host transport mechanism for event delivery. Preferred mechanisms include Playwright-exposed bindings or a similarly explicit channel. Console scraping may be used only as a bounded fallback or diagnostic technique, not as the primary durable transport contract. - -#### FR-REC-010 Incremental Persistence - -Long-running recording sessions shall persist meaningful progress incrementally so that unexpected host failures do not necessarily lose the entire session. - -### 8.2 Post-Processing and Inference Requirements - -#### FR-INF-001 Event Normalisation - -The system shall transform raw captured events into a cleaned event stream by: - -- Removing technical noise -- Merging related input bursts -- Folding navigation waits into surrounding intent -- Linking actions to the most likely observed outcomes - -#### FR-INF-002 Semantic Action Inference - -The system shall infer higher-order action meaning where possible, including but not limited to: - -- Open details page -- Save form -- Submit search -- Open modal -- Confirm dialog -- Select row in grid -- Apply filter -- Choose date -- Dismiss notification - -#### FR-INF-003 Widget Recognition - -The system shall recognise common UI widget patterns where feasible, including: - -- Forms -- Tables and grids -- Modals and dialogs -- Dropdowns and combo boxes -- Tabs -- Toast notifications -- Date pickers -- Autocomplete controls - -The recognised widget type shall be stored as metadata usable by generation and healing logic. - -#### FR-INF-004 Locator Ranking - -For each targetable step, the system shall create a ranked set of locator candidates. The ranking algorithm shall prioritise: - -1. Explicit test id or configured stable data attribute -2. Role plus accessible name -3. Label-based targeting -4. Visible text -5. Placeholder -6. Scoped parent-region plus child-role targeting -7. Stable attribute combinations -8. CSS selector fallback -9. XPath fallback - -#### FR-INF-005 Locator Validation - -Each locator candidate shall be validated at record time when possible for uniqueness, visibility, and scope confidence. - -#### FR-INF-006 Variable Classification - -The system shall support classifying entered or observed values into at least these categories: - -- Fixed test data -- Scenario variable -- Generated data -- Lookup value -- Sensitive value -- Incidental value that should not become an assertion target - -#### FR-INF-007 Assertion Suggestion - -The system shall infer candidate assertions from post-action signals including: - -- Success toast visibility -- URL pattern change -- Heading change -- Dialog closure -- Newly visible row, card, or detail view -- Relevant network success response -- Count changes where meaningful - -#### FR-INF-008 Fuzzy Outcome Strategies - -The system shall support resilience strategies for data-sensitive assertions including: - -- Exact text to contains matching -- Regex matching for identifiers -- Date and time range validation -- Row existence by key column -- Partial heading matching -- Minimum count assertions - -#### FR-INF-009 Confidence Scoring - -The system shall assign confidence scores to inferred locators, assertions, widget classifications, and healing suggestions. Confidence values shall be explainable and based on deterministic evidence where possible. - -### 8.3 Scenario Authoring Requirements - -#### FR-AUTH-001 Timeline Review - -The UI shall present recorded scenarios as an ordered timeline of meaningful steps with human-readable labels. - -#### FR-AUTH-002 Step Editing - -The user shall be able to: - -- Delete a step -- Suppress a step from generation -- Reorder steps where valid -- Convert a step into an assertion -- Change a step to use a different locator candidate -- Add notes and rationale -- Mark a step as variable-driven -- Mark a step as sensitive - -#### FR-AUTH-003 Assertion Approval - -Suggested assertions shall not be treated as final until approved by the user or by a configured auto-approval rule. - -#### FR-AUTH-004 Data Strategy Editing - -The author shall be able to review every captured value and choose whether it becomes: - -- A literal -- A scenario parameter -- A generated value -- A fixture reference -- A secret -- Ignored data - -#### FR-AUTH-005 Scenario Validation - -Before generation, the system shall validate that the scenario is structurally coherent, including: - -- Ordered step sequence -- Supported step types -- Required locator presence -- Required variable definitions -- No unresolved sensitive-data policy violations - -#### FR-AUTH-006 Version Creation - -Saving a material scenario change after review shall create a new immutable scenario version. -If the product supports draft editing, the draft state shall be stored separately from immutable -scenario versions until the user commits the change. The application shall preserve enough history -to identify what changed between versions. - -### 8.4 Code Generation Requirements - -#### FR-GEN-001 Derived Output Principle - -Generated source files shall always be reproducible from scenario data and template settings. Manual edits to generated code shall not silently become the source of truth. - -#### FR-GEN-002 Supported Output - -The v1 system shall generate: - -- C# Playwright test classes -- Shared locator-resolution helpers -- Optional data model or fixture classes -- Supporting test project files or export manifests where needed - -#### FR-GEN-003 Readability - -Generated code shall prioritise readability for human developers. It shall: - -- Use clear method names -- Avoid unreadable deeply nested locator code where helper abstraction is appropriate -- Separate data setup from action steps -- Use Playwright web-first assertions rather than fixed sleeps - -#### FR-GEN-004 LocatorResolver Helper - -The generated runtime helper library shall include a locator-resolution abstraction capable of: - -- Trying candidates in ranked order -- Validating uniqueness before acting -- Optionally scoping search to a parent container -- Emitting useful diagnostics when no candidate resolves uniquely - -#### FR-GEN-005 Assertion Generation - -Generated assertions shall prefer user-facing outcome checks over implementation-detail checks. - -#### FR-GEN-006 Variable and Fixture Generation - -When configured, the generator shall emit strongly typed classes or structures representing scenario variables, generated data, and fixture data. - -#### FR-GEN-007 Generation Profiles - -The system shall support generation profiles so teams can choose between styles such as: - -- Flat test methods -- Test plus helper methods -- Test plus page object scaffolding - -#### FR-GEN-008 Template Technology - -The implementation may use Roslyn, Scriban, or a hybrid approach, but the chosen approach shall support: - -- Deterministic output -- Repeatable formatting -- Testable generation logic -- Easy future extension - -#### FR-GEN-009 Repository-Friendly Output - -The generation system shall support output layouts that are easy to place under source control, including predictable file names, stable folder structure, and manifests that identify which files belong to each generation run. - -#### FR-GEN-010 Runtime Helper Compatibility - -Generated helper libraries shall remain versioned and compatible with the generator profile that emitted them. Breaking helper changes shall be traceable through generation manifests or template version metadata. - -### 8.5 Replay Requirements - -#### FR-REP-001 Scenario Replay - -The system shall replay a scenario using Playwright and the currently approved scenario version or the chosen generated artefact. - -#### FR-REP-002 Replay Diagnostics - -Each replay run shall capture: - -- Start and end timestamps -- Browser and environment settings -- Screenshots at key failure points -- Playwright traces when enabled -- Step-level pass or fail state -- Locator resolution attempts -- Assertion evaluation results -- Failure categorisation - -#### FR-REP-003 Failure Categories - -Replay failures shall be classified into categories including: - -- Locator resolution failure -- Timeout -- Unexpected navigation -- Assertion failure -- Authentication failure -- Environment availability failure -- Script or generation bug - -#### FR-REP-004 Targeted Rerun - -The UI shall allow rerunning: - -- The full scenario -- The current step onward -- A single step for diagnostic purposes where valid - -#### FR-REP-005 Replay Isolation - -Replay runs shall execute in isolated browser contexts and shall not share mutable session state unless explicitly configured for a trusted use case. - -### 8.6 Healing Requirements - -#### FR-HEAL-001 Deterministic Healing First - -When replay fails due to locator drift, the system shall first attempt deterministic recovery using: - -- Alternate persisted locator candidates -- Scoped searches -- Fuzzy accessible-name or text matching -- Parent context anchoring -- Nearby heading anchoring - -#### FR-HEAL-002 Healing Evidence - -Every healing proposal shall include evidence showing: - -- The original locator strategy -- Why it failed -- The proposed replacement -- What contextual signals supported the proposal -- Confidence scores - -#### FR-HEAL-003 Human Approval - -In v1, every healing change that alters the persisted scenario shall require explicit human approval. Automatic application of healing proposals to persisted scenarios shall not ship in v1. - -A controlled auto-apply policy for deterministic high-confidence matches may be introduced in a later phase, subject to all of the following: - -- The feature shall be disabled by default. -- Auto-apply shall be gated behind an administrator-only setting scoped per environment (never globally by default). -- Auto-apply shall be restricted to `SuggestionType` values classified as locator-only, deterministic, and non-destructive. Changes to assertion semantics, step ordering, variable classification, or sensitive-data policy shall never be auto-applied. -- A dry-run mode shall be available that records what would have been applied without mutating the scenario. -- Every auto-applied change shall still create a new immutable `ScenarioVersion` with a traceable `HealingSuggestion` link. - -#### FR-HEAL-004 AI as Adviser - -AI-assisted healing, if enabled later, shall be advisory and reviewable. It shall not bypass deterministic diagnostics or obscure the basis for a proposal. - -#### FR-HEAL-005 Healing Traceability - -When a healing proposal is approved and applied, the system shall retain a clear audit record linking the replay failure, the proposal, the approver, and the resulting scenario or locator change. - -### 8.7 Administration Requirements - -#### FR-ADM-001 Environment Configuration - -Administrators shall be able to configure: - -- Application environments -- Browser execution settings -- Base URLs -- Storage settings -- Database provider and connection -- Artefact retention rules -- Code-generation defaults - -#### FR-ADM-002 Authentication Bootstrap Options - -The system shall support configurable authentication bootstrap approaches such as: - -- Manual login during recording -- Stored Playwright storage state -- Cookie import under controlled administration - -#### FR-ADM-003 Auditability - -Administrative changes shall be auditable with who, when, and what changed. - -#### FR-ADM-004 Retention Policies - -Administrators shall be able to configure retention policies for logs, screenshots, traces, generated artefacts, and stale replay diagnostics. - -## 9. User Interface Requirements - + +### 3.2 Out of Scope for v1 + +The following are explicitly out of scope unless later prioritised: + +- Full low-code test authoring without any review step +- Support for native mobile app automation +- AI-only locator resolution at execution time +- Canvas-heavy, game-like, or non-DOM-centric application automation as a first-class supported scenario +- Rich collaborative review workflows such as branching, inline comments, or merge conflict handling between multiple editors +- Fine-grained enterprise RBAC beyond foundational role separation +- General-purpose workflow orchestration unrelated to browser test mining +- Electron or .NET MAUI desktop packaging in the initial release + +### 3.3 Deferred but Anticipated Scope + +The architecture shall allow later addition of: + +- Page object refactoring generation +- CI/CD export packs and pipeline templates +- Browser extension-based capture +- AI-assisted scenario summarisation and healing proposals +- Team collaboration and scenario versioning +- Desktop packaging for offline-friendly operator experiences + +## 4. Target Users and Primary Use Cases + +### 4.1 User Roles + +The system shall recognise these primary personas: + +1. Test Author +Records workflows, reviews steps, approves assertions, and generates tests. + +2. Developer +Consumes generated code, integrates it into solution structures, and refines helpers or templates. + +3. QA Engineer +Validates workflow coverage, replay reliability, and failure diagnostics. + +4. Platform Administrator +Configures browser runners, storage, security, code-generation defaults, and environment connections. + +### 4.2 Core Use Cases + +The platform shall support these end-to-end use cases: + +1. Record a new business workflow from a live application. +2. Review and edit the recorded timeline before generation. +3. Mark inputs as fixed, variable, generated, lookup-driven, or sensitive. +4. Accept or reject suggested assertions. +5. Generate a readable Playwright C# test suite and helper files. +6. Replay the generated scenario against the target application. +7. Investigate replay failures using screenshots, traces, DOM context, and locator diagnostics. +8. Accept a healing suggestion and regenerate the derived code. + +## 5. High-Level System Overview + +### 5.1 Product Capabilities + +The application shall consist of these major capability areas: + +1. Recording +Browser session launch, event capture, instrumentation injection, and artefact creation. + +2. Interpretation +DOM intelligence, locator ranking, variable classification, widget recognition, and assertion inference. + +3. Scenario Authoring +Timeline editing, step suppression, annotation, data strategy configuration, and acceptance of inferred outputs. + +4. Generation +Transformation of scenario models into C# Playwright tests, helpers, fixtures, and optional page object scaffolding. + +5. Replay and Healing +Execution, diagnostics, candidate re-resolution, deterministic healing, and controlled regeneration. + +6. Administration +Environment configuration, browser configuration, template settings, project export settings, and user management. + +### 5.2 Logical Architecture + +The solution shall be decomposed into the following logical components: + +1. Blazor Server UI +- Hosts recording workflows, timeline editing, generation preview, replay diagnostics, and admin surfaces. + +2. ASP.NET Core Application Host +- Provides HTTP endpoints, SignalR backplane for live session updates, background workers, authentication, orchestration, and API surfaces for the UI. + +3. Recording Engine +- Launches Playwright browsers and coordinates injected capture scripts, event streams, screenshots, traces, and browser-side metadata collection. + +4. Capture and Analysis Engine +- Normalises raw browser events, enriches them with DOM context, identifies semantic widgets, ranks locators, and produces assertion candidates. + +5. Scenario Engine +- Stores, edits, versions, validates, and materialises structured scenarios. + +6. Code Generation Engine +- Produces C# Playwright test source files, helper classes, optional page objects, and configuration assets using Roslyn or Scriban. + +7. Replay and Healing Engine +- Replays scenarios or generated tests, captures outcomes, diagnoses failures, evaluates fallback locators, and records healing proposals. + +8. Persistence Layer +- Stores structured entities, audit history, artefact references, configuration, and operational logs in PostgreSQL or SQL Server. + +9. Artefact Storage +- Stores screenshots, traces, exported code bundles, DOM snapshots, and replay attachments on a filesystem or object-backed abstraction. + +## 5.3 Operational Lifecycles + +### 5.3.1 Recording Lifecycle + +The recording lifecycle shall follow this logical sequence: + +1. User creates a recording session with target URL and settings. +2. The backend provisions a Playwright browser context and applies configured bootstrap settings. +3. Recorder instrumentation is injected before or at page startup. +4. The user performs actions in the target application. +5. Raw browser events and Playwright observations are streamed to the backend. +6. The backend normalises events and persists session artefacts incrementally. +7. The user stops recording. +8. The backend finalises the session and generates an initial scenario draft for review. + +### 5.3.2 Scenario Authoring Lifecycle + +The authoring lifecycle shall support: + +1. Reviewing the recorded step timeline. +2. Removing or suppressing noise. +3. Promoting meaningful events into semantic actions. +4. Approving or rejecting inferred assertions. +5. Classifying test data and secrets. +6. Choosing generation profile settings. +7. Saving an immutable scenario version ready for generation or replay. + +### 5.3.3 Generation Lifecycle + +The generation lifecycle shall support: + +1. Selecting an approved scenario version. +2. Applying generation profile and template settings. +3. Producing output files and a manifest. +4. Storing a checksum and file inventory. +5. Presenting the generated output for preview and export. + +### 5.3.4 Replay and Healing Lifecycle + +The replay and healing lifecycle shall support: + +1. Selecting a scenario version or generated artefact for execution. +2. Launching an isolated Playwright replay context. +3. Executing steps while recording diagnostics. +4. Classifying failures when they occur. +5. Attempting deterministic healing when drift is suspected. +6. Presenting a proposal with supporting evidence. +7. Applying approved healing changes to a new scenario version or updating approved locator preferences under controlled rules. + +## 6. Mandatory Technology Requirements + +### 6.1 Application Stack + +The implementation shall use: + +- ASP.NET Core for backend hosting and service orchestration +- Blazor Server for the primary web UI +- Microsoft.Playwright for .NET for browser control, observation, replay, trace capture, and browser-context management +- EF Core for relational persistence +- PostgreSQL or SQL Server as the durable database +- Roslyn and/or Scriban for code generation + +### 6.2 Database Provider Strategy + +To make v1 deliverable without doubling persistence cost, the platform shall adopt the following strategy: + +- PostgreSQL shall be the **primary and only required provider for v1**. Local development, shared non-production, and production-like deployments shall all use PostgreSQL unless explicitly reconfigured. +- SQL Server shall be a **deferred-but-supported secondary provider**. The persistence layer shall be designed so that SQL Server can be added later without redesigning the domain or repositories, but v1 acceptance does not require a working SQL Server deployment. +- The persistence layer shall use EF Core abstractions and migrations. +- Provider-specific SQL shall be avoided unless isolated behind clearly bounded infrastructure components and gated by a provider capability check. +- Data types, indexing strategies, JSON storage, and concurrency rules shall be designed to remain portable to SQL Server. Use of PostgreSQL-only features (for example `jsonb` operators, array columns, partial indexes) is permitted in v1 provided the feature can be substituted or approximated on SQL Server later. +- Integration tests shall run against real PostgreSQL (for example Testcontainers) rather than an in-memory provider, because in-memory providers do not exercise migration or JSON behaviour. + +### 6.3 Browser Support + +The initial supported execution browser shall be Chromium through Playwright. The architecture shall not prevent later support for WebKit and Firefox, but those browsers are not required for v1 unless explicitly enabled by configuration. + +## 7. Information Architecture and Domain Model + +### 7.1 Canonical Principle + +The canonical persisted business artefact shall be `Scenario`. Generated tests, replay runs, and healing decisions shall reference the scenario version they were derived from. + +### 7.2 Core Entities + +The persistence model shall include, at minimum, these entities. + +#### 7.2.0 Common Entity Requirements + +Unless explicitly exempted, every persistent entity defined in this section shall carry: + +- A stable primary key (`Id`) that is globally unique (GUID/UUID) and assigned by the application, not the database. +- Audit fields: `CreatedAtUtc`, `CreatedByUserId`, `UpdatedAtUtc`, `UpdatedByUserId` (the last two may be nullable where the entity is immutable by design, such as `ScenarioVersion` and `RecordedStep`). +- A `RowVersion` (optimistic concurrency token) column on entities that are editable by more than one workflow, including `Scenario`, `RecordingSession`, `HealingSuggestion`, and administrative configuration entities. +- A soft-delete marker (`IsDeleted`, `DeletedAtUtc`, `DeletedByUserId`) on `Scenario`, `RecordingSession`, `GenerationArtifact`, and artefact records. Immutable versions (e.g. `ScenarioVersion`, `RecordedStep`) shall not support soft delete; deletion of a scenario shall be a cascade-tombstoning operation documented in the data model. +- Timestamps stored in UTC; the domain shall not depend on database-local clocks for ordering decisions beyond audit. + +Field lists in the subsections below name the entity-specific fields and may omit the common fields above for brevity. + +#### 7.2.1 RecordingSession + +Represents an interactive capture session started by a user. + +Required fields: + +- `Id` +- `Name` +- `TargetBaseUrl` +- `EnvironmentName` +- `Status` +- `StartedByUserId` +- `StartedAtUtc` +- `EndedAtUtc` +- `BrowserKind` +- `BrowserChannel` +- `Headless` +- `AuthenticationMode` +- `RecordAssertionsAutomatically` +- `MaskedFieldPolicyJson` +- `SettingsJson` + +#### 7.2.2 Scenario + +Represents a business workflow compiled from one or more recording sessions and maintained as the product source of truth. + +Required fields: + +- `Id` +- `Name` +- `Description` +- `Status` +- `PrimaryRecordingSessionId` +- `TargetApplicationName` +- `TargetBaseUrl` +- `EnvironmentName` +- `CreatedByUserId` +- `CreatedAtUtc` +- `UpdatedAtUtc` +- `CurrentVersionNumber` +- `DefaultGenerationProfileId` +- `TagsJson` + +#### 7.2.3 ScenarioVersion + +Represents an immutable version of the scenario structure used for generation or replay traceability. + +In-progress edits shall not mutate an immutable `ScenarioVersion`. Any draft editing workflow shall +store draft state separately and create a new immutable scenario version when changes are committed. + +Required fields: + +- `Id` +- `ScenarioId` +- `VersionNumber` +- `ParentScenarioVersionId` +- `CreatedAtUtc` +- `CreatedByUserId` +- `ChangeSummary` +- `StructureJson` +- `InferenceMetadataJson` +- `ApprovedForGeneration` + +#### 7.2.4 RecordedStep + +Represents a normalised action or assertion candidate in sequence order. + +Required fields: + +- `Id` +- `ScenarioVersionId` +- `Sequence` +- `StepType` +- `SemanticActionType` +- `DisplayName` +- `UserNote` +- `Status` +- `UrlBefore` +- `UrlAfter` +- `TimestampUtc` +- `DurationMs` +- `RawValue` +- `NormalisedValue` +- `VariableStrategy` +- `SensitivityClassification` +- `TargetElementSnapshotId` +- `ParentContainerSnapshotId` +- `ExpectedOutcomeJson` +- `GeneratedCodeHintJson` + +#### 7.2.5 ElementSnapshot + +Represents the known properties of an element at record or replay time. + +Required fields: + +- `Id` +- `CapturePhase` +- `RecordingSessionId` (nullable; populated for recording-time capture) +- `ReplayRunId` (nullable; populated for replay-time capture) +- `RecordedStepId` +- `TagName` +- `InputType` +- `Role` +- `AccessibleName` +- `LabelText` +- `Placeholder` +- `TextContent` +- `TitleAttribute` +- `NameAttribute` +- `ValuePreview` +- `TestId` +- `AriaAttributesJson` +- `DataAttributesJson` +- `CssPath` +- `XPath` +- `BoundingBoxJson` +- `VisibilityState` +- `EnabledState` +- `ParentContextJson` +- `SiblingContextJson` +- `WidgetKind` +- `DomHash` + +#### 7.2.6 LocatorCandidate + +Represents one potential locator strategy for a target element. + +Required fields: + +- `Id` +- `RecordedStepId` +- `StrategyType` +- `Expression` +- `ScopeExpression` +- `Priority` +- `Confidence` +- `UniquenessAtRecordTime` +- `VisibilityAtRecordTime` +- `Recommended` +- `FailureReasonIfRejected` + +#### 7.2.7 AssertionCandidate + +Represents a suggested or approved expected outcome. + +Required fields: + +- `Id` +- `RecordedStepId` +- `AssertionType` +- `ExpressionJson` +- `Priority` +- `Confidence` +- `SourceSignal` +- `Approved` +- `RejectedReason` + +#### 7.2.8 ReplayRun + +Represents one execution attempt of a scenario or generated test. + +Required fields: + +- `Id` +- `ScenarioId` +- `ScenarioVersionId` +- `ReplaySourceKind` +- `GenerationArtifactId` (nullable; populated when replay uses a generated artefact bundle) +- `Status` +- `StartedAtUtc` +- `EndedAtUtc` +- `TriggeredByUserId` +- `ExecutionEnvironment` +- `BrowserKind` +- `Headless` +- `FailureCategory` +- `FailureSummary` +- `DiagnosticsJson` + +#### 7.2.9 HealingSuggestion + +Represents a proposed change when replay detects drift. + +Required fields: + +- `Id` +- `ReplayRunId` +- `RecordedStepId` +- `SuggestionType` +- `CurrentFailureEvidenceJson` +- `ProposedLocatorJson` +- `DeterministicConfidence` +- `AiConfidence` +- `ApprovalStatus` +- `AppliedAtUtc` +- `AppliedByUserId` + +#### 7.2.10 UserAccount + +Represents an authenticated principal of the platform. Identity data may be federated from an external provider, but the platform shall maintain a local record to support authorship attribution, audit trails, and role assignment. + +Required fields: + +- `Id` +- `ExternalSubjectId` (nullable; populated when federated) +- `DisplayName` +- `Email` +- `Role` (at minimum one of: `Administrator`, `Author`, `Viewer`) +- `IsDisabled` +- `LastLoginAtUtc` + +All audit references in other entities (for example `StartedByUserId`, `CreatedByUserId`, `TriggeredByUserId`) shall resolve to `UserAccount.Id`. + +#### 7.2.11 GenerationArtifact + +Represents a generated bundle or file set. + +Required fields: + +- `Id` +- `ScenarioVersionId` +- `GeneratorKind` +- `GeneratorVersion` +- `TemplateProfile` +- `TemplateVersion` +- `OutputRoot` +- `ManifestJson` +- `CreatedAtUtc` +- `CreatedByUserId` +- `Checksum` + +### 7.3 Structured Scenario Contract + +The application shall define a first-class structured scenario contract that can represent: + +- Navigation steps +- Click actions +- Fill actions +- Select actions +- Checkbox and toggle interactions +- Keyboard actions +- Dialog interactions +- Grid and list interactions +- Widget-scoped semantic actions +- Assertions +- Control flow metadata such as waits inferred from user-facing events +- Variable definitions +- Data generation strategies +- Replay and healing hints + +The scenario contract shall be expressive enough that: + +- Tests can be regenerated without rerecording. +- Helper libraries can evolve independently of the raw recording. +- Multiple code-generation styles can be supported later. +- Healing decisions can update the scenario model without directly editing generated output. + +## 7.4 Artefact Model + +The application shall treat the following as first-class artefacts: + +- Screenshots +- Playwright traces +- DOM snapshots +- Session logs +- Generated source files +- Replay diagnostics packages + +Each artefact shall have: + +- A stable identifier +- A logical owner such as recording session, scenario version, or replay run +- Content type +- Retention classification +- Storage path or provider-specific handle +- Created timestamp +- Optional checksum +- A size-in-bytes record + +The platform shall enforce default retention and size ceilings so artefact storage does not grow unbounded in the absence of administrative action: + +- Screenshots and DOM snapshots: retained for at least 30 days after the owning session or replay run completes; individual artefact size cap of 10 MB unless an administrator raises it. +- Playwright traces: retained for at least 14 days; individual trace cap of 200 MB. +- Session logs and replay diagnostic bundles: retained for at least 30 days. +- Generated source bundles: retained until the owning scenario is deleted. + +Administrators may extend any retention window or raise any size cap through the configuration surface defined in Section 11. A background cleanup process shall delete artefacts that have exceeded their retention window and record the deletion in the audit trail. + +## 7.5 State Model Expectations + +The system shall define explicit statuses for at least: + +- Recording sessions +- Scenarios +- Scenario versions +- Replay runs +- Healing suggestions +- Generation artefacts + +Status transitions shall be explicit, validated, and auditable. + +## 8. Functional Requirements + +### 8.1 Recording Requirements + +#### FR-REC-001 Guided Recording Creation + +The system shall let a user start a recording by providing: + +- Scenario name +- Target application URL +- Optional environment label +- Browser selection +- Whether automatic assertion suggestions are enabled +- Authentication or session bootstrap mode + +#### FR-REC-002 Browser Session Launch + +The system shall launch a Playwright browser context for recording with configurable: + +- Headless or headed mode +- Viewport +- Storage state +- Trace settings +- Video settings, if enabled +- Network capture settings + +#### FR-REC-003 Early Instrumentation Injection + +The system shall inject recorder logic before application content loads wherever technically possible, using Playwright-supported initialisation hooks such as `AddInitScriptAsync`. + +#### FR-REC-004 Event Capture + +The recorder shall capture, at minimum, these event types when user meaningful: + +- Page navigation +- Click +- Double click when distinguishable +- Input +- Change +- Form submit +- Key press +- Dialog open and close +- File download start +- Popup open +- Checkbox or toggle change +- Select and autocomplete selection + +The recorder shall suppress noise events such as incidental mouse movement unless explicitly enabled for diagnostics. + +#### FR-REC-005 Element Context Capture + +For each meaningful event, the system shall capture contextual element metadata including, where available: + +- HTML tag +- Input type +- Role +- Accessible name +- Label text +- Placeholder +- Visible text +- Title attribute +- Name attribute +- Test id or equivalent stable data attribute +- Bounding box +- Enabled and visible state +- Nearby heading +- Parent form, region, modal, or table context +- Candidate CSS path +- Candidate XPath as a last-resort fallback + +#### FR-REC-006 Observer Correlation + +The recording engine shall correlate user events with surrounding Playwright-observed browser state including: + +- URL changes +- Page load phases +- Network requests and responses +- Console errors and warnings, if enabled +- Dialog lifecycle +- Download lifecycle +- Popup lifecycle +- Screenshots +- Trace segments + +#### FR-REC-007 Sensitive Data Handling + +The user shall be able to mark fields or values as sensitive during or after recording. Sensitive values shall be masked in persisted logs, previews, and generated artefacts unless the user explicitly authorises retention in a secure variable form. + +#### FR-REC-008 Recording Controls + +The UI shall allow the user to: + +- Start +- Pause +- Resume +- Stop +- Cancel +- Mark current step as important +- Mark current step as ignorable +- Add an inline note + +#### FR-REC-009 Recorder Transport + +The recorder shall use a reliable browser-to-host transport mechanism for event delivery. Preferred mechanisms include Playwright-exposed bindings or a similarly explicit channel. Console scraping may be used only as a bounded fallback or diagnostic technique, not as the primary durable transport contract. + +#### FR-REC-010 Incremental Persistence + +Long-running recording sessions shall persist meaningful progress incrementally so that unexpected host failures do not necessarily lose the entire session. + +### 8.2 Post-Processing and Inference Requirements + +#### FR-INF-001 Event Normalisation + +The system shall transform raw captured events into a cleaned event stream by: + +- Removing technical noise +- Merging related input bursts +- Folding navigation waits into surrounding intent +- Linking actions to the most likely observed outcomes + +#### FR-INF-002 Semantic Action Inference + +The system shall infer higher-order action meaning where possible, including but not limited to: + +- Open details page +- Save form +- Submit search +- Open modal +- Confirm dialog +- Select row in grid +- Apply filter +- Choose date +- Dismiss notification + +#### FR-INF-003 Widget Recognition + +The system shall recognise common UI widget patterns where feasible, including: + +- Forms +- Tables and grids +- Modals and dialogs +- Dropdowns and combo boxes +- Tabs +- Toast notifications +- Date pickers +- Autocomplete controls + +The recognised widget type shall be stored as metadata usable by generation and healing logic. + +#### FR-INF-004 Locator Ranking + +For each targetable step, the system shall create a ranked set of locator candidates. The ranking algorithm shall prioritise: + +1. Explicit test id or configured stable data attribute +2. Role plus accessible name +3. Label-based targeting +4. Visible text +5. Placeholder +6. Scoped parent-region plus child-role targeting +7. Stable attribute combinations +8. CSS selector fallback +9. XPath fallback + +#### FR-INF-005 Locator Validation + +Each locator candidate shall be validated at record time when possible for uniqueness, visibility, and scope confidence. + +#### FR-INF-006 Variable Classification + +The system shall support classifying entered or observed values into at least these categories: + +- Fixed test data +- Scenario variable +- Generated data +- Lookup value +- Sensitive value +- Incidental value that should not become an assertion target + +#### FR-INF-007 Assertion Suggestion + +The system shall infer candidate assertions from post-action signals including: + +- Success toast visibility +- URL pattern change +- Heading change +- Dialog closure +- Newly visible row, card, or detail view +- Relevant network success response +- Count changes where meaningful + +#### FR-INF-008 Fuzzy Outcome Strategies + +The system shall support resilience strategies for data-sensitive assertions including: + +- Exact text to contains matching +- Regex matching for identifiers +- Date and time range validation +- Row existence by key column +- Partial heading matching +- Minimum count assertions + +#### FR-INF-009 Confidence Scoring + +The system shall assign confidence scores to inferred locators, assertions, widget classifications, and healing suggestions. Confidence values shall be explainable and based on deterministic evidence where possible. + +### 8.3 Scenario Authoring Requirements + +#### FR-AUTH-001 Timeline Review + +The UI shall present recorded scenarios as an ordered timeline of meaningful steps with human-readable labels. + +#### FR-AUTH-002 Step Editing + +The user shall be able to: + +- Delete a step +- Suppress a step from generation +- Reorder steps where valid +- Convert a step into an assertion +- Change a step to use a different locator candidate +- Add notes and rationale +- Mark a step as variable-driven +- Mark a step as sensitive + +#### FR-AUTH-003 Assertion Approval + +Suggested assertions shall not be treated as final until approved by the user or by a configured auto-approval rule. + +#### FR-AUTH-004 Data Strategy Editing + +The author shall be able to review every captured value and choose whether it becomes: + +- A literal +- A scenario parameter +- A generated value +- A fixture reference +- A secret +- Ignored data + +#### FR-AUTH-005 Scenario Validation + +Before generation, the system shall validate that the scenario is structurally coherent, including: + +- Ordered step sequence +- Supported step types +- Required locator presence +- Required variable definitions +- No unresolved sensitive-data policy violations + +#### FR-AUTH-006 Version Creation + +Saving a material scenario change after review shall create a new immutable scenario version. +If the product supports draft editing, the draft state shall be stored separately from immutable +scenario versions until the user commits the change. The application shall preserve enough history +to identify what changed between versions. + +### 8.4 Code Generation Requirements + +#### FR-GEN-001 Derived Output Principle + +Generated source files shall always be reproducible from scenario data and template settings. Manual edits to generated code shall not silently become the source of truth. + +#### FR-GEN-002 Supported Output + +The v1 system shall generate: + +- C# Playwright test classes +- Shared locator-resolution helpers +- Optional data model or fixture classes +- Supporting test project files or export manifests where needed + +#### FR-GEN-003 Readability + +Generated code shall prioritise readability for human developers. It shall: + +- Use clear method names +- Avoid unreadable deeply nested locator code where helper abstraction is appropriate +- Separate data setup from action steps +- Use Playwright web-first assertions rather than fixed sleeps + +#### FR-GEN-004 LocatorResolver Helper + +The generated runtime helper library shall include a locator-resolution abstraction capable of: + +- Trying candidates in ranked order +- Validating uniqueness before acting +- Optionally scoping search to a parent container +- Emitting useful diagnostics when no candidate resolves uniquely + +#### FR-GEN-005 Assertion Generation + +Generated assertions shall prefer user-facing outcome checks over implementation-detail checks. + +#### FR-GEN-006 Variable and Fixture Generation + +When configured, the generator shall emit strongly typed classes or structures representing scenario variables, generated data, and fixture data. + +#### FR-GEN-007 Generation Profiles + +The system shall support generation profiles so teams can choose between styles such as: + +- Flat test methods +- Test plus helper methods +- Test plus page object scaffolding + +#### FR-GEN-008 Template Technology + +The implementation may use Roslyn, Scriban, or a hybrid approach, but the chosen approach shall support: + +- Deterministic output +- Repeatable formatting +- Testable generation logic +- Easy future extension + +#### FR-GEN-009 Repository-Friendly Output + +The generation system shall support output layouts that are easy to place under source control, including predictable file names, stable folder structure, and manifests that identify which files belong to each generation run. + +#### FR-GEN-010 Runtime Helper Compatibility + +Generated helper libraries shall remain versioned and compatible with the generator profile that emitted them. Breaking helper changes shall be traceable through generation manifests or template version metadata. + +### 8.5 Replay Requirements + +#### FR-REP-001 Scenario Replay + +The system shall replay a scenario using Playwright and the currently approved scenario version or the chosen generated artefact. + +#### FR-REP-002 Replay Diagnostics + +Each replay run shall capture: + +- Start and end timestamps +- Browser and environment settings +- Screenshots at key failure points +- Playwright traces when enabled +- Step-level pass or fail state +- Locator resolution attempts +- Assertion evaluation results +- Failure categorisation + +#### FR-REP-003 Failure Categories + +Replay failures shall be classified into categories including: + +- Locator resolution failure +- Timeout +- Unexpected navigation +- Assertion failure +- Authentication failure +- Environment availability failure +- Script or generation bug + +#### FR-REP-004 Targeted Rerun + +The UI shall allow rerunning: + +- The full scenario +- The current step onward +- A single step for diagnostic purposes where valid + +#### FR-REP-005 Replay Isolation + +Replay runs shall execute in isolated browser contexts and shall not share mutable session state unless explicitly configured for a trusted use case. + +### 8.6 Healing Requirements + +#### FR-HEAL-001 Deterministic Healing First + +When replay fails due to locator drift, the system shall first attempt deterministic recovery using: + +- Alternate persisted locator candidates +- Scoped searches +- Fuzzy accessible-name or text matching +- Parent context anchoring +- Nearby heading anchoring + +#### FR-HEAL-002 Healing Evidence + +Every healing proposal shall include evidence showing: + +- The original locator strategy +- Why it failed +- The proposed replacement +- What contextual signals supported the proposal +- Confidence scores + +#### FR-HEAL-003 Human Approval + +In v1, every healing change that alters the persisted scenario shall require explicit human approval. Automatic application of healing proposals to persisted scenarios shall not ship in v1. + +A controlled auto-apply policy for deterministic high-confidence matches may be introduced in a later phase, subject to all of the following: + +- The feature shall be disabled by default. +- Auto-apply shall be gated behind an administrator-only setting scoped per environment (never globally by default). +- Auto-apply shall be restricted to `SuggestionType` values classified as locator-only, deterministic, and non-destructive. Changes to assertion semantics, step ordering, variable classification, or sensitive-data policy shall never be auto-applied. +- A dry-run mode shall be available that records what would have been applied without mutating the scenario. +- Every auto-applied change shall still create a new immutable `ScenarioVersion` with a traceable `HealingSuggestion` link. + +#### FR-HEAL-004 AI as Adviser + +AI-assisted healing, if enabled later, shall be advisory and reviewable. It shall not bypass deterministic diagnostics or obscure the basis for a proposal. + +#### FR-HEAL-005 Healing Traceability + +When a healing proposal is approved and applied, the system shall retain a clear audit record linking the replay failure, the proposal, the approver, and the resulting scenario or locator change. + +### 8.7 Administration Requirements + +#### FR-ADM-001 Environment Configuration + +Administrators shall be able to configure: + +- Application environments +- Browser execution settings +- Base URLs +- Storage settings +- Database provider and connection +- Artefact retention rules +- Code-generation defaults + +#### FR-ADM-002 Authentication Bootstrap Options + +The system shall support configurable authentication bootstrap approaches such as: + +- Manual login during recording +- Stored Playwright storage state +- Cookie import under controlled administration + +#### FR-ADM-003 Auditability + +Administrative changes shall be auditable with who, when, and what changed. + +#### FR-ADM-004 Retention Policies + +Administrators shall be able to configure retention policies for logs, screenshots, traces, generated artefacts, and stale replay diagnostics. + +## 9. User Interface Requirements + ### 9.1 Overall UI Structure The Blazor Server application shall provide at least these primary work areas: - -1. Recording workspace -2. Timeline and scenario editor + +1. Recording workspace +2. Timeline and scenario editor 3. Generated code preview 4. Replay and diagnostics workspace 5. Administration area -### 9.2 Recording Workspace - -The recording workspace shall display: - -- Browser session status -- Start, pause, resume, and stop controls -- Current URL -- Live step feed -- Ability to flag importance or ignore noise -- Inline note entry - -### 9.3 Timeline Workspace - -The timeline workspace shall provide: - -- Ordered list of steps -- Step type badges -- Locator confidence indicators -- Assertion suggestion indicators -- Variable classification editing -- Reorder and suppress controls - -### 9.4 Code Preview Workspace - -The code preview workspace shall provide: - -- Generated C# preview -- File list or manifest view -- Warnings for low-confidence locators or assertions -- Regenerate action - -### 9.5 Replay Workspace - -The replay workspace shall provide: - -- Replay status -- Step-by-step result stream -- Failure summary -- Locator diagnostics -- Screenshot and trace links -- Healing proposal review and approval actions - -### 9.6 Responsiveness - -The application shall be usable on standard desktop browser sizes and shall degrade gracefully on smaller widths, but desktop-first layout is acceptable for v1. - -## 10. API and Integration Requirements - -### 10.1 Internal API Surface - -The ASP.NET Core backend shall expose internal or authenticated HTTP APIs sufficient to support: - -- Recording lifecycle control -- Scenario CRUD -- Timeline editing -- Replay triggering -- Healing approval -- Artefact retrieval -- Administrative configuration - -### 10.2 Real-Time Updates - -The application shall provide real-time UI updates for long-running recording and replay operations. SignalR is the preferred approach. - -### 10.3 Artefact Export - -The system shall support export of generated assets into a chosen output directory structure that is suitable for committing to a repository or copying into another solution. - -### 10.4 External Target Application Interaction - -The platform shall treat the target application as an external system under test. It shall not require modifications to the target application, but it shall provide guidance when better accessibility or testability hooks would improve recording quality. - -### 10.5 API Design Expectations - -Backend APIs should: - -- Use explicit resource identifiers -- Return actionable validation errors -- Support long-running operation status retrieval -- Avoid leaking secrets or raw sensitive payloads -- Be versionable from the outset - -## 11. Configuration Requirements - -### 11.1 Configuration Sources - -The application shall support configuration from: - -- App settings files -- Environment variables -- Secure secret providers or equivalent deployment-time injection -- Database-backed administrative settings where appropriate - -### 11.2 Configurable Areas - -The following areas shall be configurable: - -- Database provider and connection string -- Artefact storage root or provider -- Browser launch defaults -- Headless defaults -- Base URL allowlists -- Session timeout and retention policies -- Masking rules -- Default generation profile -- Replay diagnostics level -- Authentication provider settings - -### 11.3 Validation - -Invalid required configuration shall fail fast at application startup or at the point of administrative save, depending on whether the setting is environment-level or runtime-editable. - -## 12. Security and Privacy Requirements - -### 12.1 Authentication and Access Control - -The platform shall require authenticated access for non-public environments. Baseline role separation shall distinguish at least: - -- Administrator -- Author -- Viewer - -### 12.2 Secret Protection - -Secrets shall never be written to plain logs, screenshots metadata, or generated code by default. This includes: - -- Target application credentials -- Session cookies -- Tokens -- Secret variable values - -### 12.3 Sensitive Data Masking - -The system shall allow masking rules by field name, selector metadata, or manual marking. Masked values shall remain masked in: - -- UI previews -- Diagnostic logs -- Stored event payloads -- Generation previews - -### 12.4 Target URL Allow-List Enforcement - -The platform shall enforce an administrator-managed allow-list of target base URLs. A recording or replay session shall fail to start if its requested target URL does not match an entry on the allow-list for the selected environment. The allow-list check shall be performed server-side before any Playwright browser context is launched. The allow-list shall be auditable and changes shall be captured in the audit trail defined in Section 12.7. - -### 12.5 Encryption of Captured Session Material - -The platform captures material that is sensitive by construction: authentication cookies, Playwright storage states, raw event payloads containing user input, and user-authored scenario variables marked sensitive. The following protections shall apply: - -- Playwright storage states, cookies, and any cached authentication material shall be encrypted at rest using platform-managed keys sourced from a secure secret provider. -- Scenario variables classified as `Sensitive` shall be encrypted at rest. Their plaintext shall never be written to generated source, generation previews, logs, or UI previews. -- Raw event payloads shall have sensitive fields masked or encrypted at rest according to the `MaskedFieldPolicyJson` in force at recording time. A field whose classification is changed to sensitive after recording shall be retroactively masked in subsequent previews and re-generation. -- Key material shall not be checked into source control. Key rotation shall be supported at least by re-encrypting affected records during a controlled administrative operation. - -### 12.6 Browser Session Isolation - -Each recording or replay session shall execute in an isolated Playwright browser context unless explicitly configured otherwise by an administrator. - -### 12.7 Audit Logging - -The system shall maintain an audit trail for: - -- Login and administrative access -- Scenario creation and approval -- Healing approval -- Artefact export -- Configuration changes - -## 13. Observability and Diagnostics Requirements - -### 13.1 Structured Logging - -The backend shall emit structured logs with identifiers such as: - -- Recording session id -- Scenario id -- Scenario version id -- Replay run id -- User id -- Browser context id - -### 13.2 Operational Metrics - -The system shall capture metrics useful for support and scaling, including: - -- Active recordings -- Active replays -- Average recording duration -- Replay success rate -- Most common failure categories -- Healing proposal frequency -- Generation duration - -### 13.3 Artefact Traceability - -Screenshots, traces, DOM snapshots, and generated outputs shall remain traceable to the session, scenario version, or replay run that created them. - -## 14. Verification and Testing Requirements - +The UI shall be implemented early enough in the delivery plan that developers can visually exercise the primary workflows during development. API-only or backend-only completion is not sufficient for the intended v1 delivery workflow. + +### 9.2 Recording Workspace + +The recording workspace shall display: + +- Browser session status +- Start, pause, resume, and stop controls +- Current URL +- Live step feed +- Ability to flag importance or ignore noise +- Inline note entry + +### 9.3 Timeline Workspace + +The timeline workspace shall provide: + +- Ordered list of steps +- Step type badges +- Locator confidence indicators +- Assertion suggestion indicators +- Variable classification editing +- Reorder and suppress controls + +### 9.4 Code Preview Workspace + +The code preview workspace shall provide: + +- Generated C# preview +- File list or manifest view +- Warnings for low-confidence locators or assertions +- Regenerate action + +### 9.5 Replay Workspace + +The replay workspace shall provide: + +- Replay status +- Step-by-step result stream +- Failure summary +- Locator diagnostics +- Screenshot and trace links +- Healing proposal review and approval actions + +### 9.6 Responsiveness + +The application shall be usable on standard desktop browser sizes and shall degrade gracefully on smaller widths, but desktop-first layout is acceptable for v1. + +## 10. API and Integration Requirements + +### 10.1 Internal API Surface + +The ASP.NET Core backend shall expose internal or authenticated HTTP APIs sufficient to support: + +- Recording lifecycle control +- Scenario CRUD +- Timeline editing +- Replay triggering +- Healing approval +- Artefact retrieval +- Administrative configuration + +### 10.2 Real-Time Updates + +The application shall provide real-time UI updates for long-running recording and replay operations. SignalR is the preferred approach. + +### 10.3 Artefact Export + +The system shall support export of generated assets into a chosen output directory structure that is suitable for committing to a repository or copying into another solution. + +### 10.4 External Target Application Interaction + +The platform shall treat the target application as an external system under test. It shall not require modifications to the target application, but it shall provide guidance when better accessibility or testability hooks would improve recording quality. + +### 10.5 API Design Expectations + +Backend APIs should: + +- Use explicit resource identifiers +- Return actionable validation errors +- Support long-running operation status retrieval +- Avoid leaking secrets or raw sensitive payloads +- Be versionable from the outset + +## 11. Configuration Requirements + +### 11.1 Configuration Sources + +The application shall support configuration from: + +- App settings files +- Environment variables +- Secure secret providers or equivalent deployment-time injection +- Database-backed administrative settings where appropriate + +### 11.2 Configurable Areas + +The following areas shall be configurable: + +- Database provider and connection string +- Artefact storage root or provider +- Browser launch defaults +- Headless defaults +- Base URL allowlists +- Session timeout and retention policies +- Masking rules +- Default generation profile +- Replay diagnostics level +- Authentication provider settings + +### 11.3 Validation + +Invalid required configuration shall fail fast at application startup or at the point of administrative save, depending on whether the setting is environment-level or runtime-editable. + +## 12. Security and Privacy Requirements + +### 12.1 Authentication and Access Control + +The platform shall require authenticated access for non-public environments. Baseline role separation shall distinguish at least: + +- Administrator +- Author +- Viewer + +### 12.2 Secret Protection + +Secrets shall never be written to plain logs, screenshots metadata, or generated code by default. This includes: + +- Target application credentials +- Session cookies +- Tokens +- Secret variable values + +### 12.3 Sensitive Data Masking + +The system shall allow masking rules by field name, selector metadata, or manual marking. Masked values shall remain masked in: + +- UI previews +- Diagnostic logs +- Stored event payloads +- Generation previews + +### 12.4 Target URL Allow-List Enforcement + +The platform shall enforce an administrator-managed allow-list of target base URLs. A recording or replay session shall fail to start if its requested target URL does not match an entry on the allow-list for the selected environment. The allow-list check shall be performed server-side before any Playwright browser context is launched. The allow-list shall be auditable and changes shall be captured in the audit trail defined in Section 12.7. + +### 12.5 Encryption of Captured Session Material + +The platform captures material that is sensitive by construction: authentication cookies, Playwright storage states, raw event payloads containing user input, and user-authored scenario variables marked sensitive. The following protections shall apply: + +- Playwright storage states, cookies, and any cached authentication material shall be encrypted at rest using platform-managed keys sourced from a secure secret provider. +- Scenario variables classified as `Sensitive` shall be encrypted at rest. Their plaintext shall never be written to generated source, generation previews, logs, or UI previews. +- Raw event payloads shall have sensitive fields masked or encrypted at rest according to the `MaskedFieldPolicyJson` in force at recording time. A field whose classification is changed to sensitive after recording shall be retroactively masked in subsequent previews and re-generation. +- Key material shall not be checked into source control. Key rotation shall be supported at least by re-encrypting affected records during a controlled administrative operation. + +### 12.6 Browser Session Isolation + +Each recording or replay session shall execute in an isolated Playwright browser context unless explicitly configured otherwise by an administrator. + +### 12.7 Audit Logging + +The system shall maintain an audit trail for: + +- Login and administrative access +- Scenario creation and approval +- Healing approval +- Artefact export +- Configuration changes + +## 13. Observability and Diagnostics Requirements + +### 13.1 Structured Logging + +The backend shall emit structured logs with identifiers such as: + +- Recording session id +- Scenario id +- Scenario version id +- Replay run id +- User id +- Browser context id + +### 13.2 Operational Metrics + +The system shall capture metrics useful for support and scaling, including: + +- Active recordings +- Active replays +- Average recording duration +- Replay success rate +- Most common failure categories +- Healing proposal frequency +- Generation duration + +### 13.3 Artefact Traceability + +Screenshots, traces, DOM snapshots, and generated outputs shall remain traceable to the session, scenario version, or replay run that created them. + +## 14. Verification and Testing Requirements + ### 14.1 Product Test Strategy The platform implementation shall include automated coverage for: - -- Domain and inference rules -- Scenario validation -- Locator ranking -- Assertion suggestion rules -- Code-generation output shape + +- Domain and inference rules +- Scenario validation +- Locator ranking +- Assertion suggestion rules +- Code-generation output shape - Replay orchestration - Healing evaluation - Persistence mappings and migrations -### 14.2 Test Layers - -At minimum, the repository should include: - -- Unit tests for core business rules and model transformations -- Integration tests for Playwright coordination, persistence, and artefact storage -- End-to-end tests against controlled sample web applications - -### 14.3 Regression Fixtures - -The implementation should maintain representative sample applications or fixture pages that cover: - -- Basic forms -- Grid interactions -- Modal dialogs -- Login flows -- Dynamic identifiers -- Toast-based success notifications - -### 14.4 Generator Verification - -Generated output shall be verified through snapshot-style approval tests or equivalent structural tests so template changes do not unintentionally degrade code readability or behaviour. - -## 15. Non-Functional Requirements - -### 15.1 Reliability - -The platform shall favour deterministic execution. Generated tests and replay flows shall use Playwright waiting behaviour and retryable assertions rather than fixed delays except where no better alternative exists. - -### 15.2 Performance - -The system shall feel responsive for interactive authoring. Initial targets, measured on a reference developer workstation (quad-core modern CPU, SSD, local PostgreSQL), expressed as percentiles over a rolling sample of at least 20 operations of the same type: - -- Time from "Start recording" click to Playwright page navigation ready: p50 ≤ 5 s, p95 ≤ 10 s. -- End-to-end latency from a captured browser event to its corresponding step appearing in the timeline: p50 ≤ 750 ms, p95 ≤ 2 s. -- Code generation for a scenario of up to 100 meaningful steps: p50 ≤ 8 s, p95 ≤ 15 s. -- Replay start-up overhead (time from replay trigger to first step execution) shall not exceed 10 s at p95. - -These are product-quality targets, not contractual SLAs. Observability required under Section 13 shall be sufficient to verify them post-deployment. - -### 15.3 Scalability - -The v1 system shall support multiple concurrent users and background replays on a single server instance. The architecture shall allow future scale-out of non-UI background execution services if demand grows. - -### 15.4 Maintainability - -The implementation shall keep clear boundaries between: - -- UI concerns -- Application services -- Playwright execution -- Scenario/inference logic -- Persistence -- Generation -- Healing - -### 15.5 Testability - -The system itself shall be designed for automated testing using unit, integration, and end-to-end coverage. The product shall not rely on untestable static global state for core flows. - -### 15.6 Accessibility - -Because the product explicitly depends on accessibility metadata from target applications, the platform UI should also model good practice. The Blazor UI should use accessible semantics, keyboard support, and meaningful announcements for long-running operations where practical. - -## 16. Deployment and Operational Requirements - -### 16.1 Host Model - -The initial application shall run as an ASP.NET Core server-hosted web application with Blazor Server integrated into the host process or solution boundary. - +The implementation shall also support a local, production-like developer test environment so changes can be exercised end to end before they are pushed to GitHub or deployed. That local environment shall include the real Blazor UI, PostgreSQL, artefact storage, and representative fixture applications unless a specific component is intentionally stubbed for local-only ergonomics. + +### 14.2 Test Layers + +At minimum, the repository should include: + +- Unit tests for core business rules and model transformations +- Integration tests for Playwright coordination, persistence, and artefact storage +- End-to-end tests against controlled sample web applications + +### 14.3 Regression Fixtures + +The implementation should maintain representative sample applications or fixture pages that cover: + +- Basic forms +- Grid interactions +- Modal dialogs +- Login flows +- Dynamic identifiers +- Toast-based success notifications + +### 14.4 Generator Verification + +Generated output shall be verified through snapshot-style approval tests or equivalent structural tests so template changes do not unintentionally degrade code readability or behaviour. + +## 15. Non-Functional Requirements + +### 15.1 Reliability + +The platform shall favour deterministic execution. Generated tests and replay flows shall use Playwright waiting behaviour and retryable assertions rather than fixed delays except where no better alternative exists. + +### 15.2 Performance + +The system shall feel responsive for interactive authoring. Initial targets, measured on a reference developer workstation (quad-core modern CPU, SSD, local PostgreSQL), expressed as percentiles over a rolling sample of at least 20 operations of the same type: + +- Time from "Start recording" click to Playwright page navigation ready: p50 ≤ 5 s, p95 ≤ 10 s. +- End-to-end latency from a captured browser event to its corresponding step appearing in the timeline: p50 ≤ 750 ms, p95 ≤ 2 s. +- Code generation for a scenario of up to 100 meaningful steps: p50 ≤ 8 s, p95 ≤ 15 s. +- Replay start-up overhead (time from replay trigger to first step execution) shall not exceed 10 s at p95. + +These are product-quality targets, not contractual SLAs. Observability required under Section 13 shall be sufficient to verify them post-deployment. + +### 15.3 Scalability + +The v1 system shall support multiple concurrent users and background replays on a single server instance. The architecture shall allow future scale-out of non-UI background execution services if demand grows. + +### 15.4 Maintainability + +The implementation shall keep clear boundaries between: + +- UI concerns +- Application services +- Playwright execution +- Scenario/inference logic +- Persistence +- Generation +- Healing + +### 15.5 Testability + +The system itself shall be designed for automated testing using unit, integration, and end-to-end coverage. The product shall not rely on untestable static global state for core flows. + +### 15.6 Accessibility + +Because the product explicitly depends on accessibility metadata from target applications, the platform UI should also model good practice. The Blazor UI should use accessible semantics, keyboard support, and meaningful announcements for long-running operations where practical. + +## 16. Deployment and Operational Requirements + +### 16.1 Host Model + +The initial application shall run as an ASP.NET Core server-hosted web application with Blazor Server integrated into the host process or solution boundary. + ### 16.2 Environment Profiles The implementation shall support at least: @@ -1326,225 +1333,233 @@ The implementation shall support at least: - Shared non-production environment - Production-like server deployment -### 16.3 Packaging Direction - -Desktop packaging through .NET MAUI Hybrid or Electron is explicitly a future option. The initial architecture shall therefore keep the backend and UI boundaries clean enough that later packaging can host the same application surfaces without re-implementing core recording, generation, or replay services. - -### 16.4 Operational Safety - -Operational processes shall account for: - -- Browser process cleanup -- Expired artefact cleanup -- Failed replay recovery -- Application restarts during long-running operations - -## 17. Suggested Repository Architecture - -The application should be introduced into this repository as a new vertical slice rather than by mutating the existing Symphony specification. A suggested project layout is: - -```text -/src - /TestMining.Platform.Host (ASP.NET Core host, Blazor Server UI, auth, APIs) - /TestMining.Platform.Core (domain models, interfaces, scenario contracts) - /TestMining.Platform.Recording (Playwright launch, injected recorder, capture coordination) - /TestMining.Platform.Analysis (DOM intelligence, inference, locator ranking, assertions) - /TestMining.Platform.Generation (Roslyn/Scriban generation) - /TestMining.Platform.Replay (replay execution and diagnostics) - /TestMining.Platform.Healing (deterministic healing services) - /TestMining.Platform.Persistence (EF Core, PostgreSQL/SQL Server adapters, repositories) - /TestMining.Platform.Artefacts (artefact storage abstraction) -/tests - /TestMining.Platform.Core.Tests - /TestMining.Platform.Integration.Tests - /TestMining.Platform.E2E.Tests -/docs - concept.md - requirements.md -``` - -## 18. Delivery Phasing - +The local developer execution profile shall mimic the actual application shape closely enough that a developer can validate the main workflows before pushing changes. At minimum, the local profile should run the ASP.NET Core host with the real Blazor Server UI, PostgreSQL, local artefact storage, and representative fixture applications or seeded test data. + +Differences between the local profile and shared/production-like environments shall be minimised and documented explicitly. Developer convenience shortcuts shall not bypass core safety behaviour such as allow-list enforcement, masking, audit logging, and encryption of sensitive stored material unless a local-only exception is intentionally documented and risk-accepted. + +### 16.3 Packaging Direction + +Desktop packaging through .NET MAUI Hybrid or Electron is explicitly a future option. The initial architecture shall therefore keep the backend and UI boundaries clean enough that later packaging can host the same application surfaces without re-implementing core recording, generation, or replay services. + +### 16.4 Operational Safety + +Operational processes shall account for: + +- Browser process cleanup +- Expired artefact cleanup +- Failed replay recovery +- Application restarts during long-running operations + +## 17. Suggested Repository Architecture + +The application should be introduced into this repository as a new vertical slice rather than by mutating the existing Symphony specification. A suggested project layout is: + +```text +/src + /TestMining.Platform.Host (ASP.NET Core host, Blazor Server UI, auth, APIs) + /TestMining.Platform.Core (domain models, interfaces, scenario contracts) + /TestMining.Platform.Recording (Playwright launch, injected recorder, capture coordination) + /TestMining.Platform.Analysis (DOM intelligence, inference, locator ranking, assertions) + /TestMining.Platform.Generation (Roslyn/Scriban generation) + /TestMining.Platform.Replay (replay execution and diagnostics) + /TestMining.Platform.Healing (deterministic healing services) + /TestMining.Platform.Persistence (EF Core, PostgreSQL/SQL Server adapters, repositories) + /TestMining.Platform.Artefacts (artefact storage abstraction) +/tests + /TestMining.Platform.Core.Tests + /TestMining.Platform.Integration.Tests + /TestMining.Platform.E2E.Tests +/docs + concept.md + requirements.md +``` + +## 18. Delivery Phasing + ### 18.1 Phase 1: Core Proof Phase 1 shall target: +- Local production-like developer environment bootstrap for safe pre-push validation - Recording of navigation, click, fill, select, checkbox, and simple assertion-relevant actions -- Timeline UI +- Timeline UI and core workflow surfaces sufficient for visual testing by developers - Locator ranking - Basic scenario persistence with immutable scenario version creation - C# Playwright generation -- Basic replay - -### 18.2 Phase 2: Robustness - -Phase 2 shall add: - -- Assertion inference -- Variable extraction -- Regex and partial matching -- Scoped locators -- Replay diagnostics -- Screenshot capture per important step - -### 18.3 Phase 3: Product Hardening - -Phase 3 shall add: - -- Login and session bootstrap management -- Export packaging -- Trace viewer integration -- Advanced healing review workflow and approval ergonomics -- Scenario history comparison and version-diff views - -### 18.4 Phase 4: AI Assistance - -Phase 4 may add: - -- Scenario naming suggestions -- Assertion suggestion assistance -- Healing proposal assistance -- Page object refactoring suggestions - -### 18.5 Phase Exit Criteria - -A phase shall not be declared complete until the criteria below are demonstrably met. These are conformance gates, not aspirational targets. - +- Basic replay + +### 18.2 Phase 2: Robustness + +Phase 2 shall add: + +- Assertion inference +- Variable extraction +- Regex and partial matching +- Scoped locators +- Replay diagnostics +- Screenshot capture per important step + +### 18.3 Phase 3: Product Hardening + +Phase 3 shall add: + +- Login and session bootstrap management +- Export packaging +- Trace viewer integration +- Advanced healing review workflow and approval ergonomics +- Scenario history comparison and version-diff views + +### 18.4 Phase 4: AI Assistance + +Phase 4 may add: + +- Scenario naming suggestions +- Assertion suggestion assistance +- Healing proposal assistance +- Page object refactoring suggestions + +### 18.5 Phase Exit Criteria + +A phase shall not be declared complete until the criteria below are demonstrably met. These are conformance gates, not aspirational targets. + Phase 1 exit: - A user can authenticate, record a simple CRUD workflow end-to-end against a fixture web application, and see the persisted scenario. - Locator candidates are ranked and persisted for every targetable step. - Generated C# Playwright output compiles against the emitted helper library. - Replay executes the generated scenario against the same fixture and reports pass/fail per step. - URL allow-list enforcement (Section 12.4) and encryption of storage state (Section 12.5) are enforced. - -Phase 2 exit: -- Assertion inference produces at least one outcome-oriented assertion suggestion for every save/submit/navigate step in the Phase 1 fixture library. -- Variable classification is editable in the UI and round-trips through regeneration without loss. -- Replay diagnostics bundles include screenshots at each failure, locator resolution attempts, and a failure category. - -Phase 3 exit: -- Login bootstrap options (Section FR-ADM-002) are demonstrable end-to-end. -- Healing review workflow can approve a deterministic locator healing proposal and create a new immutable scenario version referencing the originating replay run. -- Artefact retention cleanup (Section 7.4) runs on schedule and is observable. - -Phase 4 exit (if undertaken): -- Any AI-assisted suggestion is always advisory, never auto-applied, and is labelled as AI-sourced in the UI and audit trail. -- Deterministic diagnostics continue to run and are presented alongside any AI suggestion. - -## 19. Acceptance Criteria - -The implementation shall be considered to satisfy this specification only when all of the following are demonstrably true and covered by at least one automated test traceable to the cited requirement identifiers. - -1. A user can record a real browser workflow through the UI and the system persists a structured scenario. [FR-REC-001, FR-REC-004, FR-REC-010] -2. The scenario can be reviewed, edited, and approved without direct database manipulation. [FR-AUTH-001 through FR-AUTH-006] -3. The system generates readable C# Playwright artefacts from the approved scenario. [FR-GEN-001 through FR-GEN-010] -4. The generated or scenario-derived replay executes through Playwright and produces useful diagnostics. [FR-REP-001, FR-REP-002, FR-REP-003] -5. Locator candidates are ranked and persisted rather than reduced to a single opaque selector. [FR-INF-004, FR-INF-005] -6. Sensitive values can be masked and remain masked across previews and logs, and are encrypted at rest where stored. [FR-REC-007, 12.3, 12.5] -7. Assertions are outcome-oriented and can be approved or rejected before generation. [FR-INF-007, FR-AUTH-003] -8. A locator drift failure can produce a deterministic healing proposal with evidence and require human approval before altering a scenario in v1. [FR-HEAL-001 through FR-HEAL-005] -9. The application runs on ASP.NET Core with a Blazor Server frontend and a PostgreSQL-backed persistence layer, with SQL Server pluggability preserved as a deferred option. [6.1, 6.2] +- A developer can run a local, production-like environment with the Blazor UI and visually exercise the Phase 1 workflow before pushing changes. + +Phase 2 exit: +- Assertion inference produces at least one outcome-oriented assertion suggestion for every save/submit/navigate step in the Phase 1 fixture library. +- Variable classification is editable in the UI and round-trips through regeneration without loss. +- Replay diagnostics bundles include screenshots at each failure, locator resolution attempts, and a failure category. + +Phase 3 exit: +- Login bootstrap options (Section FR-ADM-002) are demonstrable end-to-end. +- Healing review workflow can approve a deterministic locator healing proposal and create a new immutable scenario version referencing the originating replay run. +- Artefact retention cleanup (Section 7.4) runs on schedule and is observable. + +Phase 4 exit (if undertaken): +- Any AI-assisted suggestion is always advisory, never auto-applied, and is labelled as AI-sourced in the UI and audit trail. +- Deterministic diagnostics continue to run and are presented alongside any AI suggestion. + +## 19. Acceptance Criteria + +The implementation shall be considered to satisfy this specification only when all of the following are demonstrably true and covered by at least one automated test traceable to the cited requirement identifiers. + +1. A user can record a real browser workflow through the UI and the system persists a structured scenario. [FR-REC-001, FR-REC-004, FR-REC-010] +2. The scenario can be reviewed, edited, and approved without direct database manipulation. [FR-AUTH-001 through FR-AUTH-006] +3. The system generates readable C# Playwright artefacts from the approved scenario. [FR-GEN-001 through FR-GEN-010] +4. The generated or scenario-derived replay executes through Playwright and produces useful diagnostics. [FR-REP-001, FR-REP-002, FR-REP-003] +5. Locator candidates are ranked and persisted rather than reduced to a single opaque selector. [FR-INF-004, FR-INF-005] +6. Sensitive values can be masked and remain masked across previews and logs, and are encrypted at rest where stored. [FR-REC-007, 12.3, 12.5] +7. Assertions are outcome-oriented and can be approved or rejected before generation. [FR-INF-007, FR-AUTH-003] +8. A locator drift failure can produce a deterministic healing proposal with evidence and require human approval before altering a scenario in v1. [FR-HEAL-001 through FR-HEAL-005] +9. The application runs on ASP.NET Core with a Blazor Server frontend and a PostgreSQL-backed persistence layer, with SQL Server pluggability preserved as a deferred option. [6.1, 6.2] 10. Generated output remains reproducible bit-for-bit (modulo timestamps declared as non-deterministic) from scenario data plus generation profile plus template version. [FR-GEN-001, FR-GEN-010] 11. Recording and replay refuse to start against target URLs not present on the administrator-managed allow-list. [12.4] 12. Observability emits the identifiers listed in 13.1 and permits verification of the performance targets in 15.2. - -## 20. Risks and Constraints - -### 20.1 Known Challenging Target Apps - -The product shall acknowledge reduced reliability for: - -- Canvas-heavy applications -- Highly virtualised lists or grids -- Poorly accessible React or SPA interfaces -- UIs with unstable text and random generated IDs -- Drag-and-drop heavy workflows -- Rich text editors - -### 20.2 Product Response to Poor Testability - -When recording quality is degraded by target-application design, the platform should surface actionable recommendations such as: - -- Add stable `data-testid` attributes -- Improve ARIA labels -- Expose predictable success messages -- Add stable row identifiers -- Reduce dependence on random element IDs - -## 21. Open Design Decisions - -The following decisions remain implementation-level choices unless later locked by an approved architecture note: - -- Whether Roslyn, Scriban, or a hybrid is the default generator engine -- Whether artefact storage is local filesystem first or abstracted for object storage from day one -- Whether replay executes in-process or via a dedicated background worker boundary -- Which authentication provider is used for the host application -- Which database provider is used as the primary development default - -## 22. Build, Test, and Delivery Requirements - -### 22.1 Repository Boundaries - -The test mining platform shall be introduced into this repository as a new vertical slice using the `TestMining.Platform.*` naming convention described in Section 17. Existing Symphony projects (`src/Symphony.*`, `tests/Symphony.*`, `symphony_docs/`, and `SPEC.md`) are retained tooling assets and shall not be mutated by work targeting this specification unless a task explicitly requires it. - +13. A developer can run the application locally in a production-like configuration, including the real Blazor UI and PostgreSQL-backed persistence, to validate core workflows before push or deployment. [9.1, 14.1, 16.2] + +## 20. Risks and Constraints + +### 20.1 Known Challenging Target Apps + +The product shall acknowledge reduced reliability for: + +- Canvas-heavy applications +- Highly virtualised lists or grids +- Poorly accessible React or SPA interfaces +- UIs with unstable text and random generated IDs +- Drag-and-drop heavy workflows +- Rich text editors + +### 20.2 Product Response to Poor Testability + +When recording quality is degraded by target-application design, the platform should surface actionable recommendations such as: + +- Add stable `data-testid` attributes +- Improve ARIA labels +- Expose predictable success messages +- Add stable row identifiers +- Reduce dependence on random element IDs + +## 21. Open Design Decisions + +The following decisions remain implementation-level choices unless later locked by an approved architecture note: + +- Whether Roslyn, Scriban, or a hybrid is the default generator engine +- Whether artefact storage is local filesystem first or abstracted for object storage from day one +- Whether replay executes in-process or via a dedicated background worker boundary +- Which authentication provider is used for the host application +- Which database provider is used as the primary development default + +## 22. Build, Test, and Delivery Requirements + +### 22.1 Repository Boundaries + +The test mining platform shall be introduced into this repository as a new vertical slice using the `TestMining.Platform.*` naming convention described in Section 17. Existing Symphony projects (`src/Symphony.*`, `tests/Symphony.*`, `symphony_docs/`, and `SPEC.md`) are retained tooling assets and shall not be mutated by work targeting this specification unless a task explicitly requires it. + ### 22.2 Build System - The solution shall build with the repository's documented required .NET SDK version. Any SDK upgrade shall be an explicit, documented change. - `dotnet restore`, `dotnet build`, and `dotnet test` shall succeed from a clean checkout with no unresolved warnings treated as errors in core projects. - Projects shall enable nullable reference types and treat analyzer warnings as errors where practical. - -### 22.3 Continuous Integration - -The repository shall include a CI pipeline (GitHub Actions is the expected host) that, on every pull request touching the platform: - -1. Restores and builds the full solution. -2. Runs unit and integration tests, including PostgreSQL-backed integration tests using a containerised database. -3. Runs generator snapshot tests (Section 14.4). -4. Publishes test results and code coverage summaries. -5. Fails the pipeline if any new secret-looking string appears in diffs or if retention/policy violations are detected by linting rules once established. - -### 22.4 Recorder Script Delivery - -The browser-side recorder script is a first-class build artefact: - -- It shall live in source control under a clearly named project directory and shall be built deterministically as part of the platform build. -- Its version shall be pinned and surfaced in captured recordings as `RecorderVersion` metadata so replay and healing can detect incompatible captures. -- It shall be injected via Playwright-supported initialisation hooks. Runtime fetching of the recorder from an external origin is prohibited. - -### 22.5 Test Layering and Flake Budget - -- Unit tests shall be the default: deterministic, no network, no browser. -- Integration tests shall exercise real infrastructure (PostgreSQL, Playwright against fixture web applications shipped with the repo). -- End-to-end tests shall be kept small in number and tagged so they can be excluded from fast developer loops. -- Any test that becomes flaky shall be either fixed or quarantined within one working day; persistent quarantine is not acceptable. - -## 23. Implementer Guardrails - -These guardrails exist to prevent the most likely failure modes of agentic or human implementers working on this specification. - -### 23.1 Prohibited Actions - -1. Writing captured credentials, cookies, storage states, tokens, or any value classified as `Sensitive` to logs, screenshots metadata, generation previews, or generated code. -2. Launching a Playwright browser context against a target URL that does not match the configured allow-list. -3. Introducing provider-specific SQL or raw ADO.NET calls into domain or application layers. Provider-specific code shall live in clearly isolated infrastructure components. -4. Editing generated source files by hand and checking the result in. Generated output shall be reproduced from scenario data and templates only. -5. Promoting an AI-assisted suggestion to a runtime-critical path. AI may advise; deterministic logic decides. -6. Deleting or restructuring Symphony-related files while working on test mining platform tasks. -7. Silently relaxing any non-functional requirement in Section 15, security requirement in Section 12, or acceptance criterion in Section 19. - -### 23.2 Required Behaviours - -1. Cite the requirement identifier (e.g. `FR-REC-004`, `FR-HEAL-003`) that motivates a code change in the pull request description. -2. For any behaviour change, update or add a conformance test traceable to the cited requirement. -3. Use feature-flagged rollout for anything that touches healing auto-apply, AI assistance, or cross-environment configuration. -4. When a requirement is ambiguous, raise a clarification rather than guessing. Record the resolution in this document or in an architecture decision record. - -### 23.3 Pull Request Discipline - -1. Pull requests shall be scoped to a single vertical slice. Drive-by refactors, especially across the Symphony/test-mining boundary, shall be split into separate changes. -2. Every PR shall identify new or changed artefact retention characteristics, new captured data fields, or new outbound network calls. -3. Secret-scanning and dependency vulnerability checks shall run before merge. - -## 24. Final Requirement Statement - -This repository’s new application shall be a C#-based semantic test mining platform built on ASP.NET Core, Blazor Server, Microsoft.Playwright for .NET, and PostgreSQL or SQL Server. It shall record browser interactions, infer structured scenarios, rank resilient locators, generate maintainable Playwright C# tests, replay them with strong diagnostics, and support deterministic healing while keeping structured scenarios as the enduring source of truth. +- The repository should provide a documented local developer startup workflow for running a production-like application profile, including the Blazor UI and PostgreSQL dependency path. + +### 22.3 Continuous Integration + +The repository shall include a CI pipeline (GitHub Actions is the expected host) that, on every pull request touching the platform: + +1. Restores and builds the full solution. +2. Runs unit and integration tests, including PostgreSQL-backed integration tests using a containerised database. +3. Runs generator snapshot tests (Section 14.4). +4. Publishes test results and code coverage summaries. +5. Fails the pipeline if any new secret-looking string appears in diffs or if retention/policy violations are detected by linting rules once established. + +### 22.4 Recorder Script Delivery + +The browser-side recorder script is a first-class build artefact: + +- It shall live in source control under a clearly named project directory and shall be built deterministically as part of the platform build. +- Its version shall be pinned and surfaced in captured recordings as `RecorderVersion` metadata so replay and healing can detect incompatible captures. +- It shall be injected via Playwright-supported initialisation hooks. Runtime fetching of the recorder from an external origin is prohibited. + +### 22.5 Test Layering and Flake Budget + +- Unit tests shall be the default: deterministic, no network, no browser. +- Integration tests shall exercise real infrastructure (PostgreSQL, Playwright against fixture web applications shipped with the repo). +- End-to-end tests shall be kept small in number and tagged so they can be excluded from fast developer loops. +- Any test that becomes flaky shall be either fixed or quarantined within one working day; persistent quarantine is not acceptable. + +## 23. Implementer Guardrails + +These guardrails exist to prevent the most likely failure modes of agentic or human implementers working on this specification. + +### 23.1 Prohibited Actions + +1. Writing captured credentials, cookies, storage states, tokens, or any value classified as `Sensitive` to logs, screenshots metadata, generation previews, or generated code. +2. Launching a Playwright browser context against a target URL that does not match the configured allow-list. +3. Introducing provider-specific SQL or raw ADO.NET calls into domain or application layers. Provider-specific code shall live in clearly isolated infrastructure components. +4. Editing generated source files by hand and checking the result in. Generated output shall be reproduced from scenario data and templates only. +5. Promoting an AI-assisted suggestion to a runtime-critical path. AI may advise; deterministic logic decides. +6. Deleting or restructuring Symphony-related files while working on test mining platform tasks. +7. Silently relaxing any non-functional requirement in Section 15, security requirement in Section 12, or acceptance criterion in Section 19. + +### 23.2 Required Behaviours + +1. Cite the requirement identifier (e.g. `FR-REC-004`, `FR-HEAL-003`) that motivates a code change in the pull request description. +2. For any behaviour change, update or add a conformance test traceable to the cited requirement. +3. Use feature-flagged rollout for anything that touches healing auto-apply, AI assistance, or cross-environment configuration. +4. When a requirement is ambiguous, raise a clarification rather than guessing. Record the resolution in this document or in an architecture decision record. + +### 23.3 Pull Request Discipline + +1. Pull requests shall be scoped to a single vertical slice. Drive-by refactors, especially across the Symphony/test-mining boundary, shall be split into separate changes. +2. Every PR shall identify new or changed artefact retention characteristics, new captured data fields, or new outbound network calls. +3. Secret-scanning and dependency vulnerability checks shall run before merge. + +## 24. Final Requirement Statement + +This repository’s new application shall be a C#-based semantic test mining platform built on ASP.NET Core, Blazor Server, Microsoft.Playwright for .NET, and PostgreSQL or SQL Server. It shall record browser interactions, infer structured scenarios, rank resilient locators, generate maintainable Playwright C# tests, replay them with strong diagnostics, and support deterministic healing while keeping structured scenarios as the enduring source of truth. diff --git a/docs/security-plan.md b/docs/security-plan.md new file mode 100644 index 0000000..bc4f2d3 --- /dev/null +++ b/docs/security-plan.md @@ -0,0 +1,301 @@ +# Test Mining Platform Security Plan + +## 1. Purpose + +This document defines the implementation security plan for the platform described in [docs/requirements.md](./requirements.md), with emphasis on Sections 12, 16.4, 19, 22.3, and 23. + +This application handles sensitive material by construction, including cookies, storage state, captured inputs, target URLs, generated artefacts, and operator identities. Security controls must therefore be built into the default architecture rather than added later. + +## 2. Security Objectives + +1. Prevent unauthorised access to the platform and its artefacts. +2. Prevent secret leakage to logs, previews, diagnostics, and generated code. +3. Constrain browser automation to approved target URLs and isolated sessions. +4. Preserve auditable traceability for privileged and source-of-truth-changing actions. +5. Keep deterministic product features available even when optional AI or convenience features are disabled. + +## 3. Requirement Mapping + +Primary source requirements: + +- Section 11 configuration validation +- Section 12.1 through 12.7 +- Section 13 observability +- Section 16.4 operational safety +- Section 19 acceptance criteria 6 and 11 +- Section 22.3 CI secret scanning +- Section 23.1 prohibited actions + +## 4. Trust Boundaries + +Primary trust boundaries in v1: + +1. Browser under test vs platform host +2. Browser recorder script vs server-side recording engine +3. Blazor UI connection vs authenticated backend APIs +4. Application services vs persistence and artefact storage +5. Platform operators vs administrator-only configuration surfaces +6. Generated output vs canonical scenario storage + +Each cross-boundary interaction should be explicit, authenticated where applicable, validated, and logged without leaking sensitive payloads. + +## 5. Authentication and Authorization + +### 5.1 Authentication + +The platform shall require authenticated access outside public local development cases, consistent with Section 12.1. + +Recommended v1 approach: + +- local development: developer-friendly local auth toggle +- shared environments: external identity provider via ASP.NET Core authentication +- production-like environments: enforced external identity provider and secure cookie/session configuration + +The local developer environment should still exercise the real application shell and the core security-sensitive code paths wherever practical. Local convenience mode should reduce friction, not create a separate unrepresentative application path. + +### 5.2 Authorization + +Minimum role model: + +- `Administrator` +- `Author` +- `Viewer` + +Control expectations: + +- `Viewer` cannot start recording, edit scenarios, generate, replay, approve healing, or export sensitive artefacts. +- `Author` can perform workflow actions but cannot change environment security policy. +- `Administrator` can manage allow-lists, masking rules, retention, auth bootstrap settings, and key rotation workflows. + +## 6. Sensitive Data Classification + +Data should be classified at minimum as: + +- Public operational metadata +- Internal non-sensitive workflow data +- Sensitive captured data +- Secret authentication material + +Examples that must be treated as sensitive or secret: + +- credentials +- cookies +- Playwright storage state +- tokens +- manually marked sensitive input values +- scenario variables classified as `Sensitive` + +## 7. Core Security Controls + +### 7.1 Target URL Allow-List Enforcement + +Required by Section 12.4. + +Control plan: + +- store allow-list entries per environment +- validate requested target URL server-side before browser launch +- reject launch if URL does not match the selected environment policy +- audit every allow-list change + +Implementation note: + +This must happen before any Playwright browser context is created. + +### 7.2 Browser Session Isolation + +Required by Section 12.6 and `FR-REP-005`. + +Control plan: + +- create isolated browser contexts for recording and replay by default +- never share mutable session state across runs unless explicitly configured +- clean up contexts, temp files, traces, and storage state after completion or failure + +### 7.3 Secret Protection and Masking + +Required by Sections 12.2, 12.3, 12.5 and `FR-REC-007`. + +Control plan: + +- apply masking rules before persistence for previewable/raw event fields +- never log sensitive raw values +- never render secret values in UI previews +- never emit sensitive plaintext into generated source +- replace secret-backed values with secure variable references in generator output +- if a field is reclassified as sensitive after recording, future previews and regenerated output must use the updated masking rule + +### 7.4 Encryption at Rest + +Required by Section 12.5. + +Encrypt at rest: + +- Playwright storage state +- cookies +- stored authentication material +- sensitive scenario variables +- any captured raw payload fields configured for encryption + +Key management expectations: + +- keys loaded from secure deployment-time secret provider +- keys never committed to source control +- support controlled key rotation with re-encryption job +- re-encryption operations must be auditable and scoped so failed runs can be resumed safely + +### 7.5 Audit Logging + +Required by Section 12.7 and `FR-ADM-003`. + +Audit at minimum: + +- logins and access failures +- recording start/stop/cancel +- scenario version creation +- assertion approval/rejection where material +- generation export +- replay trigger +- healing approval/rejection +- configuration changes +- retention cleanup deletions +- allow-list changes +- authentication bootstrap configuration changes + +## 8. Secure Design by Capability + +### 8.1 Recording + +Threats: + +- recording against disallowed targets +- secret capture in raw payloads +- recorder transport spoofing or malformed payloads + +Controls: + +- allow-list gate +- strict payload validation +- masking before persistence +- authenticated session ownership checks + +### 8.2 Scenario Authoring + +Threats: + +- exposure of masked values in editing surfaces +- unauthorized source-of-truth changes + +Controls: + +- field-level masking in inspector and previews +- optimistic concurrency +- role-based edit permissions +- audit trail for version creation + +### 8.3 Generation + +Threats: + +- secret leakage to generated code +- export of unauthorised artefacts + +Controls: + +- generator redaction rules +- manifest review before export +- export authorization checks +- secret scanning in CI for generated fixtures/templates + +### 8.4 Replay and Healing + +Threats: + +- reusing mutable session state unsafely +- healing proposals altering source of truth without approval +- AI suggestions obscuring deterministic evidence + +Controls: + +- isolated replay context +- approval gate for persisted healing +- explicit provenance and evidence display +- AI disabled or advisory only by policy + +## 9. Logging and Telemetry Restrictions + +Structured logs must include correlation identifiers from Section 13.1, but must not contain: + +- tokens +- cookies +- storage state blobs +- secret values +- full sensitive captured field payloads + +Recommended practice: + +- structured properties for IDs and categories only +- masked previews when troubleshooting requires representative value shape +- explicit log sanitization tests +- do not place secret-bearing values in exception messages or validation summaries that may later be logged + +## 10. Secure Configuration Plan + +Configuration controls: + +- fail startup on missing required connection strings, auth config, or key references +- validate admin-managed policies before save +- separate secret configuration from ordinary app settings +- prevent unauthorised edits to allow-lists, masking policies, or retention policy +- distinguish environment-level immutable configuration from runtime-editable administrative settings + +## 11. CI/CD Security Controls + +Required by Section 22.3 and 23.3. + +Pipeline controls: + +- dependency vulnerability scanning +- secret scanning on diffs +- tests for masking and encryption rules +- policy/lint checks for retention and security-sensitive configuration + +Recommended additions: + +- SBOM generation +- container image scanning if container deployment is adopted +- restricted deployment secrets by environment + +## 12. Security Testing Plan Summary + +Security-specific tests should cover: + +- allow-list rejection +- sensitive-field masking at record and preview time +- encrypted persistence of storage state and sensitive variables +- authorization failures for admin-only actions +- audit event creation for privileged actions +- replay isolation guarantees +- healing approval enforcement + +Detailed testing ownership is defined in [docs/testing-plan.md](./testing-plan.md). + +## 13. Incident and Operational Response + +Operational readiness should include: + +- ability to revoke or rotate encryption keys +- ability to disable recording or replay by environment +- audit review of recent exports and configuration changes +- retention cleanup monitoring +- procedure for purging incorrectly retained sensitive artefacts + +## 14. Non-Negotiable Security Rules + +The following must never be relaxed in v1: + +1. No secret values in logs, previews, or generated code. +2. No recording or replay outside the configured allow-list. +3. No persisted healing changes without explicit approval. +4. No runtime-critical dependency on AI suggestions. +5. No key material in source control. diff --git a/docs/sprint-plan.md b/docs/sprint-plan.md new file mode 100644 index 0000000..06ca291 --- /dev/null +++ b/docs/sprint-plan.md @@ -0,0 +1,304 @@ +# Test Mining Platform Sprint Plan + +## 1. Purpose + +This sprint plan converts the implementation guidance in: + +- [requirements.md](./requirements.md) +- [technical-specification.md](./technical-specification.md) +- [ui-ux-plan.md](./ui-ux-plan.md) +- [security-plan.md](./security-plan.md) +- [testing-plan.md](./testing-plan.md) +- [implementation-roadmap.md](./implementation-roadmap.md) + +into an ordered execution plan for delivering the v1 application. + +This plan is subordinate to `requirements.md`. If any sprint task conflicts with the requirements, update the sprint plan rather than weakening the requirements. + +## 2. Planning Assumptions + +To keep the plan correct and implementation-ready, these assumptions are used: + +1. This sprint plan targets v1 completion, not the optional AI-assistance phase. +2. PostgreSQL is the required v1 persistence provider path. +3. Blazor Server is the required primary UI. +4. Structured `Scenario` plus immutable `ScenarioVersion` remain the source of truth throughout every sprint. +5. Security, auditability, observability, and automated verification are built into each sprint rather than deferred to the end. +6. Developers need a local production-like environment and usable UI surfaces early so workflow changes can be tested visually before push. + +## 3. Sprint Structure + +The plan uses eight implementation sprints: + +1. Sprint 0: Foundation and Architecture Decisions +2. Sprint 1: Host Shell, Persistence, and Admin Baseline +3. Sprint 2: Recording Pipeline MVP +4. Sprint 3: Scenario Authoring and Locator Intelligence +5. Sprint 4: Deterministic Generation and Export +6. Sprint 5: Replay MVP and Failure Diagnostics +7. Sprint 6: Semantic Robustness and Review Depth +8. Sprint 7: Product Hardening and v1 Completion + +Each sprint contains: + +- a sprint goal +- feature issues +- core requirement links +- explicit exit criteria + +## 4. Sprint 0: Foundation and Architecture Decisions + +### Goal + +Establish the project skeleton, architectural decisions, CI/testing baseline, and shared conventions needed to deliver the rest of the product without rework. + +### Issues + +1. Scaffold `TestMining.Platform.*` solution structure and shared project conventions. +2. Write ADRs for generator choice, authentication provider direction, replay execution boundary, and draft persistence shape. +3. Establish CI baseline for restore, build, test, secret scanning, and PostgreSQL-backed integration execution. +4. Create local production-like developer environment bootstrap plus fixture application/test harness foundation for later recording and replay coverage. + +### Requirement Links + +- Sections 6, 14, 17, 22 +- Sections 23.1 through 23.3 + +### Exit Criteria + +- New platform projects exist with clear boundaries. +- CI runs on the repository and enforces build/test/security baseline. +- Developers can start a local environment that mirrors the intended application topology closely enough for real workflow testing. +- Fixture harness exists for later sprints. +- Deferred design choices that block implementation have ADR direction recorded. + +## 5. Sprint 1: Host Shell, Persistence, and Admin Baseline + +### Goal + +Deliver the application host, authentication shell, PostgreSQL persistence baseline, admin configuration, and security-critical policy surfaces. + +### Issues + +1. Implement ASP.NET Core host and Blazor Server shell with role-aware navigation, usable for visual local testing from this sprint onward. +2. Implement PostgreSQL persistence baseline, EF Core migrations, and core domain entities. +3. Implement environment configuration, target URL allow-list management, and audit logging baseline. +4. Implement artefact storage abstraction, retention-policy model, and encrypted secret-storage plumbing. + +### Requirement Links + +- Sections 5.2, 6.1, 6.2, 7.2 through 7.5, 9.1, 10, 11, 12.1 through 12.7, 13, 16.1 +- `FR-ADM-001`, `FR-ADM-003`, `FR-ADM-004` + +### Exit Criteria + +- Users can sign in and reach a Blazor shell with role-aware access. +- PostgreSQL migrations succeed from a clean checkout. +- Core persistence entities exist for scenarios, versions, recordings, artefacts, replay runs, healing suggestions, and audit events. +- Allow-list and retention policies are managed server-side and audited. +- The local environment is usable for visually testing the host shell and admin configuration flows. + +## 6. Sprint 2: Recording Pipeline MVP + +### Goal + +Deliver the first end-to-end recording flow that can safely launch a browser, capture meaningful events, and persist recording progress incrementally. + +### Issues + +1. Implement recording session creation, validation, and browser launch with allow-list enforcement. +2. Implement recorder script delivery and init-script injection through Playwright. +3. Implement meaningful event capture, Playwright observation correlation, and reliable recorder transport. +4. Implement incremental recording persistence with pause, resume, stop, cancel, importance, ignore, and inline notes. + +### Requirement Links + +- `FR-REC-001` through `FR-REC-010` +- Sections 5.3.1, 9.2, 10.2, 10.4, 12.4, 12.6 + +### Exit Criteria + +- A user can start and control a recording from the UI. +- Browser launch is blocked for disallowed target URLs. +- Meaningful events are captured and persisted incrementally. +- Sensitive inputs are masked per active policy during capture and preview. +- The recording workflow is visually testable through the local UI. + +## 7. Sprint 3: Scenario Authoring and Locator Intelligence + +### Goal + +Transform recordings into structured drafts, expose a usable timeline editor, and persist ranked locator intelligence plus immutable scenario versions. + +### Issues + +1. Implement event normalisation and semantic action inference for the Phase 1 step set. +2. Implement element snapshots, locator candidate generation, ranking, and record-time validation. +3. Implement scenario draft creation, timeline workspace, and step editing workflows. +4. Implement scenario validation and immutable `ScenarioVersion` creation with change history. + +### Requirement Links + +- `FR-INF-001` through `FR-INF-005` +- `FR-AUTH-001`, `FR-AUTH-002`, `FR-AUTH-005`, `FR-AUTH-006` +- Sections 7.1 through 7.5, 9.3 + +### Exit Criteria + +- Recorded sessions compile into structured scenario drafts. +- Locator candidates are ranked and persisted for every targetable step. +- Users can review, suppress, reorder, annotate, and validate steps in the timeline UI. +- Material edits create new immutable scenario versions rather than mutating committed history. +- Developers can visually test timeline editing and versioning locally. + +## 8. Sprint 4: Deterministic Generation and Export + +### Goal + +Generate readable and reproducible C# Playwright output from approved scenario versions, with manifesting and repository-friendly export. + +### Issues + +1. Implement deterministic generation pipeline and generation profile model for the first supported output style. +2. Implement readable C# Playwright test generation plus runtime helper compatibility metadata. +3. Implement `LocatorResolver` helper generation and low-confidence warning surfacing in preview. +4. Implement generated artefact persistence, manifesting, preview, and repository-friendly export workflow. + +### Requirement Links + +- `FR-GEN-001` through `FR-GEN-010` +- Sections 5.3.3, 9.4, 10.3, 14.4, 19 + +### Exit Criteria + +- Approved scenario versions generate deterministic file sets. +- Generated code is previewable in the UI and exportable to a repository-friendly layout. +- Generation manifests include template version, helper compatibility, and checksum details. +- Generator output is verified by snapshot/approval tests. +- Developers can visually validate generation previews and warnings locally. + +## 9. Sprint 5: Replay MVP and Failure Diagnostics + +### Goal + +Replay scenario versions or generated artefacts in isolated contexts and provide actionable step-by-step diagnostics for failures. + +### Issues + +1. Implement replay execution orchestration from scenario version or generation artefact. +2. Implement step-level replay status streaming and replay workspace UI. +3. Implement replay diagnostics capture, failure categorisation, and artefact linking. +4. Implement targeted rerun support for full replay, current-step onward, and single-step diagnostics where valid. + +### Requirement Links + +- `FR-REP-001` through `FR-REP-005` +- Sections 5.3.4, 9.5, 10.2, 13.3 + +### Exit Criteria + +- Users can run replay from the UI against the approved scenario version or chosen generated artefact. +- Replay runs use isolated browser contexts. +- Failures produce categorized diagnostics with step-level evidence. +- Replay status survives long-running execution and remains observable. +- Developers can visually validate replay progress and diagnostics through the local UI. + +## 10. Sprint 6: Semantic Robustness and Review Depth + +### Goal + +Increase generated test resilience and author review quality with assertions, variables, fuzzy strategies, scoped locators, and richer diagnostic evidence. + +### Issues + +1. Implement variable classification model and UI editing for literals, parameters, generated values, fixtures, secrets, and ignored data. +2. Implement assertion suggestion engine, approval workflows, and outcome-oriented assertion persistence. +3. Implement fuzzy assertion strategies, scoped locators, and confidence explanation surfaces. +4. Implement richer replay diagnostics packages including screenshots and locator-resolution evidence. + +### Requirement Links + +- `FR-INF-006` through `FR-INF-009` +- `FR-AUTH-003`, `FR-AUTH-004` +- `FR-REP-002`, `FR-REP-003` +- Sections 9.3 through 9.5, 14.1 through 14.4 + +### Exit Criteria + +- Authors can classify all captured data with masking-safe behavior. +- Suggested assertions are reviewable and approval-gated before generation. +- Replay diagnostics include richer artefacts and locator evidence. +- Regeneration preserves approved variable and assertion choices. +- Developers can visually validate the deeper authoring and diagnostics flows locally. + +## 11. Sprint 7: Product Hardening and v1 Completion + +### Goal + +Finish the remaining hardening work needed to satisfy the v1 requirements and acceptance criteria: authentication bootstrap, healing approval, history/diff visibility, retention cleanup, and export/review polish. + +### Issues + +1. Implement authentication bootstrap options for manual login, stored storage state, and controlled cookie import. +2. Implement deterministic healing proposal generation, approval workflow, and linked immutable version creation. +3. Implement scenario history, version comparison, and trace viewer/export review surfaces. +4. Implement retention cleanup scheduling, artefact lifecycle enforcement, and operational safety/cleanup observability. + +### Requirement Links + +- `FR-ADM-002` +- `FR-HEAL-001` through `FR-HEAL-005` +- Sections 7.4, 12.5, 12.7, 16.4, 18.5, 19 + +### Exit Criteria + +- Authentication bootstrap flows work end to end under controlled administration. +- Healing proposals are deterministic, evidence-backed, and approval-gated. +- Approved healing creates a new immutable scenario version linked to the originating replay run. +- Retention cleanup is scheduled, observable, and audited. +- The v1 acceptance criteria in Section 19 are demonstrably satisfied. +- The full core workflow is visually testable locally before push or deployment. + +## 12. Cross-Sprint Working Agreements + +These apply in every sprint: + +1. Every implementation issue must cite the requirement IDs it satisfies. +2. Every behaviour change must include automated coverage linked to those requirements. +3. Security-sensitive work must document masking, encryption, audit, and retention impact. +4. Generated code is always derived output, never the editing source of truth. +5. `Symphony.*` tooling assets remain outside the change scope unless explicitly requested. + +## 13. GitHub Execution Model + +To keep GitHub planning easy to manage: + +- each sprint should be represented by a GitHub milestone +- each deliverable issue should be assigned to exactly one sprint milestone +- issue titles should stay vertical-slice oriented instead of layer-only +- issue bodies should include scope, requirement links, dependencies, and exit expectations + +Recommended labels for later use: + +- `area/host` +- `area/recording` +- `area/analysis` +- `area/generation` +- `area/replay` +- `area/healing` +- `area/security` +- `area/testing` + +## 14. Definition of Done Per Sprint + +A sprint is only done when: + +1. Its milestone issues are closed. +2. The sprint exit criteria in this document are met. +3. Build, test, and security checks pass for the implemented scope. +4. New behaviour is documented where needed. +5. No sprint introduces drift from `requirements.md`. + +## 15. Post-v1 Backlog + +The optional AI-assistance phase remains post-v1 and should not be treated as required for initial application completion. If scheduled later, it should be tracked as a separate milestone series with feature flags and advisory-only controls. diff --git a/docs/technical-specification.md b/docs/technical-specification.md new file mode 100644 index 0000000..19d360c --- /dev/null +++ b/docs/technical-specification.md @@ -0,0 +1,486 @@ +# Test Mining Platform Technical Specification + +## 1. Purpose + +This document translates [docs/requirements.md](./requirements.md) into an implementation-oriented technical specification for the new test mining platform described in the repository [README.md](../README.md). + +It is subordinate to `docs/requirements.md`. If this document conflicts with the requirements, the requirements win. + +## 2. Source Inputs + +- Product source of truth: `docs/requirements.md` +- Repository framing: `README.md` +- Repository boundary guidance: Requirements Sections 17 and 22.1 + +## 2.1 Decision Status Legend + +To remove ambiguity, this document uses the following terms consistently: + +- `Locked`: directly required by `docs/requirements.md` for v1 +- `Recommended`: strong implementation guidance that fits the requirements and should be followed unless an ADR records a different choice +- `Deferred`: intentionally left open by the requirements and should be resolved by ADR before implementation hardens + +## 3. Product Intent + +The system is a semantic test mining and stabilisation platform, not a raw click recorder. Its durable source of truth is the structured `Scenario` model, and generated Playwright code is a reproducible derived artefact. + +This technical plan primarily supports: + +- recording guided browser workflows +- enriching captured events with DOM, accessibility, and runtime context +- compiling recordings into immutable scenario versions +- generating readable C# Playwright output +- replaying scenarios with diagnostics +- producing deterministic healing suggestions under human approval +- running the application locally in a production-like shape so end-to-end changes can be exercised before push or deployment +- delivering a usable UI early enough that major workflows can be visually tested during implementation + +Relevant requirements: + +- Sections 1, 2, 5, 7, 8, 18, 19, 24 +- `FR-REC-*`, `FR-INF-*`, `FR-AUTH-*`, `FR-GEN-*`, `FR-REP-*`, `FR-HEAL-*`, `FR-ADM-*` + +## 4. Solution Shape + +The implementation should introduce a new vertical slice using the `TestMining.Platform.*` naming convention from Section 17 and must not repurpose retained `Symphony.*` assets. + +Proposed solution structure: + +```text +/src + /TestMining.Platform.Host + /TestMining.Platform.Core + /TestMining.Platform.Application + /TestMining.Platform.Recording + /TestMining.Platform.Analysis + /TestMining.Platform.Generation + /TestMining.Platform.Replay + /TestMining.Platform.Healing + /TestMining.Platform.Persistence + /TestMining.Platform.Artefacts + /TestMining.Platform.Contracts + /TestMining.Platform.Recorder +/tests + /TestMining.Platform.Core.Tests + /TestMining.Platform.Application.Tests + /TestMining.Platform.Integration.Tests + /TestMining.Platform.Generator.Tests + /TestMining.Platform.E2E.Tests + /TestMining.Platform.Fixtures +``` + +Notes: + +- `Host` owns ASP.NET Core hosting, Blazor Server UI, DI, authentication, SignalR, HTTP endpoints, and background services. +- `Core` owns domain entities, enums, invariants, scenario contracts, and interfaces. +- `Application` owns use-case orchestration, commands, queries, validation, and DTO mapping. +- `Recording`, `Analysis`, `Generation`, `Replay`, and `Healing` own the product capability implementations. +- `Persistence` owns EF Core, PostgreSQL implementation, migrations, repositories, and provider abstractions. +- `Artefacts` owns storage abstraction and retention cleanup. +- `Contracts` owns shared API and SignalR contracts to keep host/UI boundaries explicit. +- `Recorder` owns the browser-side recorder script and deterministic build output required by Section 22.4. + +## 5. Technology Decisions + +Locked from requirements: + +- Backend: ASP.NET Core +- UI: Blazor Server +- Browser automation: Playwright for .NET +- ORM: EF Core +- Primary database for v1: PostgreSQL +- Future secondary provider: SQL Server +- Code generation: Roslyn, Scriban, or hybrid + +Recommended defaults for v1: + +- ASP.NET Core server app with integrated Blazor Server host +- EF Core migrations targeting PostgreSQL first +- Scriban templates for first-pass generation readability and snapshot testing +- Roslyn reserved for later structural generation/refactoring needs +- Filesystem artefact storage behind an abstraction so object storage can be added later +- A documented local developer environment that mirrors the production-capable application topology closely enough for end-to-end validation + +Deferred by requirements and requiring ADRs before hardening: + +- exact authentication provider +- exact generator implementation split between Scriban and Roslyn +- whether replay remains in-process in v1 or moves behind a worker boundary +- how much draft scenario state is normalized versus document-shaped + +## 6. Architecture Principles + +Derived from requirements Sections 5, 6, 15, 16, 22, and 23: + +1. `Scenario` and immutable `ScenarioVersion` are the canonical business artefacts. +2. Generated code is never edited as the source of truth. +3. Browser orchestration, inference, generation, replay, healing, persistence, and UI remain separate modules. +4. Deterministic logic is mandatory on runtime-critical paths. +5. Sensitive data is masked or encrypted before persistence and before display. +6. Long-running operations are observable, resumable where reasonable, and auditable. +7. Every significant workflow is traceable to user, scenario version, replay run, and artefacts. + +## 7. Logical Components + +### 7.1 Host and UI + +Responsibilities: + +- route handling +- Blazor Server pages/components +- authentication and authorization +- internal HTTP APIs +- SignalR hubs for recording/replay progress +- background cleanup and replay workers + +Key requirement links: + +- Sections 5.2, 9, 10, 11, 12.1, 13, 16 + +### 7.2 Recording Engine + +Responsibilities: + +- start browser contexts with validated settings +- inject recorder script via Playwright init hooks +- capture meaningful events and correlated Playwright observations +- incrementally persist recording progress + +Key requirement links: + +- `FR-REC-001` through `FR-REC-010` +- Sections 5.3.1, 10.4, 12.4, 12.6 + +### 7.3 Analysis Engine + +Responsibilities: + +- clean and normalise raw event streams +- infer semantic actions and widget types +- create, rank, and validate locator candidates +- propose assertion candidates and data strategies +- assign explainable confidence scores + +Key requirement links: + +- `FR-INF-001` through `FR-INF-009` + +### 7.4 Scenario Engine + +Responsibilities: + +- manage draft edits and immutable version commits +- enforce structural validation before generation +- preserve change summaries and version lineage + +Key requirement links: + +- Sections 7.1 through 7.5 +- `FR-AUTH-001` through `FR-AUTH-006` + +### 7.5 Generation Engine + +Responsibilities: + +- compile scenario version plus generation profile into a deterministic file set +- emit test code, runtime helpers, fixture/data types, and manifest +- persist manifest, checksum, template version, and helper compatibility metadata + +Key requirement links: + +- `FR-GEN-001` through `FR-GEN-010` +- Sections 5.3.3, 14.4, 19 + +### 7.6 Replay Engine + +Responsibilities: + +- execute scenario or generated artefact in isolated browser context +- stream step results and diagnostics +- classify failures +- support full, onward, and targeted reruns where valid + +Key requirement links: + +- `FR-REP-001` through `FR-REP-005` + +### 7.7 Healing Engine + +Responsibilities: + +- evaluate replay failures for deterministic recovery options +- produce evidence-backed healing suggestions +- require approval before persisted scenario changes +- create linked immutable scenario versions when healing is applied + +Key requirement links: + +- `FR-HEAL-001` through `FR-HEAL-005` + +### 7.8 Persistence and Artefacts + +Responsibilities: + +- EF Core entity mapping and repository access +- migrations and provider portability discipline +- encrypted data storage for sensitive material +- artefact storage, retention, and cleanup jobs +- audit trail persistence + +Key requirement links: + +- Sections 7, 11, 12.5, 12.7, 13.3, 16.4, 22.3 + +## 8. Domain Model Summary + +The data model should directly implement the entities in requirements Section 7.2: + +- `RecordingSession` +- `Scenario` +- `ScenarioVersion` +- `RecordedStep` +- `ElementSnapshot` +- `LocatorCandidate` +- `AssertionCandidate` +- `ReplayRun` +- `HealingSuggestion` +- `UserAccount` +- `GenerationArtifact` + +Additional supporting entities recommended for v1: + +- `ScenarioDraft` +- `ScenarioVariableDefinition` +- `ArtefactRecord` +- `AuditEvent` +- `EnvironmentConfiguration` +- `GenerationProfile` +- `MaskedFieldPolicy` +- `RetentionPolicy` +- `ReplayStepResult` + +Design rules: + +- Use application-assigned GUID/UUID keys. +- Use optimistic concurrency tokens for editable entities. +- Store immutable scenario content as versioned JSON plus normalized child tables where query value exists. +- Preserve audit metadata and UTC timestamps everywhere required by Section 7.2.0. +- Keep `ScenarioVersion` immutable after commit; any editable state lives in draft-specific storage only. +- Treat generated files, replay runs, and healing decisions as downstream artefacts that always reference a scenario version, never replace it. + +### 8.1 Retention and Cleanup Defaults + +The implementation should encode the Section 7.4 defaults directly in configuration and migrations-facing seed data where practical: + +- screenshots and DOM snapshots: at least 30 days retention, 10 MB default item cap +- Playwright traces: at least 14 days retention, 200 MB default item cap +- session logs and replay diagnostic bundles: at least 30 days retention +- generated source bundles: retained until the owning scenario is deleted + +Administrative overrides must remain auditable and server-enforced. + +## 9. Main Workflows + +### 9.1 Recording Workflow + +1. User creates recording request with scenario name, environment, target URL, browser, and auth bootstrap mode. +2. Server validates user permissions and allow-list before launching Playwright. +3. Recording engine creates isolated browser context and injects recorder. +4. Browser events stream to host through explicit transport. +5. Engine correlates events with navigation, network, dialog, popup, trace, and screenshot signals. +6. Incremental persistence stores session progress, steps, snapshots, and artefact references. +7. Stop action finalizes the recording and triggers initial scenario draft creation. + +### 9.2 Authoring Workflow + +1. User opens timeline workspace for a draft scenario. +2. UI shows step list, locator confidence, assertion suggestions, and variable strategy metadata. +3. User edits, suppresses, reorders, approves assertions, and marks sensitive or variable values. +4. Scenario validator enforces coherent order, locator presence, variable definitions, and masking rules. +5. Commit action creates a new immutable `ScenarioVersion`. + +### 9.3 Generation Workflow + +1. User selects approved scenario version and generation profile. +2. Generator compiles scenario structure into deterministic output. +3. Host stores manifest, checksum, generator version, template version, and helper version. +4. UI previews files and warnings before export. + +### 9.4 Replay and Healing Workflow + +1. User launches replay from scenario version or generation artefact. +2. Engine executes steps in isolated context and streams step states. +3. Failure categorizer assigns a replay failure class. +4. Healing engine evaluates persisted candidates and contextual anchors. +5. UI shows evidence-backed proposal. +6. Approval creates a linked healing record and new immutable scenario version. + +## 10. Internal API Surface + +The internal authenticated API should be versioned from day one, for example `/api/v1/...`. + +Suggested resource groups: + +- `/recordings` +- `/recordings/{id}/events` +- `/recordings/{id}/controls` +- `/scenarios` +- `/scenarios/{id}/draft` +- `/scenario-versions/{id}` +- `/scenario-versions/{id}/generate` +- `/replays` +- `/replays/{id}` +- `/healing-suggestions/{id}` +- `/artefacts/{id}` +- `/admin/environments` +- `/admin/generation-profiles` +- `/admin/retention-policies` + +API rules: + +- every mutating endpoint must use explicit resource identifiers +- validation failures must return actionable error details +- long-running operations must expose status retrieval and correlation IDs +- API contracts must avoid returning sensitive raw payloads even to authenticated clients by default + +SignalR channels should cover: + +- recording session state +- live step feed +- replay step stream +- generation status +- background cleanup notifications where useful + +## 11. Background Processing + +Recommended hosted services: + +- recording session supervisor +- replay execution queue/dispatcher +- artefact retention cleanup job +- encryption key rotation job runner +- diagnostics aggregation job + +Recommended v1 execution boundary: + +- keep replay execution in-process as a hosted service for v1 simplicity +- isolate the execution behind an application interface so it can move to a dedicated worker later without rewriting domain or UI layers + +These services should operate on explicit persisted state rather than in-memory-only state. + +## 12. Configuration Model + +Configuration sources must support Section 11: + +- `appsettings.*.json` +- environment variables +- secure secret providers +- database-backed admin settings + +Configuration areas: + +- connection strings +- artefact storage root/provider +- allowed target base URLs per environment +- browser defaults +- trace/video/network capture defaults +- masking rules +- retention windows and size limits +- generation defaults +- authentication provider configuration + +Validation rules: + +- environment-level invalid configuration fails startup +- runtime-editable invalid configuration fails save with actionable validation + +### 12.1 Local Production-Like Developer Profile + +The implementation should provide a documented local profile that mirrors the real application topology as closely as practical for development: + +- ASP.NET Core host and real Blazor Server UI +- PostgreSQL as the active local persistence provider +- local filesystem artefact storage through the same abstraction used elsewhere +- fixture applications or seeded test data for recording, generation, and replay validation + +The local profile is for pre-push confidence, not for inventing a second architecture. Differences from shared or production-like environments should stay minimal and explicit. + +## 13. Security-Critical Technical Controls + +This technical specification depends on the detailed controls in [docs/security-plan.md](./security-plan.md). At minimum: + +- server-side allow-list check before browser launch +- isolated browser contexts by default +- encrypted storage state, cookies, and sensitive variables at rest +- masking before persistence for previewable fields +- no secret leakage to logs, previews, or generated code +- audit trail for security-sensitive actions + +## 14. Observability + +The host must emit structured logs and metrics around: + +- recording start/stop/failure +- generation start/complete/failure +- replay start/step/failure/complete +- healing proposal creation/approval/rejection +- artefact cleanup and retention deletions + +Every event should include the identifiers required by Section 13.1 where applicable. + +Metrics should be sufficient to verify: + +- recording startup latency +- event-to-timeline publication latency +- generation duration +- replay startup latency +- replay pass/fail rate by failure category +- healing proposal rate and approval rate + +## 15. Performance and Scaling Targets + +Implementation should be designed so Section 15.2 is measurable: + +- record launch readiness +- event-to-timeline latency +- generation duration +- replay startup overhead + +Recommended approach: + +- capture timings in application services and background workers +- expose metrics through OpenTelemetry-compatible instrumentation +- measure per environment and per browser kind + +## 16. Open Technical Decisions + +The following remain deferred and should be resolved in ADRs before implementation hardens: + +1. Generator default: Scriban, Roslyn, or hybrid +2. Draft editing persistence shape: JSON document, normalized tables, or mixed +3. Replay execution boundary: in-process hosted service or dedicated worker process +4. Authentication provider choice for host application +5. Artefact storage abstraction depth in v1 + +## 17. Recommended First Vertical Slices + +To align with Phase 1 in Section 18: + +1. Host shell plus authenticated Blazor layout and PostgreSQL persistence bootstrap, available in a local production-like developer profile +2. Recording session creation plus allow-list validation +3. Recorder transport and incremental persistence for navigation/click/fill/select +4. Timeline review UI with draft editing so the workflow can be visually tested early +5. Locator ranking and scenario version creation +6. Deterministic generation preview and export +7. Basic replay with pass/fail diagnostics + +## 18. Definition of Ready for Implementation + +Implementation should not start until these are agreed: + +- project layout and naming +- PostgreSQL as the active v1 provider +- generator approach for Phase 1 +- fixture web applications for automated testing +- authentication approach for non-local environments +- local production-like startup workflow for developers, including how the UI and PostgreSQL are run together diff --git a/docs/testing-plan.md b/docs/testing-plan.md new file mode 100644 index 0000000..11f3847 --- /dev/null +++ b/docs/testing-plan.md @@ -0,0 +1,351 @@ +# Test Mining Platform Testing Plan + +## 1. Purpose + +This document defines the verification strategy for the platform described in [docs/requirements.md](./requirements.md). It expands on Sections 14, 15.5, 18.5, 19, and 22.3 through 22.5. + +## 2. Testing Goals + +The test strategy must prove that: + +1. structured scenarios are persisted and versioned correctly +2. recorder and analysis pipelines transform browser interactions into meaningful scenario steps +3. generated Playwright output is deterministic and readable +4. replay diagnostics and healing workflows are trustworthy +5. security and masking rules are enforced continuously + +## 3. Test Principles + +Derived from requirements: + +- unit tests are the default +- integration tests use real infrastructure where it matters +- end-to-end tests are limited but representative +- flaky tests must be fixed or quarantined quickly +- every acceptance criterion must trace to automated coverage +- the PostgreSQL-backed path is the required v1 provider path in CI and local integration testing +- developers need a local production-like environment for pre-push validation of the real application and UI + +## 4. Repository Test Layout + +Recommended test projects: + +```text +/tests + /TestMining.Platform.Core.Tests + /TestMining.Platform.Application.Tests + /TestMining.Platform.Integration.Tests + /TestMining.Platform.Generator.Tests + /TestMining.Platform.E2E.Tests + /TestMining.Platform.Fixtures +``` + +Purpose: + +- `Core.Tests`: entities, invariants, domain services, deterministic ranking/scoring rules +- `Application.Tests`: command/query handlers, validation, workflow orchestration with mocked boundaries +- `Integration.Tests`: PostgreSQL, EF Core mappings, artefact storage, Playwright coordination, encryption/masking integration +- `Generator.Tests`: snapshot/approval verification of generated output +- `E2E.Tests`: thin end-to-end flows against shipped fixture apps +- `Fixtures`: sample web apps/pages covering required target patterns + +## 5. Coverage by Requirement Area + +### 5.1 Recording + +Coverage target: + +- `FR-REC-001` through `FR-REC-010` + +Test themes: + +- session creation validation +- allow-list enforcement before launch +- recorder init script injection +- event capture for meaningful event types +- sensitive field masking during capture +- pause/resume/stop behaviour +- incremental persistence recovery + +### 5.2 Analysis and Inference + +Coverage target: + +- `FR-INF-001` through `FR-INF-009` + +Test themes: + +- event stream normalisation +- semantic action inference +- widget recognition +- locator ranking order +- locator uniqueness validation +- variable classification rules +- assertion suggestion and fuzzy matching strategies +- confidence scoring explainability + +### 5.3 Scenario Authoring + +Coverage target: + +- `FR-AUTH-001` through `FR-AUTH-006` + +Test themes: + +- step editing rules +- locator candidate switching +- assertion approval workflows +- sensitive-data policy validation +- immutable scenario version creation +- draft vs committed version behaviour + +### 5.4 Generation + +Coverage target: + +- `FR-GEN-001` through `FR-GEN-010` + +Test themes: + +- deterministic file output +- readable code structure assertions +- helper compatibility metadata +- variable/fixture generation +- repository-friendly layout and manifest generation +- generator warnings for low-confidence decisions + +### 5.5 Replay and Healing + +Coverage target: + +- `FR-REP-001` through `FR-REP-005` +- `FR-HEAL-001` through `FR-HEAL-005` + +Test themes: + +- scenario replay pass/fail execution +- failure categorisation +- targeted rerun rules +- isolated replay contexts +- deterministic healing proposal creation +- human approval gate enforcement +- traceability from replay failure to approved healing version + +### 5.6 Administration and Security + +Coverage target: + +- `FR-ADM-001` through `FR-ADM-004` +- Sections 11, 12, 13, 16.4 + +Test themes: + +- environment configuration validation +- retention policy persistence +- authentication bootstrap configuration +- audit event creation +- encryption and masking enforcement +- log sanitization +- retention cleanup behaviour and artefact traceability + +## 6. Test Layers + +### 6.1 Unit Tests + +Scope: + +- no browser +- no database +- deterministic logic only + +Examples: + +- locator ranking priority +- scenario validation rules +- assertion inference based on synthetic signals +- code generation model transformations before rendering + +### 6.2 Integration Tests + +Scope: + +- real PostgreSQL +- EF Core migrations +- real artefact storage abstraction against local filesystem +- Playwright coordination against fixture pages where appropriate + +Examples: + +- migration application from clean database +- persistence of scenario versions and related steps +- encryption-at-rest round trip +- recorder event flow into persistent draft scenario +- replay diagnostics package creation +- retention cleanup deleting expired artefacts while preserving audit evidence + +### 6.3 End-to-End Tests + +Scope: + +- run the host app +- exercise core user journeys through the UI or internal APIs +- use representative fixture applications + +Required Phase 1 path: + +1. authenticate +2. record simple CRUD workflow +3. review timeline +4. commit scenario version +5. generate C# output +6. replay and inspect result + +### 6.4 Local Developer Validation Environment + +In addition to automated coverage, the repository should support a local production-like validation profile so a developer can test changes before pushing them. + +That local profile should include: + +- the real ASP.NET Core host +- the real Blazor Server UI +- PostgreSQL +- artefact storage through the normal abstraction +- representative fixture applications or seeded data + +Recommended usage: + +- run the full application locally +- exercise the main UI workflows visually +- use the local environment before pushing substantial workflow changes + +## 7. Fixture Application Plan + +Based on Section 14.3, fixtures should cover: + +- simple forms +- editable grids +- modal dialogs +- login flows +- unstable/dynamic IDs +- toast notifications + +Recommended implementation: + +- small ASP.NET Core fixture apps or static pages served from test host +- deterministic seed data +- stable routes and reset hooks for test runs + +## 8. Generator Verification Strategy + +Required by Section 14.4. + +Recommended approach: + +- snapshot or approval tests for generated files +- normalize line endings and timestamp placeholders before comparison +- assert both full-file snapshots and structural invariants + +Structural invariants should include: + +- file names +- namespaces +- helper references +- absence of fixed sleeps by default +- redaction of sensitive values + +## 9. Performance and Reliability Testing + +Support Sections 15.1 through 15.3. + +Measure: + +- recording startup readiness +- event-to-timeline latency +- generation duration for 100-step scenarios +- replay startup overhead + +Recommended practice: + +- capture metrics during integration suites +- run periodic benchmark job outside fast PR loop +- keep thresholds visible but avoid fragile benchmark tests in ordinary CI + +CI should still fail when basic instrumentation needed to measure Section 15.2 is removed or broken. + +## 10. Security Testing + +Must include: + +- allow-list rejection tests +- authorization tests for admin-only settings +- secret redaction and masking tests +- encryption persistence tests +- no-secret-in-generated-output tests +- audit trail creation tests + +## 11. Traceability and Conformance + +Every requirement-backed feature should include: + +- test case IDs linked to `FR-*` requirements or section numbers +- location of automated coverage +- acceptance-criteria mapping + +The repository should maintain this mapping in [docs/requirements-traceability-matrix.md](./requirements-traceability-matrix.md). + +## 12. CI Pipeline Expectations + +Pull request CI should: + +1. restore packages +2. build the solution +3. run unit and integration tests +4. run generator snapshot tests +5. publish test results and coverage +6. run secret scanning and policy checks +7. run PostgreSQL-backed integration tests using a containerized database, per Section 22.3 + +The repository should also document the local startup path developers use to validate the same application shape before push. + +Suggested command groups: + +- `dotnet restore` +- `dotnet build` +- `dotnet test` + +## 13. Exit Criteria by Phase + +### Phase 1 + +- record-to-scenario flow proven against fixture CRUD app +- locator candidates persisted +- generated output compiles +- replay reports per-step pass/fail +- allow-list and encryption rules tested +- local production-like environment supports visual execution of the Phase 1 UI flow + +### Phase 2 + +- assertion inference and variable classification round-trip tested +- replay diagnostics bundle tested +- screenshot-on-failure behaviour tested + +### Phase 3 + +- authentication bootstrap tested end to end +- healing approval flow creates new immutable version +- retention cleanup job tested and observable + +## 14. Conformance Matrix Maintenance + +When implementation begins, every new feature PR should add or update: + +- the linked `FR-*` identifiers +- the automated tests proving the requirement +- any changed acceptance-criteria evidence +- any new retained artefact or sensitive-data handling impact + +### Phase 4 + +- AI-assisted suggestions verified as advisory only +- deterministic diagnostics still present alongside AI output diff --git a/docs/ui-ux-plan.md b/docs/ui-ux-plan.md new file mode 100644 index 0000000..4d787cc --- /dev/null +++ b/docs/ui-ux-plan.md @@ -0,0 +1,365 @@ +# Test Mining Platform UI/UX Plan + +## 1. Purpose + +This document turns the UI-related requirements in [docs/requirements.md](./requirements.md) into a delivery plan for the Blazor Server experience. + +Primary requirement anchors: + +- Section 4 user roles and use cases +- Section 8.1 through 8.7 +- Section 9 user interface requirements +- Section 10.2 real-time updates +- Section 12.1 access control +- Section 15.6 accessibility + +## 2. UX Principles + +The UI should reflect the product principles from requirements Section 2.3: + +1. Semantic over raw technical detail +2. Structured scenario over generated code +3. Deterministic evidence before automation or AI suggestions +4. Review before approval +5. Diagnostics always available + +Practical interpretation: + +- show business-meaningful labels first, raw technical payload second +- surface confidence, evidence, and warnings wherever the system infers behaviour +- keep every destructive or source-of-truth-changing action explicit +- preserve continuity between recording, editing, generation, and replay +- keep the scenario editor as the primary authoring surface; generated code preview is a downstream review surface, not the editing source of truth +- deliver the UI early enough that developers can visually test the real workflows during implementation, not only after backend completion + +## 3. User Roles and UX Focus + +- `Administrator`: environment configuration, policy management, audit visibility +- `Author`: recording, editing, approval, generation, replay, healing review +- `Viewer`: read-only review of scenarios, replay runs, and generated artefact metadata + +Permissions should shape visible actions, not just backend responses. + +Role visibility rules: + +- hide or disable actions the current role cannot complete +- explain why an action is unavailable when doing so helps workflow comprehension +- never show approval or export affordances that the current user is not authorized to invoke + +## 4. Primary Navigation Model + +Recommended top-level navigation: + +- Dashboard +- Recordings +- Scenarios +- Replays +- Artefacts +- Administration + +Recommended dashboard widgets: + +- recent recordings +- draft scenarios awaiting review +- recent replay failures +- healing proposals awaiting approval +- retention or configuration warnings + +This navigation shell should exist early in development so the product can be exercised visually in a local environment even while deeper capability slices are still being completed. + +## 5. Information Architecture + +### 5.1 Recording Workspace + +Supports `FR-REC-001` through `FR-REC-010` and Section 9.2. + +Core regions: + +- session setup panel +- browser status and connection state +- live step feed +- inline note and importance controls +- event detail drawer + +Critical states: + +- ready to start +- launching browser +- recording active +- paused +- stopping/finalizing +- failed with recovery guidance + +### 5.2 Timeline and Scenario Editor + +Supports `FR-AUTH-001` through `FR-AUTH-006` and Section 9.3. + +Core regions: + +- ordered step timeline +- step detail inspector +- locator candidates panel +- assertion suggestions panel +- variable classification panel +- validation summary banner + +Critical states: + +- draft dirty +- validation warning +- blocking validation error +- approval pending for suggested assertions +- ready to version +- version committed + +### 5.3 Code Preview Workspace + +Supports `FR-GEN-001` through `FR-GEN-010` and Section 9.4. + +Core regions: + +- file tree / manifest list +- generated code viewer +- warning panel for low-confidence decisions +- generation profile selector +- export summary + +Critical states: + +- no approved version selected +- generating +- generated successfully +- generation warning +- generation failed + +### 5.4 Replay Workspace + +Supports `FR-REP-001` through `FR-REP-005`, `FR-HEAL-001` through `FR-HEAL-005`, and Section 9.5. + +Core regions: + +- replay launch panel +- live step execution stream +- diagnostics sidebar +- failure evidence tabs +- healing proposal review card + +Critical states: + +- queued/starting +- running +- passed +- failed with diagnostics +- healing available +- healing approved/rejected + +### 5.5 Administration Area + +Supports `FR-ADM-001` through `FR-ADM-004`, Sections 11 and 12.7. + +Core regions: + +- environments and base URL allow-lists +- browser and replay defaults +- masking and sensitive-data rules +- retention policies +- authentication bootstrap options +- audit activity feed + +## 6. Screen-Level Plan + +### 6.1 Dashboard + +Purpose: + +- orient users quickly +- highlight work in progress and required action + +Recommended modules: + +- active recordings +- scenarios needing approval +- recent generations +- replay health summary +- outstanding healing reviews + +### 6.2 New Recording Flow + +User journey: + +1. enter scenario name +2. choose environment +3. enter target URL +4. choose browser and recording options +5. choose authentication bootstrap mode +6. pass server-side allow-list validation +7. start recording + +Validation expectations: + +- invalid URL format +- URL not in environment allow-list +- missing required environment settings +- unsupported authentication bootstrap selection + +### 6.3 Live Recording Screen + +Priority interactions: + +- pause/resume/stop without losing progress +- mark important or ignorable steps +- add inline notes at time of capture +- inspect current URL and recorder health + +Design note: + +The live feed should focus on meaningful user actions and collapse noise by default. + +### 6.4 Scenario Editor + +The editor should make system inference reviewable rather than magical. + +Recommended step row content: + +- sequence number +- semantic label +- step type badge +- locator confidence indicator +- assertion indicator +- variable/sensitive badges +- suppression state + +Recommended inspector tabs: + +- Overview +- Locator Candidates +- Assertions +- Data Strategy +- Context Evidence +- Notes and Audit + +Approval checkpoints in this screen: + +- suggested assertions remain visually distinct from approved assertions +- locator candidate changes should show previous and newly selected strategy +- sensitive-value classifications should update every preview surface immediately after save + +### 6.5 Generation Preview + +Users should see: + +- which scenario version is being generated +- which generation profile and template version were used +- file manifest with warnings +- source preview +- export destination summary + +Warnings should clearly separate: + +- low-confidence locator use +- unapproved suggestions excluded from output +- sensitive values replaced with secure references + +### 6.6 Replay and Healing + +Replay diagnostics should be layered: + +1. human-readable failure summary first +2. step-level timeline second +3. raw diagnostics and traces third + +Healing review should always show: + +- original locator +- proposed locator +- evidence for proposal +- confidence and deterministic basis +- explicit approval or rejection action + +Healing review must never imply automatic application in v1. The UI language should explicitly state that approval creates a new immutable scenario version. + +## 7. UX Rules for Long-Running Operations + +Because recording, generation, replay, export, and cleanup are asynchronous, the UI should: + +- use SignalR updates for live progress +- persist operation status so page reload does not lose state +- show elapsed time and current stage +- provide retry guidance when failures occur +- avoid blocking the entire app shell for operation-specific failures + +## 8. Local Visual Testing Expectations + +Because the product is intended to be developed and validated through its actual UI, the local developer environment should support visual testing of: + +- sign-in and application shell navigation +- recording session creation and recording status +- timeline editing and scenario validation feedback +- generation preview and warnings +- replay execution, diagnostics, and healing review surfaces + +Local visual testing should use the same Blazor Server UI that will ship, not a separate mock frontend. + +## 9. Accessibility Plan + +Required to support Section 15.6: + +- keyboard access for all recording, editing, approval, and replay actions +- semantic landmarks and headings +- live region announcements for recording/replay status changes +- sufficient contrast for confidence states and warnings +- non-color indicators for pass/warn/fail/confidence levels +- focus management when drawers, dialogs, or review panels open + +## 10. Responsive Behaviour + +The UI is desktop-first for v1, but should degrade gracefully. + +Recommended breakpoints: + +- desktop: multi-panel workflow surfaces +- tablet/small laptop: collapsible side panels +- narrow widths: stacked panels, reduced preview width, preserved key actions + +Do not hide critical validation, approval, or security warnings on smaller layouts. + +## 11. Design System Guidance + +Suggested component set: + +- status badge +- confidence pill +- timeline row +- evidence drawer +- split-pane code/diagnostic viewer +- validation summary banner +- audit event list + +Suggested state taxonomy: + +- neutral +- in-progress +- success +- warning +- blocking +- sensitive + +## 12. Source-of-Truth UX Rules + +To stay aligned with Sections 7.1 and 8.4: + +1. Users edit scenarios, not generated code. +2. Generated code views must clearly show the source scenario version and generation profile. +3. Replay and healing screens must always show which scenario version they are derived from. +4. Any action that changes persisted scenario behaviour must route through scenario versioning, not ad hoc direct mutation. + +## 13. UX Acceptance Checks + +The following checks should be true before UI slices are considered complete: + +1. An author can complete the Phase 1 recording-to-replay path without leaving the web UI. +2. Every inferred locator or assertion can be inspected with evidence before approval. +3. Sensitive values never appear in clear text in preview surfaces. +4. Long-running operations recover gracefully from refresh or reconnect. +5. Role-restricted actions are hidden or disabled with clear rationale. +6. A developer can run the local environment and visually exercise the primary UI workflows before pushing changes. diff --git a/src/Symphony.Infrastructure.Persistence.Sqlite/Symphony.Infrastructure.Persistence.Sqlite.csproj b/src/Symphony.Infrastructure.Persistence.Sqlite/Symphony.Infrastructure.Persistence.Sqlite.csproj index fd821b5..3f0e99f 100644 --- a/src/Symphony.Infrastructure.Persistence.Sqlite/Symphony.Infrastructure.Persistence.Sqlite.csproj +++ b/src/Symphony.Infrastructure.Persistence.Sqlite/Symphony.Infrastructure.Persistence.Sqlite.csproj @@ -4,16 +4,19 @@ - - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - - - + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + all + + net10.0