ci: add test tags (priority/criticality/layer) and split smoke/regression in CI

.github/workflows/robot-tests.yml

@@ -5,7 +5,22 @@ on:
     branches: [ main ]
   push:
     branches: [ main ]
-  workflow_dispatch: {}   # allows manual trigger from the Actions tab
+  # Nightly full regression run — DISABLED for now.
+  # To turn it back on: uncomment the two lines below and merge this file
+  # into `main` (GitHub only reads `schedule` triggers from the workflow
+  # file version on the repo's default branch, not from feature branches).
+  # schedule:
+  #   - cron: '0 3 * * *'   # 03:00 UTC = 00:00 (meia-noite) em Brasília
+  workflow_dispatch:        # allows manual trigger from the Actions tab
+    inputs:
+      tag:
+        description: 'Robot tag to run'
+        required: false
+        default: regression
+        type: choice
+        options:
+          - smoke
+          - regression
 
 concurrency:
   group: robot-tests-${{ github.ref }}
@@ -21,6 +36,21 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v4
 
+      - name: Determine which tag to run
+        id: tag
+        shell: bash
+        run: |
+          # PRs get the fast "smoke" subset for quick feedback; pushes to
+          # main and the nightly schedule run the full "regression" suite.
+          # workflow_dispatch lets a human pick either one on demand.
+          if [ "${{ github.event_name }}" = "pull_request" ]; then
+            echo "value=smoke" >> "$GITHUB_OUTPUT"
+          elif [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
+            echo "value=${{ github.event.inputs.tag }}" >> "$GITHUB_OUTPUT"
+          else
+            echo "value=regression" >> "$GITHUB_OUTPUT"
+          fi
+
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
@@ -53,6 +83,7 @@ jobs:
         run: |
           robot \
             --variable HEADLESS:True \
+            --include ${{ steps.tag.outputs.value }} \
             --outputdir results \
             --xunit xunit.xml \
             tests/
@@ -61,14 +92,21 @@ jobs:
         # global_page.resource's "New Browser" keyword (CLI --variable takes
         # precedence over the *** Variables *** default), so the suite runs
         # headless in CI while local runs keep the resource's own default.
+        # --include filters by [Tags]: "smoke" is the small, fast,
+        # critical-path subset run on every PR; "regression" covers the
+        # whole suite (every test carries it) and runs on merges to main
+        # and the nightly schedule.
 
       - name: Publish summary on GitHub Actions
         if: always()
         run: |
           {
             echo "### Robot Framework Suite Result"
+            echo "Tag executed: \`${{ steps.tag.outputs.value }}\`"
             if [ -f results/output.xml ]; then
-              echo "Full report available in the **robot-framework-results** artifact."
+              echo "Full report (including the **Statistics by Tag** breakdown"
+              echo "for smoke/regression/critical/high/medium and ui) is"
+              echo "available in the **robot-framework-results** artifact."
             fi
           } >> "$GITHUB_STEP_SUMMARY"
 

README.md

@@ -264,17 +264,50 @@ button on the Cadastro de Usuário card`), which simply delegate to the
 `newUser_page.resource` / `newProduct_page.resource` / `home_page.resource`
 keywords described above.
 
-## Continuous Integration
+## Tags
+
+Every test case carries three kinds of `[Tags]`, so the suite can be sliced
+without touching test code:
+
+| Dimension        | Tags                              | Meaning |
+|-------------------|------------------------------------|---------|
+| **Execution set**  | `smoke`, `regression`             | `regression` is on every test (the full suite). `smoke` marks the small, fast subset of critical happy paths — currently 5 of the 14 tests — meant to run on every PR for quick feedback. |
+| **Criticality**    | `critical`, `high`, `medium`      | `critical` = core journeys the app is unusable without (login, admin create user/product, add to cart). `high` = important supporting flows (listing, quantity, clearing). `medium` = negative/validation edge cases. |
+| **Layer**          | `ui`                               | All current tests drive the browser end-to-end (API is only used for setup/teardown). Kept as an explicit tag so future API-only suites can be filtered out (`--exclude ui`) or in (`--include ui`) separately. |
+
+Module tags (`login`, `admin`, `store`/`list`) are also kept so a single
+feature area can be run in isolation, e.g. `robot --include admin tests/`.
+
+Tags are combinable, e.g. run every critical test regardless of area:
 
-`.github/workflows/robot-tests.yml` runs the full suite on GitHub Actions:
+```bash
+robot --include critical tests/
+```
+
+Robot Framework's `report.html` automatically breaks down pass/fail stats
+per tag ("Statistics by Tag") with no extra flags required — once the tags
+above are applied, that view already gives a success rate by criticality
+(`critical`/`high`/`medium`) for free, which is what gets reported to
+stakeholders.
+
+## Continuous Integration
 
-- **Triggers**: pull requests and pushes to `main`, plus manual runs via
-  `workflow_dispatch`. Runs are cancelled/deduped per branch (`concurrency`).
+`.github/workflows/robot-tests.yml` runs the suite on GitHub Actions, using
+tags to control how much of it runs per trigger:
+
+- **Triggers**:
+  - `pull_request` to `main` → runs `--include smoke` only, for fast PR
+    feedback.
+  - `push` to `main` and a nightly `schedule` (03:00 UTC) → run
+    `--include regression`, the full suite.
+  - `workflow_dispatch` → manual run with a `tag` input (`smoke` or
+    `regression`, defaults to `regression`).
+  - Runs are cancelled/deduped per branch (`concurrency`).
 - Sets up Python and Node.js (the Browser library driver needs Node),
   installs `requirements.txt`, and runs `rfbrowser init` (with the
   downloaded Playwright browsers cached across runs).
-- Executes `robot --outputdir results --xunit xunit.xml tests/`; the job
-  fails if any test fails.
+- Executes `robot --include <tag> --outputdir results --xunit xunit.xml
+  tests/`; the job fails if any test fails.
 - Uploads `log.html`, `report.html`, and `output.xml` as the
   `robot-framework-results` artifact, and `xunit.xml` as the
   `robot-junit-report` artifact (both kept for 15 days), and writes a short

tests/admin/admin.robot

@@ -11,7 +11,7 @@ Test Teardown    Cleanup Login Test
 *** Test Cases ***
 Admin Wants To Create a New User
     [Documentation]    This test case verifies that an admin can create a new user.
-    [Tags]    admin    smoke
+    [Tags]    admin    ui    smoke    regression    critical
     ${name}    ${email}    ${password}=    Generate Random User
     Given the admin wants to create a new user   
     When the admin clicks on the "Cadastrar" button on the Cadastro de Usuário card
@@ -21,14 +21,14 @@ Admin Wants To Create a New User
 
 Admin Wants To See The List Of Users
     [Documentation]    This test case verifies that an admin can see the list of users.
-    [Tags]    admin    smoke
+    [Tags]    admin    ui    regression    high
     Given the admin wants to see the list of users
     When the admin clicks on the "Listar" button on the Listar Usuários card
     Then the list of users should be displayed
 
 Admin Wants To Create A New Product
     [Documentation]    This test case verifies that an admin can create a new product.
-    [Tags]    admin    smoke
+    [Tags]    admin    ui    smoke    regression    critical
     ${productName}      ${price}    ${description}      ${quantity}=    Generate Random Product
     Given the admin wants to create a new product
     When the admin clicks on the "Cadastrar" button on the Cadastrar Produtos card
@@ -38,7 +38,7 @@ Admin Wants To Create A New Product
 
 Admin Wants To List All Products
     [Documentation]    This test case verifies that an admin can see the list of products.
-    [Tags]    admin    smoke
+    [Tags]    admin    ui    regression    high
     Given the admin wants to see the list of products
     When the admin clicks on the "Listar" button on the Listar Produtos card
     Then the list of products should be displayed

tests/login/login.robot

@@ -13,7 +13,7 @@ Test Teardown    Cleanup Login Test
 User With Valid Credentials Should Be Redirected To The Home
     [Documentation]    A user who enters a valid email and password and clicks
     ...                "Entrar" must be redirected to the home page.
-    [Tags]    login    smoke
+    [Tags]    login    ui    smoke    regression    critical
     Given a user is trying to login
     When this user types the email    ${EMAIL}
     And this user types the password    ${PASSWORD}
@@ -22,7 +22,7 @@ User With Valid Credentials Should Be Redirected To The Home
 
 Admin Can Access Admin Page
     [Documentation]    This test case verifies that an admin user can access the admin page.
-    [Tags]    admin    smoke
+    [Tags]    admin    login    ui    smoke    regression    critical
     Given the admin user wants to access the admin page   
     When this admin user types the email and password
     Then the admin page should be displayed
@@ -31,7 +31,7 @@ User With Invalid Email Should See Error Message
     [Documentation]    A user who enters an invalid email and/or password and clicks
     ...                "Entrar" must see a message informing that the
     ...                credentials are invalid.
-    [Tags]    login
+    [Tags]    login    ui    regression    medium
     Given a user is trying to login
     When this user types the email    email.invalido@qa.com.br
     And this user types the password    ${PASSWORD}
@@ -42,7 +42,7 @@ User With Invalid Password Should See Error Message
     [Documentation]    A user who enters an invalid email and/or password and clicks
     ...                "Entrar" must see a message informing that the
     ...                credentials are invalid.
-    [Tags]    login
+    [Tags]    login    ui    regression    medium
     Given a user is trying to login
     When this user types the email    ${EMAIL}
     And this user types the password    senhaInvalida123
@@ -53,7 +53,7 @@ User With Blank Email Should See Error Message
     [Documentation]    A user who enters an invalid email and/or password and clicks
     ...                "Entrar" must see a message informing that the
     ...                credentials are invalid.
-    [Tags]    login
+    [Tags]    login    ui    regression    medium
     Given a user is trying to login
     When this user types the password     ${PASSWORD}
     And clicks on Entrar
@@ -63,7 +63,7 @@ User With Blank Password Should See Error Message
     [Documentation]    A user who enters an invalid email and/or password and clicks
     ...                "Entrar" must see a message informing that the
     ...                credentials are invalid.
-    [Tags]    login
+    [Tags]    login    ui    regression    medium
     Given a user is trying to login
     When this user types the email    ${EMAIL}
     And clicks on Entrar
@@ -72,7 +72,7 @@ User With Blank Password Should See Error Message
 User With Invalid Email Format Should See Error Message
     [Documentation]    A user who enters a valid email and password and clicks
     ...                "Entrar" must be redirected to the home page.
-    [Tags]    login    smoke
+    [Tags]    login    ui    regression    medium
     Given a user is trying to login
     When this user types the email    teste@teste
     And this user types the password    ${PASSWORD}

tests/store/store.robot

@@ -10,23 +10,23 @@ Test Teardown    Cleanup Login Test
 *** Test Cases ***
 User Can Add 2 Items to List
     [Documentation]    This test case verifies that a user can add two items to the list.
-    [Tags]    list    smoke
+    [Tags]    store    list    ui    smoke    regression    critical
     Given the user wants to add 2 items to the list   
     When the user adds the first item to the list   Logitech MX Vertical
     And the user adds the second item to the list   Samsung 60 polegadas
     Then the list should contain 2 items
 
 User Can Increase Quantity of an Item in the List
     [Documentation]    This test case verifies that a user can increase the quantity of an item in the list.
-    [Tags]    list    smoke
+    [Tags]    store    list    ui    regression    high
     Given the user wants to increase item in the list   
     When the user adds an item to the list   Logitech MX Vertical
     And the user increases the quantity of the item in the list
     Then the quantity of the item in the list should be bigger than 1
 
 User Can Clear the List
     [Documentation]    This test case verifies that a user can clear the list.
-    [Tags]    list    smoke
+    [Tags]    store    list    ui    regression    high
     Given the user wants to clear the list   
     When the user adds an item to the list   Logitech MX Vertical
     And the user clears the list