Add quick and full demo tenant smoke tests

[merill]

Summary

• rename the workflow to Demo tenant Invoke-Maester smoke test
• add a secretless approved-review listener and a trusted default-branch quick runner for MT.1068
• publish a stable Demo tenant quick smoke check on the current PR test merge commit
• run the full public Graph suite on every push to main and on the weekly schedule
• keep Exchange, Teams, private, preview, and long-running tests disabled
• distinguish quick and full private HTML report assets
• document the protected environment, private report repository, trigger model, and all 24 read-only Graph application permissions

Security

The review-event workflow has no token permissions, secrets, checkout, or OIDC access. The credentialed workflow_run workflow is loaded from main, never executes PR code or artifacts, and independently verifies the source job, open PR, current head and test merge SHAs, current approval, and reviewer repository permission before creating a check or entering the protected maester-smoke-test environment.

The dedicated Entra app has been updated out of band to declare and consent exactly the 24 default read-only Microsoft Graph application permissions returned by Get-MtGraphScope. It has no permissions for other resource APIs, no Azure role, no client secret, and no certificate credential. Its federated credential remains scoped to repo:maester365/maester:environment:maester-smoke-test.

Validation

• actionlint v1.7.12 on both workflows
• Ruby YAML parser on both workflows
• git diff --check
• Get-MtGraphScope.Tests.ps1: 4 passed
• verified the source and permission documentation resolve to the same 24 Graph scopes
• audited the service principal: 24 assignments, all to Microsoft Graph

Rollout

Do not add the new required check before this workflow reaches main; doing that would deadlock this pull request. After merge, let an approved PR create Demo tenant quick smoke, then add that exact context to active ruleset Protect main (ID 533197). At that point, a failed or absent quick run blocks merging.

Draft PR #1990 fixes the existing NotRun warning flood and is recommended before the first full run, but remains a separate change.

Summary by CodeRabbit

• New Features

  • Added an approval-triggered demo tenant quick smoke test for eligible pull requests.
  • Added configurable quick and full smoke-test runs through pushes, schedules, manual dispatch, and approved pull request workflows.
  • Added validation and status reporting through a dedicated “Demo tenant quick smoke” check.
  • Added private HTML report publishing with updated run and asset naming.

• Documentation

  • Updated smoke-test workflow guidance, authentication setup, permissions, review processes, and merge-gate requirements.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 13fc1b14-2efb-4ee8-a50b-30ff7aec9485

📥 Commits

Reviewing files that changed from the base of the PR and between 02e0db0 and ef41d2c.

📒 Files selected for processing (3)

• .github/workflows/demo-tenant-quick-approval-trigger.yaml
• .github/workflows/maester-action-smoke-test.md
• .github/workflows/maester-action-smoke-test.yaml

---

📝 Walkthrough

Walkthrough

Adds a trusted approval-trigger workflow and rewrites the Demo tenant smoke-test workflow to validate approval context, support quick/full runs, verify results, publish private reports, and complete PR checks. Documentation updates describe configuration, permissions, report handling, and merge-gate behavior.

Changes

Demo tenant smoke testing

Layer / File(s)	Summary
Approval-trigger workflow .github/workflows/demo-tenant-quick-approval-trigger.yaml	Approved reviews on non-draft pull requests targeting main trigger a secretless workflow that requests a trusted quick smoke test.
Workflow-run context validation .github/workflows/maester-action-smoke-test.yaml	The Demo tenant workflow supports quick, full, scheduled, and manual runs, validates workflow-run approval context, derives parameters, and creates PR check linkage.
Smoke execution and reporting .github/workflows/maester-action-smoke-test.yaml	The smoke job selects tags by run type, applies revised result thresholds, publishes private HTML reports, and completes the Demo-tenant quick smoke check.
Smoke-test configuration and operating procedure .github/workflows/maester-action-smoke-test.md	Documentation updates cover run modes, protected secrets, Graph application permissions, private reports, and the approval merge-gate strategy.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Reviewer
  participant ApprovalTrigger
  participant SmokeWorkflow
  participant Maester
  participant PrivateReportRelease
  participant GitHubChecks
  Reviewer->>ApprovalTrigger: submit approved review
  ApprovalTrigger->>SmokeWorkflow: trigger workflow_run
  SmokeWorkflow->>SmokeWorkflow: validate approval and derive run parameters
  SmokeWorkflow->>Maester: run Demo tenant smoke tests
  Maester-->>SmokeWorkflow: return test results
  SmokeWorkflow->>PrivateReportRelease: publish HTML report
  SmokeWorkflow->>GitHubChecks: complete PR smoke check

Loading

Possibly related PRs

• maester365/maester#1986: Reworks the same smoke-test workflow and quick/full result-verification behavior.
• maester365/maester#1988: Uses a related trusted approval mechanism and updated smoke-test gating logic.

Suggested labels: documentation, enhancement, github_actions, automation

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/demo-tenant-quick-full-smoke

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}

[cloudflare-workers-and-pages]

Deploying maester with    Cloudflare Pages

Latest commit:	a6fa2ba
Status:	✅  Deploy successful!
Preview URL:	https://eb81bf14.maester.pages.dev
Branch Preview URL:	https://codex-demo-tenant-quick-full.maester.pages.dev

View logs