Skip to content

docs(tests): npm/PowerShell wrappers, remove legacy shells (#562 4/4) #574

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
WilliamBerryiii wants to merge 1 commit into
base: feature/562-pr3-ci-wiring
Choose a base branch
from
Open

docs(tests): npm/PowerShell wrappers, remove legacy shells (#562 4/4) #574

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions blueprints/full-single-node-cluster/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ The tests directory contains contract tests and end-to-end deployment validation

- **Contract Tests** - Fast static validation ensuring output declarations match test expectations (zero cost, no Azure resources)
- **Deployment Tests** - Full end-to-end deployment validation with infrastructure creation and functional testing
- **Helper Scripts** - `run-contract-tests.sh` and `run-deployment-tests.sh` for simplified test execution
- **npm Entry Points** - Package scripts for local contract and deployment test execution

**See:** [tests/README.md](tests/README.md) for complete testing documentation including setup, usage, and troubleshooting

Expand All @@ -359,7 +359,7 @@ When modifying this blueprint:

#### Before Making Changes

1. **Run contract tests** to establish baseline: `cd tests && ./run-contract-tests.sh both`
1. **Run contract tests** to establish baseline: `npm run go-test`
2. **Review test structure** in [tests/outputs.go](tests/outputs.go) to understand output contract

#### After Making Changes
Expand All @@ -368,16 +368,16 @@ When modifying this blueprint:
- Update struct fields in [tests/outputs.go](tests/outputs.go)
- Update framework configurations: [terraform/outputs.tf](terraform/outputs.tf) or [bicep/main.bicep](bicep/main.bicep)

2. **Run contract tests** to verify declarations: `cd tests && ./run-contract-tests.sh both`
2. **Run contract tests** to verify declarations: `npm run go-test`

3. **Update deployment tests** if changing deployment behavior:
- Review [tests/deploy_terraform_test.go](tests/deploy_terraform_test.go) or [tests/deploy_bicep_test.go](tests/deploy_bicep_test.go)
- Update validation logic in [tests/validation.go](tests/validation.go) if needed

4. **Run deployment tests** to validate changes: `cd tests && ./run-deployment-tests.sh <framework>`
- Valid framework values: 'terraform', 'bicep', or 'both'
- Set `CLEANUP_RESOURCES=true` to auto-delete resources after testing
- Expected duration: 30-45 minutes for full deployment
4. **Run deployment tests** to validate changes: `npm run go-test:deploy:<framework>`
- Valid framework suffixes: `terraform` or `bicep`; use `npm run go-test:deploy` for both
- Set `CLEANUP_RESOURCES=true` to auto-delete resources after testing
- Expected duration: 30-45 minutes for full deployment

5. **Update documentation** if changing parameters, modules, or architecture:
- Update this README.md with new parameters or module references
Expand Down
70 changes: 39 additions & 31 deletions blueprints/full-single-node-cluster/tests/README.md
View file
Open in desktop

@rezatnoMsirhC rezatnoMsirhC Jun 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Environment Variables Reference" section contains an "Auto-Detected Variables" subsection header and lists ARM_SUBSCRIPTION_ID and CUSTOM_LOCATIONS_OID with a fallback of "Script fails if not detected". The shell scripts that performed this auto-detection (run-deployment-tests.sh) are removed in this PR, and the npm deployment scripts (go-test:deploy:*) call go test directly without any env-var detection. This directly contradicts the updated Setup section added by this PR ("Set ARM_SUBSCRIPTION_ID before running deployment tests").

Consider renaming the subsection from "Auto-Detected Variables" to "Required Variables" and updating the Fallback column to reflect that these must now be set manually.

@rezatnoMsirhC rezatnoMsirhC Jun 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Key Differences: Terraform vs Bicep Tests" table has two cells that still reference the deleted shell scripts: (+) Required (auto-generated by script) for ADMIN_PASSWORD and (+) Required (auto-detected by script) for Custom Locations OID. With run-deployment-tests.sh removed, neither the npm scripts nor the Go tests auto-generate or auto-detect these values.

Suggested update:

  • ADMIN_PASSWORD: (+) Required (set via ADMIN_PASSWORD env var)
  • CUSTOM_LOCATIONS_OID: (+) Required (set via CUSTOM_LOCATIONS_OID env var)

Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@ This directory contains comprehensive integration and contract tests for the [Fu

```bash
# Contract tests (fast, zero-cost validation)
./run-contract-tests.sh both
npm run go-test

# Terraform deployment tests (creates Azure resources)
./run-deployment-tests.sh terraform
npm run go-test:deploy:terraform

# Bicep deployment tests (creates Azure resources, auto-generates password)
./run-deployment-tests.sh bicep
# Bicep deployment tests (creates Azure resources)
npm run go-test:deploy:bicep

# Run both deployment frameworks
./run-deployment-tests.sh both
npm run go-test:deploy
```

## Test Architecture
Expand Down Expand Up @@ -69,7 +69,6 @@ The test suite provides two complementary testing strategies:
**Recommended:**

- `kubectl` - Kubernetes cluster validation
- `jq` - JSON parsing in helper scripts

## Setup

Expand All @@ -87,19 +86,26 @@ Run `go mod download` from the tests directory to fetch required Go packages.

### 3. Configure Azure Credentials (Deployment Tests Only)

Authenticate with `az login`, optionally set subscription with `az account set`, and verify with `az account show`. The deployment scripts automatically export `ARM_SUBSCRIPTION_ID`.

**See:** [run-deployment-tests.sh](run-deployment-tests.sh) for automatic credential handling
Authenticate with `az login`, optionally set subscription with `az account set`, and verify with `az account show`. Set `ARM_SUBSCRIPTION_ID` before running deployment tests.

## Running Contract Tests

Contract tests validate that all outputs defined in the `BlueprintOutputs` struct are properly declared in both Terraform and Bicep configurations. These tests run entirely offline and complete in seconds.

### Using Helper Script (Recommended)
### Using npm for Contract Tests

Run contract tests from the repository root:

```bash
npm run go-test
```

Use `run-contract-tests.sh` with framework argument (`both`, `terraform`, or `bicep`) and optional `-v` flag for verbose output.
Run framework-specific contract tests when you only need one implementation:

**See:** [run-contract-tests.sh](run-contract-tests.sh) for script implementation
```bash
npm run go-test:contract:terraform
npm run go-test:contract:bicep
```

### Using Go Test Directly

Expand All @@ -112,21 +118,23 @@ Run `go test -v -run Contract` to execute all contract tests, or specify individ

## Running Deployment Tests

### Using the Helper Script (Recommended)

The `run-deployment-tests.sh` script handles environment setup, auto-detection, and password generation.
### Using npm for Deployment Tests

**Usage:** Framework argument (`terraform`, `bicep`, or `both`) with optional `-v` verbose flag
Run deployment tests from the repository root after configuring Azure credentials and required environment variables.

**See:** [run-deployment-tests.sh](run-deployment-tests.sh) for complete implementation
```bash
npm run go-test:deploy:terraform
npm run go-test:deploy:bicep
npm run go-test:deploy
```

### Environment Variables

**Auto-Detected Variables** (script handles these):
**Required Variables**:

- `ARM_SUBSCRIPTION_ID` - Azure subscription ID from `az account show`
- `CUSTOM_LOCATIONS_OID` - Custom Locations RP object ID from Azure AD
- `ADMIN_PASSWORD` - Auto-generated secure password (if not provided)
- `CUSTOM_LOCATIONS_OID` - Custom Locations RP object ID from Azure AD for Bicep tests
- `ADMIN_PASSWORD` - VM admin password for Bicep tests

**Configuration Variables** (override as needed):

Expand Down Expand Up @@ -184,10 +192,10 @@ go test -v -run TestBicepFullSingleNodeClusterDeploy -timeout 2h
- [validation.go](validation.go) - Comprehensive validation test suites for deployed infrastructure
- [setup.go](setup.go) - Post-deployment configuration (Arc proxy, RBAC permissions)

### Automation Scripts
### Automation Entry Points

- [run-contract-tests.sh](run-contract-tests.sh) - Contract test runner with dependency checks
- [run-deployment-tests.sh](run-deployment-tests.sh) - Deployment test runner with auto-configuration
- `npm run go-test` - Static contract tests used by CI
- `npm run go-test:deploy:*` - Deployment test shortcuts for the local inner loop

## Test Files Reference

Expand Down Expand Up @@ -344,18 +352,18 @@ Implemented in `validateMessagingInfrastructure()` (Terraform only):

```bash
# Step 1: Validate IaC output contract (fast, no costs)
./run-contract-tests.sh both
npm run go-test

# Step 2: Initial deployment (resources remain for inspection)
./run-deployment-tests.sh terraform
npm run go-test:deploy:terraform

# Step 3: Iterate on validation logic without redeploying
export SKIP_BICEP_DEPLOYMENT=true
go test -v -run TestBicepFullSingleNodeClusterDeploy -timeout 30m

# Step 4: Test cleanup functionality before final commit
export CLEANUP_RESOURCES=true
./run-deployment-tests.sh terraform
npm run go-test:deploy:terraform
```

### CI/CD Integration
Expand All @@ -364,11 +372,11 @@ export CLEANUP_RESOURCES=true

```bash
# Stage 1: PR validation (every commit, ~30 seconds)
./run-contract-tests.sh both
npm run go-test

# Stage 2: Nightly integration tests (scheduled, ~90 minutes)
export CLEANUP_RESOURCES=true
./run-deployment-tests.sh both
npm run go-test:deploy
```

### Adding New Infrastructure Outputs
Expand All @@ -385,14 +393,14 @@ export CLEANUP_RESOURCES=true
# - Add struct field with terraform:"/" or bicep:"/" tag

# Step 3: Verify contract validation passes
./run-contract-tests.sh both
npm run go-test

# Step 4: Add validation logic (if needed)
# - Edit validation.go
# - Add assertions for new output values

# Step 5: Run end-to-end deployment test
./run-deployment-tests.sh terraform
npm run go-test:deploy:terraform
```

### Debugging Failed Deployments
Expand All @@ -402,7 +410,7 @@ export CLEANUP_RESOURCES=true
```bash
# Deploy without cleanup to inspect resources
export CLEANUP_RESOURCES=false
./run-deployment-tests.sh terraform
npm run go-test:deploy:terraform

# Manually inspect resources in Azure Portal or CLI
az resource list --resource-group t6-terraform -o table
Expand Down
2 changes: 1 addition & 1 deletion blueprints/full-single-node-cluster/tests/deploy_bicep_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ func TestBicepFullSingleNodeClusterDeploy(t *testing.T) {
deploymentName = "bicep-deployment-test"
}

// Get test configuration from environment variables (set in run-deployment-tests.sh)
// Get test configuration from environment variables.
resourceGroupName := os.Getenv("TEST_RESOURCE_GROUP_NAME")
require.NotEmpty(t, resourceGroupName, "TEST_RESOURCE_GROUP_NAME environment variable must be set")

Expand Down
2 changes: 1 addition & 1 deletion blueprints/full-single-node-cluster/tests/deploy_terraform_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ func TestTerraformFullSingleNodeClusterDeploy(t *testing.T) {

terraformDir := "../terraform"

// Get test configuration from environment variables (set in run-deployment-tests.sh)
// Get test configuration from environment variables.
environment := os.Getenv("TEST_ENVIRONMENT")
require.NotEmpty(t, environment, "TEST_ENVIRONMENT environment variable must be set")

Expand Down
165 changes: 0 additions & 165 deletions blueprints/full-single-node-cluster/tests/run-contract-tests.sh
View file
Open in desktop

This file was deleted.

Loading