From 0a42c00867e2988c5a43e11b95a93a5a0df1140e Mon Sep 17 00:00:00 2001 From: Bill Berry Date: Wed, 27 May 2026 14:11:49 -0700 Subject: [PATCH] docs(tests): replace shell wrappers with npm + PowerShell guidance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - blueprints/full-single-node-cluster/README.md: update Testing section and 'When modifying' workflow to reference 'npm run go-test' and 'npm run go-test:deploy:*' instead of run-contract-tests.sh/run-deployment-tests.sh - blueprints/full-single-node-cluster/tests/README.md: rewrite Quick Start, Setup, Running Contract Tests, Running Deployment Tests, Automation Entry Points, and end-to-end example sections to use npm scripts; drop helper-script bullets and the 'jq' helper-script dependency - blueprints/full-single-node-cluster/tests/deploy_bicep_test.go: remove '(set in run-deployment-tests.sh)' comment now that env vars are set directly - blueprints/full-single-node-cluster/tests/deploy_terraform_test.go: same comment cleanup for the Terraform deployment test - blueprints/full-single-node-cluster/tests/run-contract-tests.sh: removed (superseded by 'npm run go-test') - blueprints/full-single-node-cluster/tests/run-deployment-tests.sh: removed (superseded by 'npm run go-test:deploy:*') - docs/contributing/testing-validation.md: switch Running Contract Tests, Running Deployment Tests, and example test-directory layout to npm scripts; drop 'helper scripts' bullet from the suite-overview and new-blueprint checklist - docs/getting-started/blueprint-developer.md: update the blueprint test-author checklist and Running Tests examples to point at 'npm run go-test' / 'npm run go-test:deploy:terraform' Closes #570 Relates to #562 🔒 - Generated by Copilot --- blueprints/full-single-node-cluster/README.md | 14 +- .../full-single-node-cluster/tests/README.md | 70 ++++---- .../tests/deploy_bicep_test.go | 2 +- .../tests/deploy_terraform_test.go | 2 +- .../tests/run-contract-tests.sh | 165 ------------------ .../tests/run-deployment-tests.sh | 163 ----------------- docs/contributing/testing-validation.md | 24 ++- docs/getting-started/blueprint-developer.md | 9 +- 8 files changed, 62 insertions(+), 387 deletions(-) delete mode 100755 blueprints/full-single-node-cluster/tests/run-contract-tests.sh delete mode 100755 blueprints/full-single-node-cluster/tests/run-deployment-tests.sh diff --git a/blueprints/full-single-node-cluster/README.md b/blueprints/full-single-node-cluster/README.md index d1c7bddf3..e8141f971 100644 --- a/blueprints/full-single-node-cluster/README.md +++ b/blueprints/full-single-node-cluster/README.md @@ -349,7 +349,7 @@ The tests directory contains contract tests and end-to-end deployment validation - **Contract Tests** - Fast static validation ensuring output declarations match test expectations (zero cost, no Azure resources) - **Deployment Tests** - Full end-to-end deployment validation with infrastructure creation and functional testing -- **Helper Scripts** - `run-contract-tests.sh` and `run-deployment-tests.sh` for simplified test execution +- **npm Entry Points** - Package scripts for local contract and deployment test execution **See:** [tests/README.md](tests/README.md) for complete testing documentation including setup, usage, and troubleshooting @@ -359,7 +359,7 @@ When modifying this blueprint: #### Before Making Changes -1. **Run contract tests** to establish baseline: `cd tests && ./run-contract-tests.sh both` +1. **Run contract tests** to establish baseline: `npm run go-test` 2. **Review test structure** in [tests/outputs.go](tests/outputs.go) to understand output contract #### After Making Changes @@ -368,16 +368,16 @@ When modifying this blueprint: - Update struct fields in [tests/outputs.go](tests/outputs.go) - Update framework configurations: [terraform/outputs.tf](terraform/outputs.tf) or [bicep/main.bicep](bicep/main.bicep) -2. **Run contract tests** to verify declarations: `cd tests && ./run-contract-tests.sh both` +2. **Run contract tests** to verify declarations: `npm run go-test` 3. **Update deployment tests** if changing deployment behavior: - Review [tests/deploy_terraform_test.go](tests/deploy_terraform_test.go) or [tests/deploy_bicep_test.go](tests/deploy_bicep_test.go) - Update validation logic in [tests/validation.go](tests/validation.go) if needed -4. **Run deployment tests** to validate changes: `cd tests && ./run-deployment-tests.sh ` - - Valid framework values: 'terraform', 'bicep', or 'both' - - Set `CLEANUP_RESOURCES=true` to auto-delete resources after testing - - Expected duration: 30-45 minutes for full deployment +4. **Run deployment tests** to validate changes: `npm run go-test:deploy:` + - Valid framework suffixes: `terraform` or `bicep`; use `npm run go-test:deploy` for both + - Set `CLEANUP_RESOURCES=true` to auto-delete resources after testing + - Expected duration: 30-45 minutes for full deployment 5. **Update documentation** if changing parameters, modules, or architecture: - Update this README.md with new parameters or module references diff --git a/blueprints/full-single-node-cluster/tests/README.md b/blueprints/full-single-node-cluster/tests/README.md index 3e684868b..9ae74bc3a 100644 --- a/blueprints/full-single-node-cluster/tests/README.md +++ b/blueprints/full-single-node-cluster/tests/README.md @@ -6,16 +6,16 @@ This directory contains comprehensive integration and contract tests for the [Fu ```bash # Contract tests (fast, zero-cost validation) -./run-contract-tests.sh both +npm run go-test # Terraform deployment tests (creates Azure resources) -./run-deployment-tests.sh terraform +npm run go-test:deploy:terraform -# Bicep deployment tests (creates Azure resources, auto-generates password) -./run-deployment-tests.sh bicep +# Bicep deployment tests (creates Azure resources) +npm run go-test:deploy:bicep # Run both deployment frameworks -./run-deployment-tests.sh both +npm run go-test:deploy ``` ## Test Architecture @@ -69,7 +69,6 @@ The test suite provides two complementary testing strategies: **Recommended:** - `kubectl` - Kubernetes cluster validation -- `jq` - JSON parsing in helper scripts ## Setup @@ -87,19 +86,26 @@ Run `go mod download` from the tests directory to fetch required Go packages. ### 3. Configure Azure Credentials (Deployment Tests Only) -Authenticate with `az login`, optionally set subscription with `az account set`, and verify with `az account show`. The deployment scripts automatically export `ARM_SUBSCRIPTION_ID`. - -**See:** [run-deployment-tests.sh](run-deployment-tests.sh) for automatic credential handling +Authenticate with `az login`, optionally set subscription with `az account set`, and verify with `az account show`. Set `ARM_SUBSCRIPTION_ID` before running deployment tests. ## Running Contract Tests Contract tests validate that all outputs defined in the `BlueprintOutputs` struct are properly declared in both Terraform and Bicep configurations. These tests run entirely offline and complete in seconds. -### Using Helper Script (Recommended) +### Using npm for Contract Tests + +Run contract tests from the repository root: + +```bash +npm run go-test +``` -Use `run-contract-tests.sh` with framework argument (`both`, `terraform`, or `bicep`) and optional `-v` flag for verbose output. +Run framework-specific contract tests when you only need one implementation: -**See:** [run-contract-tests.sh](run-contract-tests.sh) for script implementation +```bash +npm run go-test:contract:terraform +npm run go-test:contract:bicep +``` ### Using Go Test Directly @@ -112,21 +118,23 @@ Run `go test -v -run Contract` to execute all contract tests, or specify individ ## Running Deployment Tests -### Using the Helper Script (Recommended) - -The `run-deployment-tests.sh` script handles environment setup, auto-detection, and password generation. +### Using npm for Deployment Tests -**Usage:** Framework argument (`terraform`, `bicep`, or `both`) with optional `-v` verbose flag +Run deployment tests from the repository root after configuring Azure credentials and required environment variables. -**See:** [run-deployment-tests.sh](run-deployment-tests.sh) for complete implementation +```bash +npm run go-test:deploy:terraform +npm run go-test:deploy:bicep +npm run go-test:deploy +``` ### Environment Variables -**Auto-Detected Variables** (script handles these): +**Required Variables**: - `ARM_SUBSCRIPTION_ID` - Azure subscription ID from `az account show` -- `CUSTOM_LOCATIONS_OID` - Custom Locations RP object ID from Azure AD -- `ADMIN_PASSWORD` - Auto-generated secure password (if not provided) +- `CUSTOM_LOCATIONS_OID` - Custom Locations RP object ID from Azure AD for Bicep tests +- `ADMIN_PASSWORD` - VM admin password for Bicep tests **Configuration Variables** (override as needed): @@ -184,10 +192,10 @@ go test -v -run TestBicepFullSingleNodeClusterDeploy -timeout 2h - [validation.go](validation.go) - Comprehensive validation test suites for deployed infrastructure - [setup.go](setup.go) - Post-deployment configuration (Arc proxy, RBAC permissions) -### Automation Scripts +### Automation Entry Points -- [run-contract-tests.sh](run-contract-tests.sh) - Contract test runner with dependency checks -- [run-deployment-tests.sh](run-deployment-tests.sh) - Deployment test runner with auto-configuration +- `npm run go-test` - Static contract tests used by CI +- `npm run go-test:deploy:*` - Deployment test shortcuts for the local inner loop ## Test Files Reference @@ -344,10 +352,10 @@ Implemented in `validateMessagingInfrastructure()` (Terraform only): ```bash # Step 1: Validate IaC output contract (fast, no costs) -./run-contract-tests.sh both +npm run go-test # Step 2: Initial deployment (resources remain for inspection) -./run-deployment-tests.sh terraform +npm run go-test:deploy:terraform # Step 3: Iterate on validation logic without redeploying export SKIP_BICEP_DEPLOYMENT=true @@ -355,7 +363,7 @@ go test -v -run TestBicepFullSingleNodeClusterDeploy -timeout 30m # Step 4: Test cleanup functionality before final commit export CLEANUP_RESOURCES=true -./run-deployment-tests.sh terraform +npm run go-test:deploy:terraform ``` ### CI/CD Integration @@ -364,11 +372,11 @@ export CLEANUP_RESOURCES=true ```bash # Stage 1: PR validation (every commit, ~30 seconds) -./run-contract-tests.sh both +npm run go-test # Stage 2: Nightly integration tests (scheduled, ~90 minutes) export CLEANUP_RESOURCES=true -./run-deployment-tests.sh both +npm run go-test:deploy ``` ### Adding New Infrastructure Outputs @@ -385,14 +393,14 @@ export CLEANUP_RESOURCES=true # - Add struct field with terraform:"/" or bicep:"/" tag # Step 3: Verify contract validation passes -./run-contract-tests.sh both +npm run go-test # Step 4: Add validation logic (if needed) # - Edit validation.go # - Add assertions for new output values # Step 5: Run end-to-end deployment test -./run-deployment-tests.sh terraform +npm run go-test:deploy:terraform ``` ### Debugging Failed Deployments @@ -402,7 +410,7 @@ export CLEANUP_RESOURCES=true ```bash # Deploy without cleanup to inspect resources export CLEANUP_RESOURCES=false -./run-deployment-tests.sh terraform +npm run go-test:deploy:terraform # Manually inspect resources in Azure Portal or CLI az resource list --resource-group t6-terraform -o table diff --git a/blueprints/full-single-node-cluster/tests/deploy_bicep_test.go b/blueprints/full-single-node-cluster/tests/deploy_bicep_test.go index 8a068cd8b..bcf37a898 100644 --- a/blueprints/full-single-node-cluster/tests/deploy_bicep_test.go +++ b/blueprints/full-single-node-cluster/tests/deploy_bicep_test.go @@ -40,7 +40,7 @@ func TestBicepFullSingleNodeClusterDeploy(t *testing.T) { deploymentName = "bicep-deployment-test" } - // Get test configuration from environment variables (set in run-deployment-tests.sh) + // Get test configuration from environment variables. resourceGroupName := os.Getenv("TEST_RESOURCE_GROUP_NAME") require.NotEmpty(t, resourceGroupName, "TEST_RESOURCE_GROUP_NAME environment variable must be set") diff --git a/blueprints/full-single-node-cluster/tests/deploy_terraform_test.go b/blueprints/full-single-node-cluster/tests/deploy_terraform_test.go index 99ef88aca..61d8e82ad 100644 --- a/blueprints/full-single-node-cluster/tests/deploy_terraform_test.go +++ b/blueprints/full-single-node-cluster/tests/deploy_terraform_test.go @@ -22,7 +22,7 @@ func TestTerraformFullSingleNodeClusterDeploy(t *testing.T) { terraformDir := "../terraform" - // Get test configuration from environment variables (set in run-deployment-tests.sh) + // Get test configuration from environment variables. environment := os.Getenv("TEST_ENVIRONMENT") require.NotEmpty(t, environment, "TEST_ENVIRONMENT environment variable must be set") diff --git a/blueprints/full-single-node-cluster/tests/run-contract-tests.sh b/blueprints/full-single-node-cluster/tests/run-contract-tests.sh deleted file mode 100755 index 5c888de22..000000000 --- a/blueprints/full-single-node-cluster/tests/run-contract-tests.sh +++ /dev/null @@ -1,165 +0,0 @@ -#!/bin/bash -# Helper script to run contract tests for Terraform and Bicep -# Contract tests validate IaC outputs without deployment (fast, $0 cost) - -set -e - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -cd "$SCRIPT_DIR" - -# Color output -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -BLUE='\033[0;34m' -NC='\033[0m' # No Color - -print_usage() { - cat </dev/null; then - echo -e "${RED}✗ Go not found. Please install Go toolchain.${NC}" - exit 1 -fi -echo -e "${GREEN}✓ Go: $(go version | awk '{print $3}')${NC}" - -# Check terraform-docs -if [[ "$TEST_TYPE" == "terraform" || "$TEST_TYPE" == "both" ]]; then - if ! command -v terraform-docs &>/dev/null; then - echo -e "${RED}✗ terraform-docs not found${NC}" - echo -e "${YELLOW} Install: brew install terraform-docs${NC}" - exit 1 - fi - echo -e "${GREEN}✓ terraform-docs: $(terraform-docs version | head -n1)${NC}" -fi - -# Check az bicep -if [[ "$TEST_TYPE" == "bicep" || "$TEST_TYPE" == "both" ]]; then - if ! command -v az &>/dev/null; then - echo -e "${RED}✗ Azure CLI not found${NC}" - echo -e "${YELLOW} Install: https://docs.microsoft.com/cli/azure/install-azure-cli${NC}" - exit 1 - fi - - # Check bicep is installed - if ! az bicep version &>/dev/null; then - echo -e "${RED}✗ Bicep not installed${NC}" - echo -e "${YELLOW} Install: az bicep install${NC}" - exit 1 - fi -fi - -echo "" - -# Run tests -EXIT_CODE=0 - -run_test() { - local test_name=$1 - local test_pattern=$2 - - echo -e "${BLUE}──────────────────────────────────────────────────────────${NC}" - echo -e "${YELLOW}Running: $test_name${NC}" - echo -e "${BLUE}──────────────────────────────────────────────────────────${NC}" - - if go test $VERBOSE_FLAG -run "$test_pattern" .; then - echo -e "${GREEN}✓ $test_name PASSED${NC}" - else - echo -e "${RED}✗ $test_name FAILED${NC}" - EXIT_CODE=1 - fi - echo "" -} - -case $TEST_TYPE in - terraform) - run_test "Terraform Contract Test" "TestTerraformOutputsContract" - ;; - bicep) - run_test "Bicep Contract Test" "TestBicepOutputsContract" - ;; - both) - run_test "Terraform Contract Test" "TestTerraformOutputsContract" - run_test "Bicep Contract Test" "TestBicepOutputsContract" - ;; -esac - -# Summary -echo -e "${BLUE}╔════════════════════════════════════════════════════════════╗${NC}" -if [[ $EXIT_CODE -eq 0 ]]; then - echo -e "${BLUE}║${GREEN} All Tests PASSED ✓ ${BLUE}║${NC}" -else - echo -e "${BLUE}║${RED} Some Tests FAILED ✗ ${BLUE}║${NC}" -fi -echo -e "${BLUE}╚════════════════════════════════════════════════════════════╝${NC}" - -exit $EXIT_CODE diff --git a/blueprints/full-single-node-cluster/tests/run-deployment-tests.sh b/blueprints/full-single-node-cluster/tests/run-deployment-tests.sh deleted file mode 100755 index 2f5868fea..000000000 --- a/blueprints/full-single-node-cluster/tests/run-deployment-tests.sh +++ /dev/null @@ -1,163 +0,0 @@ -#!/bin/bash -# Helper script to run deployment tests for both Terraform and Bicep - -set -e - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -cd "$SCRIPT_DIR" - -# Color output -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -NC='\033[0m' # No Color - -print_usage() { - echo "Usage: $0 [terraform|bicep|both] [options]" - echo "" - echo "Arguments:" - echo " terraform Run only Terraform deployment tests" - echo " bicep Run only Bicep deployment tests" - echo " both Run both Terraform and Bicep tests (default)" - echo "" - echo "Options:" - echo " -v, --verbose Enable verbose test output" - echo " -h, --help Show this help message" - echo "" - echo "Environment Variables:" - echo " ARM_SUBSCRIPTION_ID Azure subscription ID (auto-detected if not set)" - echo " ADMIN_PASSWORD (Required for Bicep) VM admin password" - echo " CUSTOM_LOCATIONS_OID Custom Locations OID (auto-detected if not set)" - echo "" - echo "Examples:" - echo " $0 terraform" - echo " $0 bicep -v" - echo " $0 both" -} - -# Parse arguments -DEPLOYMENT_TYPE="both" -VERBOSE_FLAG="" - -while [[ $# -gt 0 ]]; do - case $1 in - terraform | bicep | both) - DEPLOYMENT_TYPE="$1" - shift - ;; - -v | --verbose) - VERBOSE_FLAG="-v" - shift - ;; - -h | --help) - print_usage - exit 0 - ;; - *) - echo -e "${RED}Unknown option: $1${NC}" - print_usage - exit 1 - ;; - esac -done - -# Auto-detect ARM_SUBSCRIPTION_ID if not set -if [[ -z "${ARM_SUBSCRIPTION_ID}" ]]; then - echo -e "${YELLOW}ARM_SUBSCRIPTION_ID not set, detecting from Azure CLI...${NC}" - ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv 2>/dev/null) - if [[ -z "${ARM_SUBSCRIPTION_ID}" ]]; then - echo -e "${RED}Error: Could not auto-detect ARM_SUBSCRIPTION_ID. Please run 'az login' or set ARM_SUBSCRIPTION_ID${NC}" - exit 1 - fi - echo -e "${GREEN}Detected subscription: ${ARM_SUBSCRIPTION_ID}${NC}" - export ARM_SUBSCRIPTION_ID -fi - -# Auto-detect CUSTOM_LOCATIONS_OID if not set (for Bicep tests) -if [[ -z "${CUSTOM_LOCATIONS_OID}" ]] && [[ "$DEPLOYMENT_TYPE" == "bicep" || "$DEPLOYMENT_TYPE" == "both" ]]; then - echo -e "${YELLOW}CUSTOM_LOCATIONS_OID not set, detecting from Azure AD...${NC}" - CUSTOM_LOCATIONS_OID=$(az ad sp show --id bc313c14-388c-4e7d-a58e-70017303ee3b --query id -o tsv 2>/dev/null) - if [[ -z "${CUSTOM_LOCATIONS_OID}" ]]; then - echo -e "${RED}Error: Could not auto-detect CUSTOM_LOCATIONS_OID. Please ensure you have permissions to query Azure AD${NC}" - exit 1 - fi - echo -e "${GREEN}Detected Custom Locations OID: ${CUSTOM_LOCATIONS_OID}${NC}" - export CUSTOM_LOCATIONS_OID -fi - -# Generate strong admin password if not provided (for Bicep tests) -if [[ -z "${ADMIN_PASSWORD}" ]] && [[ "$DEPLOYMENT_TYPE" == "bicep" || "$DEPLOYMENT_TYPE" == "both" ]]; then - echo -e "${YELLOW}ADMIN_PASSWORD not set, generating strong password...${NC}" - ADMIN_PASSWORD=$(openssl rand -base64 32 | tr -d "=+/" | cut -c1-24) - # Ensure password meets Azure complexity requirements (uppercase, lowercase, digit, special char) - ADMIN_PASSWORD="Aa1!${ADMIN_PASSWORD}" - echo -e "${GREEN}Generated admin password (save this): ${ADMIN_PASSWORD}${NC}" - export ADMIN_PASSWORD -fi - -echo -e "${GREEN}=== Deployment Tests ===${NC}" -# Set default test configuration values -export TEST_ENVIRONMENT="${TEST_ENVIRONMENT:-dev}" -export TEST_LOCATION="${TEST_LOCATION:-eastus2}" -export TEST_RESOURCE_PREFIX="${TEST_RESOURCE_PREFIX:-t$RANDOM}" -export TEST_INSTANCE="${TEST_INSTANCE:-001}" - -echo "Deployment Type: $DEPLOYMENT_TYPE" -echo "ARM Subscription: ${ARM_SUBSCRIPTION_ID}" -echo "Resource Prefix: ${TEST_RESOURCE_PREFIX}" -echo "Location: ${TEST_LOCATION}" -echo "" - -run_terraform_tests() { - export TEST_RESOURCE_GROUP_NAME="${TEST_RESOURCE_GROUP_NAME_PREFIX:-test-}terraform" - echo "Resource Group: ${TEST_RESOURCE_GROUP_NAME}" - - echo -e "${YELLOW}Running Terraform deployment tests...${NC}" - if go test $VERBOSE_FLAG -run TestTerraformFullSingleNodeClusterDeploy -timeout 2h; then - echo -e "${GREEN}✓ Terraform tests passed${NC}" - return 0 - else - echo -e "${RED}✗ Terraform tests failed${NC}" - return 1 - fi -} - -run_bicep_tests() { - export TEST_RESOURCE_GROUP_NAME="${TEST_RESOURCE_GROUP_NAME_PREFIX:-test-}bicep" - echo "Resource Group: ${TEST_RESOURCE_GROUP_NAME}" - - echo -e "${YELLOW}Running Bicep deployment tests...${NC}" - if go test $VERBOSE_FLAG -run TestBicepFullSingleNodeClusterDeploy -timeout 2h; then - echo -e "${GREEN}✓ Bicep tests passed${NC}" - return 0 - else - echo -e "${RED}✗ Bicep tests failed${NC}" - return 1 - fi -} - -# Run tests based on deployment type -EXIT_CODE=0 - -case $DEPLOYMENT_TYPE in - terraform) - run_terraform_tests || EXIT_CODE=$? - ;; - bicep) - run_bicep_tests || EXIT_CODE=$? - ;; - both) - run_terraform_tests || EXIT_CODE=$? - echo "" - run_bicep_tests || EXIT_CODE=$? - ;; -esac - -echo "" -if [[ $EXIT_CODE -eq 0 ]]; then - echo -e "${GREEN}=== All tests completed successfully ===${NC}" -else - echo -e "${RED}=== Some tests failed ===${NC}" -fi - -exit $EXIT_CODE diff --git a/docs/contributing/testing-validation.md b/docs/contributing/testing-validation.md index cf92de56c..f7747bb3f 100644 --- a/docs/contributing/testing-validation.md +++ b/docs/contributing/testing-validation.md @@ -333,7 +333,7 @@ Complete test suite demonstrating: - Contract tests for both Terraform and Bicep - End-to-end deployment validation -- Helper scripts for test execution +- npm scripts for test execution - Output contract definitions ### Contract Testing @@ -350,16 +350,15 @@ Complete test suite demonstrating: **Running Contract Tests:** ```bash -cd blueprints/full-single-node-cluster/tests - # Test both frameworks -./run-contract-tests.sh both +npm run go-test # Test specific framework -./run-contract-tests.sh terraform -./run-contract-tests.sh bicep +npm run go-test:contract:terraform +npm run go-test:contract:bicep # Direct Go execution +cd blueprints/full-single-node-cluster/tests go test -v -run Contract ``` @@ -377,16 +376,15 @@ go test -v -run Contract **Running Deployment Tests:** ```bash -cd blueprints/full-single-node-cluster/tests - # Enable automatic cleanup export CLEANUP_RESOURCES=true # Test specific framework -./run-deployment-tests.sh terraform -./run-deployment-tests.sh bicep +npm run go-test:deploy:terraform +npm run go-test:deploy:bicep # Direct Go execution +cd blueprints/full-single-node-cluster/tests go test -v -run TestTerraformFullSingleNodeClusterDeploy -timeout 2h go test -v -run TestBicepFullSingleNodeClusterDeploy -timeout 2h ``` @@ -411,9 +409,7 @@ blueprints/{blueprint-name}/tests/ ├── deploy_terraform_test.go # Terraform deployment test ├── deploy_bicep_test.go # Bicep deployment test ├── validation.go # Shared validation functions -├── setup.go # Post-deployment setup -├── run-contract-tests.sh # Contract test runner -└── run-deployment-tests.sh # Deployment test runner +└── setup.go # Post-deployment setup ``` ### Blueprint Integration Testing @@ -443,7 +439,7 @@ When creating a new blueprint, add comprehensive test coverage: 1. **Define output contract** in `tests/outputs.go` with struct tags for both frameworks 2. **Create contract tests** for static validation 3. **Create deployment tests** for end-to-end validation -4. **Add helper scripts** for simplified test execution +4. **Add package scripts** for local and CI test execution 5. **Document test requirements** in blueprint README **See:** [Blueprint Developer Guide](../getting-started/blueprint-developer.md#testing-and-validation) for detailed instructions diff --git a/docs/getting-started/blueprint-developer.md b/docs/getting-started/blueprint-developer.md index d51970b77..08dd4c80f 100644 --- a/docs/getting-started/blueprint-developer.md +++ b/docs/getting-started/blueprint-developer.md @@ -347,17 +347,16 @@ Selected blueprints include comprehensive test suites using Go and the Terratest **See:** [full-single-node-cluster/tests/deploy_terraform_test.go](https://github.com/microsoft/edge-ai/blob/main/blueprints/full-single-node-cluster/tests/deploy_terraform_test.go) -6. **Add helper scripts**: +6. **Add npm entry points**: - Copy and adapt `run-contract-tests.sh` and `run-deployment-tests.sh` from the reference implementation. + Add package scripts that run the Go contract and deployment tests for the blueprint. ### Running Tests **Contract tests** (fast, run before every commit): ```bash -cd blueprints/my-custom-blueprint/tests -./run-contract-tests.sh both +npm run go-test ``` **Deployment tests** (slow, run before PR): @@ -367,7 +366,7 @@ cd blueprints/my-custom-blueprint/tests export CLEANUP_RESOURCES=true # Run tests -./run-deployment-tests.sh terraform # or bicep, or both +npm run go-test:deploy:terraform ``` **Documentation:** See [src/900-tools-utilities/904-test-utilities/README.md](https://github.com/microsoft/edge-ai/blob/main/src/900-tools-utilities/904-test-utilities/README.md) for complete testing guide and API reference