Gauge External Parameters Plugin

A powerful Gauge plugin that enables dynamic resolution of placeholders in spec files from external sources like environment variables, JSON/YAML files, HTTP APIs, HashiCorp Vault, AWS Secrets Manager, and Kubernetes ConfigMaps/Secrets.

Features

🔐 Multiple Data Sources: Environment variables, files, HTTP APIs, Vault, AWS Secrets Manager, Kubernetes
🎯 Flexible Syntax: <name:source#key|default> format with fallback support
⚡ Smart Caching: Configurable TTL and automatic cache invalidation
🔄 Source Precedence: env > file > vault/aws/k8s > http > default
🛡️ Security: Automatic secret masking in logs and error messages
🔧 Preprocessor Mode: CLI tool for spec transformation in CI pipelines
📊 Comprehensive Logging: Detailed error reporting and debugging support

Installation

Using Gauge Package Manager

gauge install gauge-external-params

Manual Installation

# Clone the repository
git clone https://github.com/your-org/gauge-external-params.git
cd gauge-external-params

# Install dependencies
npm install

# Build and package
npm pack

# Install the plugin
gauge install gauge-external-params --file gauge-external-params-1.0.0.tgz

Quick Start

Create Configuration File

Create gauge-external-params.json in your project root:

{
  "cacheTimeout": 60,
  "sources": {
    "env": { "enabled": true },
    "file": { "enabled": true, "basePath": "." },
    "http": { "enabled": true, "timeout": 3000 }
  }
}

Use Placeholders in Specs

# Login Feature
## Admin Login
* Login with user <admin_user:env#ADMIN_USER|admin@example.com> and password <admin_pass:file#secrets.json#admin_password>

Set Environment Variables

export ADMIN_USER="admin@mycompany.com"

Create Secrets File

{
  "admin_password": "secure_password_123"
}

Run Tests

gauge run specs/

Placeholder Syntax

Basic Format

<name:source#key|default>

name: Descriptive identifier for the placeholder
source: Data source type (env, file, http, vault, aws, k8s)
key: Source-specific key or path
default: Optional fallback value

Examples by Source

Environment Variables

<user:env#USERNAME>
<token:env#API_TOKEN|default-token>
<url:env#SERVICE_URL>

JSON/YAML Files

<user:file#secrets.json#admin_user>
<host:file#config.yaml#database.host>
<config:file#app-settings.json#api.endpoints.users>

HTTP APIs

<token:http#https://auth-service.com/token>
<config:http#https://config-api.com/settings#database.url>
<data:http#POST:https://api.com/auth:{"user":"admin"}#access_token>

HashiCorp Vault

<secret:vault#secret/myapp:password>
<token:vault#secret/tokens:api_key>
<config:vault#secret/config>

AWS Secrets Manager

<secret:aws#prod/myapp/db:password>
<token:aws#prod/api-keys:github_token>
<config:aws#prod/config@AWSPENDING:database_url>

Kubernetes

<secret:k8s#secret:my-secret:password>
<config:k8s#configmap:app-config:database.url>
<token:k8s#secret:prod-namespace/api-tokens:github>

Configuration

Complete Configuration Example

{
  "cacheTimeout": 60,
  "sources": {
    "env": {
      "enabled": true,
      "prefix": "APP_",
      "transformCase": "upper"
    },
    "file": {
      "enabled": true,
      "basePath": "./config",
      "allowedExtensions": [".json", ".yaml", ".yml"],
      "cacheFiles": true,
      "maxFileSize": 1048576
    },
    "http": {
      "enabled": true,
      "timeout": 3000,
      "retries": 2,
      "baseURL": "https://api.example.com",
      "headers": {
        "User-Agent": "gauge-external-params/1.0.0"
      },
      "auth": {
        "token": "bearer-token-here"
      },
      "cacheResponses": true,
      "cacheTimeout": 300000
    },
    "vault": {
      "enabled": true,
      "url": "https://vault.example.com",
      "token": "vault-token-here",
      "namespace": "myapp",
      "mount": "secret",
      "version": "v2",
      "timeout": 5000,
      "retries": 2,
      "cacheTimeout": 300000
    },
    "aws": {
      "enabled": true,
      "region": "us-west-2",
      "profile": "production",
      "roleArn": "arn:aws:iam::123456789012:role/gauge-secrets-role",
      "timeout": 5000,
      "retries": 2,
      "cacheTimeout": 300000
    },
    "k8s": {
      "enabled": true,
      "kubeconfig": "~/.kube/config",
      "namespace": "production",
      "context": "prod-cluster",
      "timeout": 5000,
      "retries": 2,
      "cacheTimeout": 120000
    }
  },
  "logging": {
    "level": "info",
    "maskSecrets": true
  }
}

Source Configuration Details

Environment Variables (`env`)

prefix: Add prefix to all environment variable names
transformCase: Transform case (upper, lower, none)

File Source (`file`)

basePath: Base directory for relative file paths
allowedExtensions: Allowed file extensions for security
cacheFiles: Enable file content caching
maxFileSize: Maximum file size in bytes

HTTP Source (`http`)

baseURL: Base URL for relative requests
timeout: Request timeout in milliseconds
retries: Number of retry attempts
headers: Default headers for all requests
auth: Authentication configuration

Vault Source (`vault`)

url: Vault server URL
token: Vault authentication token
namespace: Vault namespace (Vault Enterprise)
mount: KV secret engine mount path
version: KV engine version (v1 or v2)

AWS Secrets Source (`aws`)

region: AWS region
profile: AWS CLI profile name
roleArn: IAM role to assume
accessKeyId, secretAccessKey: Direct credentials

Kubernetes Source (`k8s`)

kubeconfig: Path to kubeconfig file
namespace: Default namespace
context: Kubernetes context to use

Usage Modes

Plugin Mode (Recommended)

The plugin automatically resolves placeholders during test execution:

gauge run specs/

Preprocessor Mode

For CI/CD pipelines or when plugin mode isn't available:

# Process specs and output resolved versions
npx gauge-external-params preprocess --spec-dir specs/ --out-dir specs_resolved/

# Run tests with resolved specs
gauge run specs_resolved/

CLI Options

# Start plugin server
npx gauge-external-params start

# Preprocess specs
npx gauge-external-params preprocess --spec-dir specs/ --out-dir resolved/

# Validate placeholders
npx gauge-external-params validate --spec-dir specs/

# Show usage statistics
npx gauge-external-params stats --spec-dir specs/

CI/CD Integration

GitHub Actions

name: Gauge Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Setup Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '16'
          
      - name: Install Gauge
        run: |
          curl -SsL https://downloads.gauge.org/stable | sh
          gauge install java
          
      - name: Install dependencies
        run: npm install
        
      - name: Install Gauge External Params
        run: gauge install gauge-external-params
        
      - name: Set environment variables
        env:
          ADMIN_USER: ${{ secrets.ADMIN_USER }}
          API_TOKEN: ${{ secrets.API_TOKEN }}
        run: |
          echo "ADMIN_USER=$ADMIN_USER" >> $GITHUB_ENV
          echo "API_TOKEN=$API_TOKEN" >> $GITHUB_ENV
          
      - name: Run tests
        run: gauge run specs/

Jenkins Pipeline

pipeline {
    agent any
    
    environment {
        ADMIN_USER = credentials('admin-user')
        API_TOKEN = credentials('api-token')
    }
    
    stages {
        stage('Setup') {
            steps {
                sh 'npm install'
                sh 'gauge install gauge-external-params'
            }
        }
        
        stage('Test') {
            steps {
                sh 'gauge run specs/'
            }
        }
    }
    
    post {
        always {
            publishHTML([
                allowMissing: false,
                alwaysLinkToLastBuild: true,
                keepAll: true,
                reportDir: 'reports/html-report',
                reportFiles: 'index.html',
                reportName: 'Gauge Test Report'
            ])
        }
    }
}

GitLab CI

stages:
  - test

test:
  stage: test
  image: node:16
  before_script:
    - curl -SsL https://downloads.gauge.org/stable | sh
    - gauge install java
    - npm install
    - gauge install gauge-external-params
  script:
    - gauge run specs/
  variables:
    ADMIN_USER: $ADMIN_USER
    API_TOKEN: $API_TOKEN
  artifacts:
    reports:
      junit: reports/xml-report/result.xml
    paths:
      - reports/

Security Best Practices

1. Environment Variables

# Use secure variable storage in CI/CD
export VAULT_TOKEN="hvs.secret_token_here"
export AWS_SECRET_ACCESS_KEY="secret_key_here"

2. File Permissions

# Restrict access to sensitive files
chmod 600 secrets.json
chmod 600 ~/.kube/config

3. Vault Authentication

# Use short-lived tokens
vault write -field=token auth/aws/login role=gauge-runner

4. AWS IAM Policies

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-west-2:123456789012:secret:prod/myapp/*"
    }
  ]
}

5. Kubernetes RBAC

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: gauge-external-params
rules:
- apiGroups: [""]
  resources: ["secrets", "configmaps"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: gauge-external-params
subjects:
- kind: ServiceAccount
  name: gauge-runner
roleRef:
  kind: Role
  name: gauge-external-params
  apiGroup: rbac.authorization.k8s.io

Troubleshooting

Common Issues

1. Plugin Not Starting

# Check Gauge plugin directory
gauge --version
ls ~/.gauge/plugins/

# Reinstall plugin
gauge uninstall gauge-external-params
gauge install gauge-external-params

2. Configuration Not Found

# Verify config file location
ls -la gauge-external-params.json

# Use absolute path
export GAUGE_EXTERNAL_PARAMS_CONFIG=/absolute/path/to/config.json

3. Source Authentication Failures

# Check environment variables
env | grep -E "(VAULT_|AWS_|KUBE)"

# Test source connectivity
npx gauge-external-params test-sources

4. Placeholder Resolution Errors

# Enable verbose logging
export GAUGE_EXTERNAL_PARAMS_VERBOSE=true

# Validate placeholders
npx gauge-external-params validate --spec-dir specs/

Debug Mode

Enable debug logging:

{
  "logging": {
    "level": "debug",
    "maskSecrets": false
  }
}

Logs Location

Plugin logs: ~/.gauge/logs/
Application logs: ./logs/gauge-external-params.log

Development

Setup Development Environment

git clone https://github.com/your-org/gauge-external-params.git
cd gauge-external-params
npm install

Run Tests

# Unit tests
npm test

# Integration tests
npm run test:integration

# Test with coverage
npm run test:coverage

Build and Package

# Build the plugin
npm run build

# Create package
npm pack

# Install locally for testing
gauge install gauge-external-params --file gauge-external-params-1.0.0.tgz

API Reference

ParamResolver Class

const ParamResolver = require('gauge-external-params/src/resolver/ParamResolver');

const resolver = new ParamResolver('./config.json');
await resolver.initialize();

// Resolve text with placeholders
const resolved = await resolver.resolveText('Hello <user:env#USERNAME>!');

// Parse placeholder syntax
const parsed = ParamResolver.parsePlaceholder('<user:env#USERNAME|admin>');

await resolver.cleanup();

Source Interfaces

All sources implement the same interface:

class SourceInterface {
  async initialize() { /* Setup source */ }
  async resolve(key) { /* Resolve value for key */ }
  async cleanup() { /* Cleanup resources */ }
  async refreshCache() { /* Clear cached data */ }
}

Contributing

Fork the repository
Create a feature branch: git checkout -b feature/amazing-feature
Commit changes: git commit -m 'Add amazing feature'
Push to branch: git push origin feature/amazing-feature
Open a Pull Request

Code Style

Use ESLint configuration
Follow conventional commit messages
Add tests for new features
Update documentation

License

MIT License - see LICENSE file for details.

Support

Changelog

v1.0.0

Initial release
Support for 6 data sources
Plugin and preprocessor modes
Comprehensive configuration options
Full test coverage

Name		Name	Last commit message	Last commit date
.github		.github
bin		bin
examples		examples
proto		proto
src		src
test		test
.gitignore		.gitignore
.scripts.json		.scripts.json
LICENSE		LICENSE
README.md		README.md
datadog-ci.json		datadog-ci.json
gauge-external-params.json		gauge-external-params.json
index.js		index.js
package-lock.json		package-lock.json
package.json		package.json
secrets.json		secrets.json

Latest commit

History

Folders and files

Repository files navigation