feat(terraform): conditional s3 integrator support

[MichaelThamm]

Issue

Solution

Summary

Two major changes to the Mimir Terraform module:

1. Replace three hardcoded worker modules with a single for_each-based module supporting dynamic roles including monolithic mode
2. Make the S3 integrator conditional — callers can use external storage backends (e.g. SeaweedFS)

Motivation

• Worker duplication: mimir_backend, mimir_read, mimir_write were identical modules differing only in the role config key. This made it impossible to deploy in monolithic mode (role-all) without a code change.
• Storage flexibility: The S3 integrator was always deployed, preventing use cases where storage is provided externally (e.g. SeaweedFS integrated at the product module level).

Dynamic Workers

The three worker modules are replaced by a single module.mimir_worker with for_each over var.workers:

# Microservices (default):
workers = {
  backend = { units = 3, storage_directives = { "data" = "50G" } }
  read    = { units = 2 }
  write   = { units = 2 }
}

# Monolithic:
workers = {
  all = { units = 3 }
}

Each worker entry supports: units, config, constraints, storage_directives, app_name (all optional with sensible defaults). The role config (role-backend = true, etc.) is injected automatically from the map key.

Validations:

• Keys must be one of: all, backend, read, write
• all cannot coexist with other role keys
• Units must be >= 1

Anti-affinity: The top-level var.anti_affinity bool overrides per-worker constraints when enabled, computing anti-affinity tags from the resolved app name.

Optional S3 Integrator

When var.s3_endpoint is null (the new default), no S3 resources are created:

• juju_secret.mimir_s3_credentials_secret
• juju_access_secret.mimir_s3_secret_access
• juju_application.s3_integrator
• juju_integration.coordinator_to_s3_integrator

The caller is responsible for integrating their chosen storage backend with the coordinator's s3 endpoint.

Validation: s3_endpoint, s3_access_key, and s3_secret_key must all be set or all be null.

Breaking Changes

Before	After
var.backend_name / var.read_name / var.write_name	var.workers.<role>.app_name
var.backend_config / var.read_config / var.write_config	var.workers.<role>.config
var.backend_units / var.read_units / var.write_units	var.workers.<role>.units
var.backend_worker_storage_directives / etc.	var.workers.<role>.storage_directives
var.worker_constraints	var.workers.<role>.constraints
var.s3_endpoint (required)	var.s3_endpoint (optional, default null)
var.s3_access_key (required)	var.s3_access_key (optional, default null)
var.s3_secret_key (required)	var.s3_secret_key (optional, default null)
module.mimir_backend	module.mimir_worker["backend"]
juju_integration.coordinator_to_backend	juju_integration.coordinator_to_worker["backend"]

State migration: Existing deployments will need to recreate worker and integration resources due to the for_each address change. S3 resources gain a [0] index.

Unchanged

• Coordinator module and its variables
• var.worker_resources and var.worker_revision (shared across all workers)
• provides / requires outputs
• All s3_integrator_* customization variables (channel, name, config, etc.)

Context

Testing Instructions

Upgrade Notes