diff --git a/packaging/helm/openwork-ee/README.md b/packaging/helm/openwork-ee/README.md index 97d7419ff8..66a2f41b97 100644 --- a/packaging/helm/openwork-ee/README.md +++ b/packaging/helm/openwork-ee/README.md @@ -284,20 +284,86 @@ Published self-host planning pages: ## Secrets -The chart can create an Opaque Secret from `secret.values`, or consume an existing Secret: +The deployment declares how it manages secrets with a single key: ```yaml secret: + secretsMode: inline # inline | existingSecret | externalSecrets +``` + +- `inline` (default): the chart renders an Opaque Secret from `secret.values`. + Local evaluation only — the values live wherever the values file lives, so + never commit real credentials. +- `existingSecret`: workloads consume a pre-created Secret named by + `secret.existingSecret` (requires `secret.create: false`); the chart renders + no secret resource. +- `externalSecrets`: the chart renders an + [External Secrets Operator](https://external-secrets.io/) `ExternalSecret` + that materializes the workload Secret from an external provider — the + GitOps/ArgoCD-safe path, where git holds only store references and remote + key paths (requires `secret.create: false`). + +Any `secret.keys` override applies in every mode, since all three resolve the +workload Secret through the same names. The mode combinations are enforced at +render time: an unknown `secretsMode`, a missing `secret.existingSecret` in +`existingSecret` mode, `secret.existingSecret` set in any other mode, and +`secret.create: true` outside `inline` mode all fail the render. + +### existingSecret + +```yaml +secret: + secretsMode: existingSecret create: false existingSecret: openwork-ee-secrets ``` -The existing Secret must contain the keys listed under `secret.keys`, especially: +The existing Secret must be created in the `namespace` where the chart deploys +(default `openwork`) before the workloads start, and must contain the keys +listed under `secret.keys`, especially: - `DATABASE_URL` - `BETTER_AUTH_SECRET` - `DEN_DB_ENCRYPTION_KEY` +### externalSecrets (GitOps / ArgoCD) + +For GitOps flows (ArgoCD runs `helm template`, so anything in values lands in +git and in rendered manifests), use ESO mode. The chart renders an +`ExternalSecret` that materializes the same-named workload Secret from your +provider in-cluster: + +The chart renders `spec.data` — the oldest stable ESO shape, unchanged since +`external-secrets.io/v1beta1` — pulling every `secret.keys.*` entry from +`/` in the provider. The key list is generated from +`secret.keys`, so it can never drift from what the workloads consume: + +```yaml +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + # References an existing (Cluster)SecretStore; for AWS Secrets Manager the + # store itself carries spec.provider.aws (region, auth), the chart only + # points at it by name. + name: external-secrets + kind: ClusterSecretStore + refreshInterval: 5m + # Every secret.keys.* value must exist as a JSON property under this trunk, + # e.g. eks/openwork/prod/den/DATABASE_URL. + pathPrefix: "eks/openwork/prod/den" +``` + +Every property the workloads consume must exist in your provider under +`pathPrefix`, named like `secret.keys.*` values (`DATABASE_URL`, +`BETTER_AUTH_SECRET`, ...) — the chart pulls each key by name and cannot +invent missing ones. `target.deletionPolicy` defaults to `Retain`, so +uninstalling the release keeps the materialized Secret. ESO must be installed +on the destination cluster with a `SecretStore`/`ClusterSecretStore`; the +chart selects `external-secrets.io/v1` or `v1beta1` from cluster capabilities +and fails loudly at sync time if the CRDs are missing. + Set optional `DATABASE_REDIS_URL` to enable Den API Redis-backed session and query caching. Set `DAYTONA_API_KEY` when `config.provisioner.mode` is `daytona`. Set `POLAR_ACCESS_TOKEN` when Polar feature gating is enabled. Set `OPENROUTER_MANAGEMENT_API_KEY` when enabling OpenWork Models management. Redis cache examples: diff --git a/packaging/helm/openwork-ee/templates/_helpers.tpl b/packaging/helm/openwork-ee/templates/_helpers.tpl index a90fc818b0..e842fb3794 100644 --- a/packaging/helm/openwork-ee/templates/_helpers.tpl +++ b/packaging/helm/openwork-ee/templates/_helpers.tpl @@ -7,7 +7,13 @@ {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}} {{- else -}} {{- $name := default .Chart.Name .Values.nameOverride -}} -{{- if contains $name .Release.Name -}} +{{- /* + Dedupe in both directions: contains handles release names that already + include the chart name (my-openwork-ee); hasPrefix handles release names + that prefix the chart name (release "openwork", chart "openwork-ee"), which + would otherwise produce doubled names like openwork-openwork-ee-secret. +*/ -}} +{{- if or (contains $name .Release.Name) (hasPrefix .Release.Name $name) -}} {{- .Release.Name | trunc 63 | trimSuffix "-" -}} {{- else -}} {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} @@ -59,11 +65,101 @@ app.kubernetes.io/component: {{ .component }} {{- end -}} {{- end -}} +{{/* + Returns the workload Secret name as a quoted string: existingSecret values + are user-supplied and may look like YAML scalars (true, 1234), which would + otherwise render non-string manifest fields and fail at apply time. + Consumers that need the bare name trim the quotes. +*/}} {{- define "openwork-ee.secretName" -}} +{{- if eq .Values.secret.secretsMode "existingSecret" -}} +{{- .Values.secret.existingSecret | toString | trim | quote -}} +{{- else -}} +{{- printf "%s-secret" (include "openwork-ee.fullname" .) | quote -}} +{{- end -}} +{{- end -}} + +{{/* Bare (unquoted) Secret name for contexts that need it (e.g. lookup). */}} +{{- define "openwork-ee.secretNameRaw" -}} +{{- include "openwork-ee.secretName" . | trimAll "\"" -}} +{{- end -}} + +{{/* Bare (unquoted) namespace name for contexts that need it. */}} +{{- define "openwork-ee.namespaceRaw" -}} +{{- include "openwork-ee.namespace" . | trimAll "\"" -}} +{{- end -}} + +{{- define "openwork-ee.secretsMode.validate" -}} +{{- if not (has .Values.secret.secretsMode (list "inline" "existingSecret" "externalSecrets")) -}} +{{- fail "secretsMode must be one of inline, existingSecret, externalSecrets" -}} +{{- end -}} +{{- if eq .Values.secret.secretsMode "existingSecret" -}} +{{- if not (.Values.secret.existingSecret | toString | trim) -}} +{{- fail "secret.existingSecret is required when secretsMode=existingSecret" -}} +{{- end -}} +{{- end -}} +{{- if ne .Values.secret.secretsMode "existingSecret" -}} {{- if .Values.secret.existingSecret -}} -{{- .Values.secret.existingSecret -}} +{{- fail "secret.existingSecret is only allowed when secretsMode=existingSecret" -}} +{{- end -}} +{{- end -}} +{{- if ne .Values.secret.secretsMode "inline" -}} +{{- if .Values.secret.create -}} +{{- fail "secret.create must be false when secretsMode is not inline" -}} +{{- end -}} +{{- end -}} +{{- if and (eq .Values.secret.secretsMode "inline") (not .Values.secret.create) -}} +{{- /* Legacy migration shim: values files from before secretsMode shipped that + set create=false with untouched placeholder values meant "no inline + secrets" — treat that as existingSecret mode. Real-looking values with + create=false are incoherent and must fail, not be silently rerouted. */ -}} +{{- $dsn := .Values.secret.values.databaseUrl | toString -}} +{{- $auth := .Values.secret.values.betterAuthSecret | toString -}} +{{- $enc := .Values.secret.values.denDbEncryptionKey | toString -}} +{{- /* + Reroute only when ALL three required values are still placeholders: a + partially-filled inline block means someone set real values and create=false + is incoherent — fail rather than silently ignoring their real values. +*/ -}} +{{- $dsnIsPlaceholder := or (contains "change-me@" $dsn) (contains "******" $dsn) -}} +{{- if and (hasPrefix "CHANGE_ME" $auth) (hasPrefix "CHANGE_ME" $enc) $dsnIsPlaceholder -}} +{{- $_ := set .Values.secret "secretsMode" "existingSecret" -}} +{{- if not (.Values.secret.existingSecret | toString | trim) -}} +{{- $_ := set .Values.secret "existingSecret" (include "openwork-ee.fullname" . | printf "%s-secret") -}} +{{- end -}} {{- else -}} -{{- include "openwork-ee.fullname" . }}-secret +{{- fail "secret.create must be true when secretsMode=inline (set secretsMode=existingSecret or externalSecrets to source secrets externally)" -}} +{{- end -}} +{{- end -}} +{{- end -}} + +{{- define "openwork-ee.externalSecrets.apiVersion" -}} +{{- if .Capabilities.APIVersions.Has "external-secrets.io/v1" -}} +external-secrets.io/v1 +{{- else -}} +external-secrets.io/v1beta1 +{{- end -}} +{{- end -}} + +{{- define "openwork-ee.externalSecrets.validate" -}} +{{- if eq .Values.secret.secretsMode "externalSecrets" -}} +{{- $storeName := "" -}} +{{- if .Values.externalSecrets.secretStoreRef -}} +{{- $storeName = .Values.externalSecrets.secretStoreRef.name | toString | trim -}} +{{- end -}} +{{- if not $storeName -}} +{{- fail "externalSecrets.secretStoreRef.name is required when secretsMode=externalSecrets" -}} +{{- end -}} +{{- $storeKind := "" -}} +{{- if .Values.externalSecrets.secretStoreRef -}} +{{- $storeKind = .Values.externalSecrets.secretStoreRef.kind | toString -}} +{{- end -}} +{{- if not (has $storeKind (list "SecretStore" "ClusterSecretStore")) -}} +{{- fail "externalSecrets.secretStoreRef.kind must be SecretStore or ClusterSecretStore" -}} +{{- end -}} +{{- if not (.Values.externalSecrets.pathPrefix | toString | trim) -}} +{{- fail "externalSecrets.pathPrefix is required when secretsMode=externalSecrets" -}} +{{- end -}} {{- end -}} {{- end -}} diff --git a/packaging/helm/openwork-ee/templates/configmap.yaml b/packaging/helm/openwork-ee/templates/configmap.yaml index 01c51ef30d..4c84f95490 100644 --- a/packaging/helm/openwork-ee/templates/configmap.yaml +++ b/packaging/helm/openwork-ee/templates/configmap.yaml @@ -1,3 +1,4 @@ +{{- include "openwork-ee.secretsMode.validate" . }} apiVersion: v1 kind: ConfigMap metadata: diff --git a/packaging/helm/openwork-ee/templates/externalsecret.yaml b/packaging/helm/openwork-ee/templates/externalsecret.yaml new file mode 100644 index 0000000000..caa803c8fd --- /dev/null +++ b/packaging/helm/openwork-ee/templates/externalsecret.yaml @@ -0,0 +1,43 @@ +{{- include "openwork-ee.secretsMode.validate" . }} +{{- if eq .Values.secret.secretsMode "externalSecrets" }} +{{- include "openwork-ee.externalSecrets.validate" . }} +apiVersion: {{ include "openwork-ee.externalSecrets.apiVersion" . }} +kind: ExternalSecret +metadata: + name: {{ include "openwork-ee.secretName" . }} + namespace: {{ include "openwork-ee.namespace" . }} + labels: + {{- include "openwork-ee.labels" . | nindent 4 }} + {{- /* + When the migration Job runs as a pre-install/pre-upgrade hook, the + ExternalSecret must apply first so ESO can start materializing the Secret + the Job consumes. Weight -10 precedes the Job's -5. ArgoCD maps these to + PreSync the same way, so GitOps installs get the same ordering. + */}} + {{- if .Values.migrations.hook }} + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-10" + "helm.sh/hook-delete-policy": before-hook-creation + {{- end }} +spec: + refreshInterval: {{ .Values.externalSecrets.refreshInterval | quote }} + secretStoreRef: + name: {{ .Values.externalSecrets.secretStoreRef.name | toString | trim | quote }} + kind: {{ .Values.externalSecrets.secretStoreRef.kind }} + target: + name: {{ include "openwork-ee.secretName" . }} + creationPolicy: {{ .Values.externalSecrets.target.creationPolicy | default "Owner" }} + deletionPolicy: {{ .Values.externalSecrets.target.deletionPolicy | default "Retain" }} + data: + {{- $prefix := .Values.externalSecrets.pathPrefix | toString | trim | trimSuffix "/" }} + {{- range $name := keys .Values.secret.keys | sortAlpha }} + {{- $envKey := index $.Values.secret.keys $name }} + - secretKey: {{ $envKey | quote }} + remoteRef: + key: {{ printf "%s/%s" $prefix $envKey | quote }} + conversionStrategy: {{ $.Values.externalSecrets.conversionStrategy }} + decodingStrategy: {{ $.Values.externalSecrets.decodingStrategy }} + metadataPolicy: {{ $.Values.externalSecrets.metadataPolicy }} + {{- end }} +{{- end }} diff --git a/packaging/helm/openwork-ee/templates/migration-job.yaml b/packaging/helm/openwork-ee/templates/migration-job.yaml index ef26df86f1..bbdd32a92a 100644 --- a/packaging/helm/openwork-ee/templates/migration-job.yaml +++ b/packaging/helm/openwork-ee/templates/migration-job.yaml @@ -1,4 +1,6 @@ {{- include "openwork-ee.customCa.validate" . }} +{{- /* Runs the legacy create=false shim before any secretsMode check below. */}} +{{- include "openwork-ee.secretsMode.validate" . }} {{- if .Values.migrations.enabled }} apiVersion: batch/v1 kind: Job @@ -28,6 +30,9 @@ spec: {{- end }} spec: restartPolicy: Never + {{- if ne .Values.secret.secretsMode "inline" }} + serviceAccountName: {{ include "openwork-ee.fullname" . }}-migrate + {{- end }} {{- with .Values.imagePullSecrets }} imagePullSecrets: {{- toYaml . | nindent 8 }} @@ -36,6 +41,26 @@ spec: volumes: {{- include "openwork-ee.customCa.volume" . | nindent 8 }} {{- end }} + {{- /* + When the Secret is materialized asynchronously (ESO or out-of-band + creation), the Job must not fail on a missing Secret before it exists. + Block in an init container until it appears instead of erroring on the + env secretKeyRef. + */}} + {{- if ne .Values.secret.secretsMode "inline" }} + initContainers: + - name: wait-for-secret + image: "{{ .Values.migrations.kubectlImage.repository }}:{{ .Values.migrations.kubectlImage.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: + - sh + - -c + - | + until kubectl get secret {{ include "openwork-ee.secretNameRaw" . }} -n {{ include "openwork-ee.namespaceRaw" . }} > /dev/null 2>&1; do + echo "waiting for secret {{ include "openwork-ee.secretNameRaw" . }}..." + sleep 3 + done + {{- end }} containers: - name: migrate image: "{{ .Values.denApi.image.repository }}:{{ default .Values.image.tag .Values.denApi.image.tag }}" @@ -52,22 +77,22 @@ spec: - name: DB_MODE value: {{ .Values.config.databaseMode | quote }} - name: DATABASE_URL - {{- if .Values.secret.create }} + {{- if eq .Values.secret.secretsMode "inline" }} value: {{ .Values.secret.values.databaseUrl | quote }} {{- else }} valueFrom: secretKeyRef: name: {{ include "openwork-ee.secretName" . }} - key: {{ .Values.secret.keys.databaseUrl }} + key: {{ .Values.secret.keys.databaseUrl | quote }} {{- end }} - name: DEN_DB_ENCRYPTION_KEY - {{- if .Values.secret.create }} + {{- if eq .Values.secret.secretsMode "inline" }} value: {{ .Values.secret.values.denDbEncryptionKey | quote }} {{- else }} valueFrom: secretKeyRef: name: {{ include "openwork-ee.secretName" . }} - key: {{ .Values.secret.keys.denDbEncryptionKey }} + key: {{ .Values.secret.keys.denDbEncryptionKey | quote }} {{- end }} {{- if .Values.customCa.enabled }} {{- include "openwork-ee.customCa.env" . | nindent 12 }} diff --git a/packaging/helm/openwork-ee/templates/migration-rbac.yaml b/packaging/helm/openwork-ee/templates/migration-rbac.yaml new file mode 100644 index 0000000000..850bbe4820 --- /dev/null +++ b/packaging/helm/openwork-ee/templates/migration-rbac.yaml @@ -0,0 +1,66 @@ +{{- /* + RBAC for the migration Job's wait-for-secret init container: the Job polls + for an asynchronously materialized Secret (ESO / out-of-band creation) + instead of failing on the env secretKeyRef. Rendered only outside inline + mode. In hook mode these must be hooks with an earlier weight than the Job + (-6 precedes the Job's -5) so they exist before it starts. +*/}} +{{- /* Runs the legacy create=false shim before the secretsMode check below. */}} +{{- include "openwork-ee.secretsMode.validate" . }} +{{- if and .Values.migrations.enabled (ne .Values.secret.secretsMode "inline") }} +{{- $asHook := .Values.migrations.hook }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "openwork-ee.fullname" . }}-migrate + namespace: {{ include "openwork-ee.namespace" . }} + labels: + {{- include "openwork-ee.componentLabels" (dict "root" . "component" "migration") | nindent 4 }} + {{- if $asHook }} + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-6" + "helm.sh/hook-delete-policy": before-hook-creation + {{- end }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "openwork-ee.fullname" . }}-migrate + namespace: {{ include "openwork-ee.namespace" . }} + labels: + {{- include "openwork-ee.componentLabels" (dict "root" . "component" "migration") | nindent 4 }} + {{- if $asHook }} + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-6" + "helm.sh/hook-delete-policy": before-hook-creation + {{- end }} +rules: + - apiGroups: [""] + resources: ["secrets"] + resourceNames: [{{ include "openwork-ee.secretName" . }}] + verbs: ["get"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "openwork-ee.fullname" . }}-migrate + namespace: {{ include "openwork-ee.namespace" . }} + labels: + {{- include "openwork-ee.componentLabels" (dict "root" . "component" "migration") | nindent 4 }} + {{- if $asHook }} + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-6" + "helm.sh/hook-delete-policy": before-hook-creation + {{- end }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "openwork-ee.fullname" . }}-migrate +subjects: + - kind: ServiceAccount + name: {{ include "openwork-ee.fullname" . }}-migrate + namespace: {{ include "openwork-ee.namespace" . }} +{{- end }} diff --git a/packaging/helm/openwork-ee/templates/namespace.yaml b/packaging/helm/openwork-ee/templates/namespace.yaml new file mode 100644 index 0000000000..35c0822310 --- /dev/null +++ b/packaging/helm/openwork-ee/templates/namespace.yaml @@ -0,0 +1,46 @@ +{{- /* + lookup returns empty under helm template / ArgoCD repo-server rendering, so + GitOps flows still get the Namespace manifest; a live helm install/upgrade + skips it when the namespace already exists, avoiding AlreadyExists failures + against pre-provisioned namespaces. + + When migrations run as pre-install/pre-upgrade hooks, the hook chain + (ExternalSecret, migration RBAC, migration Job) is applied before normal + manifests — so a first-time install into a fresh namespace would create + namespaced hook resources before the Namespace exists. Render the Namespace + as the earliest hook (weight -11, preceding the ExternalSecret's -10) in + that case so the whole install proceeds without pre-provisioning. A hook + Namespace is cluster-scoped and immediately ready, so hook completion is + not delayed. helm uninstall does not remove hook resources, so the + Namespace outlives the release (intentional — deleting it would cascade to + any externally-created resources in it). +*/ -}} +{{- $namespaceName := include "openwork-ee.namespace" . | trimAll "\"" -}} +{{- /* + Gate the cluster-scoped lookup behind createNamespace so restricted-RBAC + helm install/upgrade (no get on cluster Namespaces) still renders when the + feature is off. +*/ -}} +{{- $existing := dict -}} +{{- if .Values.createNamespace -}} +{{- $existing = lookup "v1" "Namespace" "" $namespaceName -}} +{{- end -}} +{{- if and .Values.createNamespace (not $existing) }} +apiVersion: v1 +kind: Namespace +metadata: + name: {{ include "openwork-ee.namespace" . }} + labels: + {{- include "openwork-ee.labels" . | nindent 4 }} + {{- /* + No hook-delete-policy here: before-hook-creation would delete and recreate + the Namespace on any hook re-run, cascade-deleting everything inside it. + Hook + weight alone give the ordering; the lookup guard above already + suppresses re-creation on live installs. + */}} + {{- if .Values.migrations.hook }} + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-11" + {{- end }} +{{- end }} diff --git a/packaging/helm/openwork-ee/templates/secret.yaml b/packaging/helm/openwork-ee/templates/secret.yaml index aba5d68cff..f91af8b192 100644 --- a/packaging/helm/openwork-ee/templates/secret.yaml +++ b/packaging/helm/openwork-ee/templates/secret.yaml @@ -1,4 +1,5 @@ -{{- if .Values.secret.create }} +{{- include "openwork-ee.secretsMode.validate" . }} +{{- if eq .Values.secret.secretsMode "inline" }} apiVersion: v1 kind: Secret metadata: diff --git a/packaging/helm/openwork-ee/tests/external-secrets.sh b/packaging/helm/openwork-ee/tests/external-secrets.sh new file mode 100755 index 0000000000..ed740ba970 --- /dev/null +++ b/packaging/helm/openwork-ee/tests/external-secrets.sh @@ -0,0 +1,397 @@ +#!/usr/bin/env bash +set -euo pipefail + +chart_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +tmp_dir="$(mktemp -d)" +trap 'rm -rf "$tmp_dir"' EXIT + +assert_count() { + local file="$1" + local needle="$2" + local expected="$3" + local count + count="$(grep -F -c -- "$needle" "$file" || true)" + if [[ "$count" != "$expected" ]]; then + printf 'Expected %s occurrences of %s, found %s\n' "$expected" "$needle" "$count" >&2 + return 1 + fi +} + +assert_contains() { + local file="$1" + local needle="$2" + if ! grep -F -q -- "$needle" "$file"; then + printf 'Expected rendered chart to contain %s\n' "$needle" >&2 + return 1 + fi +} + +assert_not_contains() { + local file="$1" + local needle="$2" + if grep -F -q -- "$needle" "$file"; then + printf 'Expected rendered chart not to contain %s\n' "$needle" >&2 + return 1 + fi +} + +assert_failure() { + local values_file="$1" + local expected="$2" + local output_file="$tmp_dir/failure-output.yaml" + local error_file="$tmp_dir/failure-error.txt" + + if helm template openwork-ee "$chart_dir" -f "$values_file" > "$output_file" 2> "$error_file"; then + printf 'Expected helm template to fail for %s\n' "$values_file" >&2 + return 1 + fi + assert_contains "$error_file" "$expected" +} + +# Default render: no ExternalSecret, Secret still rendered from inline values. +default_rendered="$tmp_dir/default.yaml" +helm template openwork-ee "$chart_dir" > "$default_rendered" +assert_count "$default_rendered" 'kind: ExternalSecret' 0 +assert_count "$default_rendered" 'kind: Secret' 1 + +# externalSecrets mode: ExternalSecret rendered from secret.keys, no v1/Secret. +enabled_values="$tmp_dir/enabled-values.yaml" +cat > "$enabled_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + name: external-secrets + kind: ClusterSecretStore + pathPrefix: "eks/openwork/prod/den" + refreshInterval: 5m +YAML +enabled_rendered="$tmp_dir/enabled.yaml" +helm template openwork-ee "$chart_dir" -f "$enabled_values" > "$enabled_rendered" +assert_count "$enabled_rendered" 'kind: ExternalSecret' 1 +assert_count "$enabled_rendered" 'kind: Secret' 0 +assert_contains "$enabled_rendered" 'apiVersion: external-secrets.io/v1beta1' +assert_contains "$enabled_rendered" 'name: "external-secrets"' +assert_contains "$enabled_rendered" 'kind: ClusterSecretStore' +assert_count "$enabled_rendered" 'refreshInterval: "5m"' 1 +assert_contains "$enabled_rendered" 'creationPolicy: Owner' +assert_contains "$enabled_rendered" 'deletionPolicy: Retain' +assert_count "$enabled_rendered" 'dataFrom:' 0 +# spec.data rendered from secret.keys: one entry per key (20 in values.yaml). +assert_count "$enabled_rendered" 'secretKey:' 20 +assert_contains "$enabled_rendered" 'secretKey: "DATABASE_URL' +assert_contains "$enabled_rendered" 'secretKey: "BETTER_AUTH_SECRET' +assert_contains "$enabled_rendered" 'secretKey: "DEN_DB_ENCRYPTION_KEY' +assert_contains "$enabled_rendered" 'secretKey: "DEN_INITIAL_ADMIN_BOOTSTRAP_CODE' +# Remote keys resolve to pathPrefix + env key name. +assert_contains "$enabled_rendered" 'key: "eks/openwork/prod/den/DATABASE_URL"' +assert_contains "$enabled_rendered" 'key: "eks/openwork/prod/den/DEN_INITIAL_ADMIN_BOOTSTRAP_CODE"' +# Uniform strategies on every entry. +assert_count "$enabled_rendered" 'conversionStrategy: Default' 20 +assert_count "$enabled_rendered" 'decodingStrategy: None' 20 +assert_count "$enabled_rendered" 'metadataPolicy: None' 20 +# Target Secret keeps the chart secret name so envFrom/secretKeyRef wiring holds. +# 7 name: occurrences: ExternalSecret metadata.name + target.name, envFrom in +assert_count "$enabled_rendered" 'name: "openwork-ee-secret"' 7 +assert_count "$enabled_rendered" 'secretKeyRef:' 2 +# The migration Job waits for the asynchronously materialized Secret instead of +# failing on a missing one, and the ExternalSecret applies before the Job +# (hook weight -10 precedes the Job's -5). +assert_contains "$enabled_rendered" 'name: wait-for-secret' +assert_contains "$enabled_rendered" 'until kubectl get secret openwork-ee-secret' +assert_contains "$enabled_rendered" 'image: "bitnami/kubectl:1.33.4"' +assert_not_contains "$enabled_rendered" 'kubectl:latest' + +# Whitespace is trimmed at render, matching validation: a padded store name, +# prefix, and existingSecret render trimmed rather than failing or embedding +# spaces. +padded_values="$tmp_dir/padded-values.yaml" +cat > "$padded_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + name: " external-secrets " + pathPrefix: " eks/openwork/prod/den " +YAML +padded_rendered="$tmp_dir/padded.yaml" +helm template openwork-ee "$chart_dir" -f "$padded_values" > "$padded_rendered" +assert_contains "$padded_rendered" 'name: "external-secrets"' +assert_contains "$padded_rendered" 'key: "eks/openwork/prod/den/DATABASE_URL"' +assert_not_contains "$padded_rendered" ' external-secrets' +assert_not_contains "$padded_rendered" ' eks/openwork' + +# existingSecret with padding renders the trimmed name everywhere. +padded_existing_values="$tmp_dir/padded-existing-values.yaml" +cat > "$padded_existing_values" <<'YAML' +secret: + secretsMode: existingSecret + create: false + existingSecret: " padded-secret " +YAML +padded_existing_rendered="$tmp_dir/padded-existing.yaml" +helm template openwork-ee "$chart_dir" -f "$padded_existing_values" > "$padded_existing_rendered" +assert_count "$padded_existing_rendered" 'name: padded-secret' 5 +assert_not_contains "$padded_existing_rendered" ' padded-secret' +assert_count "$enabled_rendered" 'helm.sh/hook-weight": "-10"' 1 +assert_count "$enabled_rendered" 'helm.sh/hook-weight": "-6"' 3 +assert_count "$enabled_rendered" 'helm.sh/hook-weight": "-5"' 1 + +# Inline mode renders no wait RBAC/initContainer and keeps the migration hook +# without the ExternalSecret hook annotations. +assert_count "$default_rendered" 'name: wait-for-secret' 0 +assert_count "$default_rendered" 'kind: ServiceAccount' 0 +assert_contains "$default_rendered" 'helm.sh/hook": pre-install,pre-upgrade' +assert_count "$default_rendered" 'helm.sh/hook-weight": "-10"' 0 +# Inline secret values never appear in any rendered manifest. +assert_not_contains "$enabled_rendered" 'CHANGE_ME_32_CHARS_MINIMUM_BETTER_AUTH' +assert_not_contains "$enabled_rendered" 'change-me@mysql' + +# Renamed target keys via secret.keys flow into both secretKey and remote path. +renamed_values="$tmp_dir/renamed-values.yaml" +cat > "$renamed_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false + keys: + databaseUrl: CUSTOM_DB_URL +externalSecrets: + secretStoreRef: + name: store + pathPrefix: "trunk" +YAML +renamed_rendered="$tmp_dir/renamed.yaml" +helm template openwork-ee "$chart_dir" -f "$renamed_values" > "$renamed_rendered" +assert_contains "$renamed_rendered" 'secretKey: "CUSTOM_DB_URL' +assert_contains "$renamed_rendered" 'key: "trunk/CUSTOM_DB_URL"' +assert_not_contains "$renamed_rendered" 'secretKey: "DATABASE_URL' + +# Strategy overrides apply uniformly to every entry. +strategy_values="$tmp_dir/strategy-values.yaml" +cat > "$strategy_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + name: store + pathPrefix: "trunk" + decodingStrategy: Base64 + target: + deletionPolicy: Delete +YAML +strategy_rendered="$tmp_dir/strategy.yaml" +helm template openwork-ee "$chart_dir" -f "$strategy_values" > "$strategy_rendered" +assert_count "$strategy_rendered" 'decodingStrategy: Base64' 20 +assert_contains "$strategy_rendered" 'deletionPolicy: Delete' + +# Capability-aware apiVersion selection: v1 served -> v1 rendered. +v1_rendered="$tmp_dir/v1.yaml" +helm template openwork-ee "$chart_dir" -f "$enabled_values" \ + --api-versions external-secrets.io/v1 > "$v1_rendered" +assert_contains "$v1_rendered" 'apiVersion: external-secrets.io/v1' +assert_not_contains "$v1_rendered" 'apiVersion: external-secrets.io/v1beta1' + +# Validation failures. +bad_mode_values="$tmp_dir/bad-mode-values.yaml" +cat > "$bad_mode_values" <<'YAML' +secret: + secretsMode: vault +YAML +assert_failure "$bad_mode_values" 'secretsMode must be one of inline, existingSecret, externalSecrets' + +existing_no_name_values="$tmp_dir/existing-no-name-values.yaml" +cat > "$existing_no_name_values" <<'YAML' +secret: + secretsMode: existingSecret + create: false +YAML +assert_failure "$existing_no_name_values" 'secret.existingSecret is required when secretsMode=existingSecret' + +inline_with_existing_values="$tmp_dir/inline-with-existing-values.yaml" +cat > "$inline_with_existing_values" <<'YAML' +secret: + existingSecret: manually-managed +YAML +assert_failure "$inline_with_existing_values" 'secret.existingSecret is only allowed when secretsMode=existingSecret' + +eso_with_create_values="$tmp_dir/eso-with-create-values.yaml" +cat > "$eso_with_create_values" <<'YAML' +secret: + secretsMode: externalSecrets +externalSecrets: + secretStoreRef: + name: store + pathPrefix: trunk +YAML +assert_failure "$eso_with_create_values" 'secret.create must be false when secretsMode is not inline' + +no_store_values="$tmp_dir/no-store-values.yaml" +cat > "$no_store_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + pathPrefix: trunk + secretStoreRef: + name: "" +YAML +assert_failure "$no_store_values" 'externalSecrets.secretStoreRef.name is required when secretsMode=externalSecrets' + +blank_store_values="$tmp_dir/blank-store-values.yaml" +cat > "$blank_store_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + name: " " + pathPrefix: trunk +YAML +assert_failure "$blank_store_values" 'externalSecrets.secretStoreRef.name is required when secretsMode=externalSecrets' + +bad_kind_values="$tmp_dir/bad-kind-values.yaml" +cat > "$bad_kind_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + name: store + kind: VaultStore + pathPrefix: trunk +YAML +assert_failure "$bad_kind_values" 'externalSecrets.secretStoreRef.kind must be SecretStore or ClusterSecretStore' + +no_prefix_values="$tmp_dir/no-prefix-values.yaml" +cat > "$no_prefix_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + name: store +YAML +assert_failure "$no_prefix_values" 'externalSecrets.pathPrefix is required when secretsMode=externalSecrets' + +blank_prefix_values="$tmp_dir/blank-prefix-values.yaml" +cat > "$blank_prefix_values" <<'YAML' +secret: + secretsMode: externalSecrets + create: false +externalSecrets: + secretStoreRef: + name: store + pathPrefix: " " +YAML +assert_failure "$blank_prefix_values" 'externalSecrets.pathPrefix is required when secretsMode=externalSecrets' + +blank_existing_values="$tmp_dir/blank-existing-values.yaml" +cat > "$blank_existing_values" <<'YAML' +secret: + secretsMode: existingSecret + create: false + existingSecret: " " +YAML +assert_failure "$blank_existing_values" 'secret.existingSecret is required when secretsMode=existingSecret' + +existing_with_create_values="$tmp_dir/existing-with-create-values.yaml" +cat > "$existing_with_create_values" <<'YAML' +secret: + secretsMode: existingSecret + existingSecret: manually-managed +YAML +assert_failure "$existing_with_create_values" 'secret.create must be false when secretsMode is not inline' + +empty_mode_values="$tmp_dir/empty-mode-values.yaml" +cat > "$empty_mode_values" <<'YAML' +secret: + secretsMode: "" +YAML +assert_failure "$empty_mode_values" 'secretsMode must be one of inline, existingSecret, externalSecrets' + +# Legacy shim: inline mode with create=false and untouched placeholder values +# auto-corrects to existingSecret (no Secret rendered, no inline env values). +legacy_shim_values="$tmp_dir/legacy-shim-values.yaml" +cat > "$legacy_shim_values" <<'YAML' +secret: + create: false +YAML +legacy_shim_rendered="$tmp_dir/legacy-shim.yaml" +helm template openwork-ee "$chart_dir" -f "$legacy_shim_values" > "$legacy_shim_rendered" +assert_count "$legacy_shim_rendered" 'kind: Secret' 0 +assert_count "$legacy_shim_rendered" 'kind: ExternalSecret' 0 +assert_not_contains "$legacy_shim_rendered" 'change-me@mysql' +assert_not_contains "$legacy_shim_rendered" 'CHANGE_ME_32_CHARS_MINIMUM' + +# The shim must also apply when the migration Job is rendered in isolation: +# --show-only skips configmap/secret templates, so the Job template has to run +# the validator (and shim) itself before its secretsMode check. +job_only_rendered="$tmp_dir/job-only.yaml" +helm template openwork-ee "$chart_dir" -f "$legacy_shim_values" \ + --show-only templates/migration-job.yaml > "$job_only_rendered" +assert_contains "$job_only_rendered" 'secretKeyRef:' +assert_not_contains "$job_only_rendered" 'change-me@mysql' +assert_not_contains "$job_only_rendered" 'CHANGE_ME_32_CHARS_MINIMUM' + +# The RBAC template applies the same validation/shim when rendered in +# isolation: with legacy create=false values it renders the wait RBAC +# (existingSecret branch), and an invalid secretsMode fails fast. +rbac_only_rendered="$tmp_dir/rbac-only.yaml" +helm template openwork-ee "$chart_dir" -f "$legacy_shim_values" \ + --show-only templates/migration-rbac.yaml > "$rbac_only_rendered" +# ServiceAccount manifest + RoleBinding subject both carry "kind: ServiceAccount". +assert_count "$rbac_only_rendered" 'kind: ServiceAccount' 2 + +rbac_only_bad="$tmp_dir/rbac-only-bad.yaml" +cat > "$rbac_only_bad" <<'YAML' +secret: + secretsMode: vault +YAML +if helm template openwork-ee "$chart_dir" -f "$rbac_only_bad" \ + --show-only templates/migration-rbac.yaml > /dev/null 2>&1; then + printf 'Expected helm template to fail for invalid secretsMode in migration-rbac.yaml\n' >&2 + exit 1 +fi + +# inline + create=false with REAL-looking values is incoherent: fail, never +# silently reroute someone's credentials. +inline_real_values="$tmp_dir/inline-real-values.yaml" +cat > "$inline_real_values" <<'YAML' +secret: + create: false + values: + databaseUrl: "mysql://app:s3cret@db.internal:3306/openwork_den" + betterAuthSecret: "real-auth-secret-value-here-32chars" + denDbEncryptionKey: "real-encryption-key-value-here-32ch" +YAML +assert_failure "$inline_real_values" 'secret.create must be true when secretsMode=inline' + +# Partially real values must also fail: a real DSN with untouched CHANGE_ME +# placeholders is not a legacy untouched file — rerouting it would silently +# drop the real DSN the operator set. +inline_partial_values="$tmp_dir/inline-partial-values.yaml" +cat > "$inline_partial_values" <<'YAML' +secret: + create: false + values: + databaseUrl: "mysql://app:s3cret@db.internal:3306/openwork_den" +YAML +assert_failure "$inline_partial_values" 'secret.create must be true when secretsMode=inline' + +# existingSecret mode renders no Secret and no ExternalSecret. +existing_rendered="$tmp_dir/existing.yaml" +cat > "$tmp_dir/existing-values.yaml" <<'YAML' +secret: + secretsMode: existingSecret + create: false + existingSecret: manually-managed +YAML +helm template openwork-ee "$chart_dir" -f "$tmp_dir/existing-values.yaml" > "$existing_rendered" +assert_count "$existing_rendered" 'kind: Secret' 0 +assert_count "$existing_rendered" 'kind: ExternalSecret' 0 +assert_count "$existing_rendered" 'name: manually-managed' 5 + +printf 'external-secrets chart checks passed\n' diff --git a/packaging/helm/openwork-ee/tests/namespace.sh b/packaging/helm/openwork-ee/tests/namespace.sh index 8d2133d7aa..72e4884971 100755 --- a/packaging/helm/openwork-ee/tests/namespace.sh +++ b/packaging/helm/openwork-ee/tests/namespace.sh @@ -17,18 +17,68 @@ assert_count() { fi } +assert_contains() { + local file="$1" + local needle="$2" + if ! grep -F -q -- "$needle" "$file"; then + printf 'Expected rendered chart to contain %s\n' "$needle" >&2 + return 1 + fi +} + # Default render: every namespaced resource lands in "openwork". -# 8 resources: Secret, ConfigMap, den-api/den-web Services+Deployments, -# migration Job, env-probe test Job. +# 9 resources: Namespace, Secret, ConfigMap, den-api/den-web +# Services+Deployments, migration Job, env-probe test Job. default_rendered="$tmp_dir/default.yaml" helm template openwork-ee "$chart_dir" > "$default_rendered" +assert_count "$default_rendered" 'kind: Namespace' 1 +assert_contains "$default_rendered" 'name: "openwork"' assert_count "$default_rendered" ' namespace: "openwork"' 8 assert_count "$default_rendered" ' namespace: "kube-system"' 0 -# Full render (ingress + inference enabled): 11 namespaced resources. +# With the migration hook enabled (default), the Namespace renders as the +# earliest hook so first-time installs into a fresh namespace work: hook +# resources (ExternalSecret, migration RBAC/Job) are namespaced and would +# otherwise be created before a normal-manifest Namespace exists. +assert_contains "$default_rendered" 'helm.sh/hook-weight": "-11"' +# The Namespace hook must never carry before-hook-creation: a hook re-run +# would delete and recreate the Namespace, cascade-deleting everything in it. +# (The env-probe test Job legitimately uses before-hook-creation, so scope the +# check to the Namespace document.) +assert_namespace_hook_safe() { + local file="$1" + local in_ns=0 + local line + while IFS= read -r line; do + if [[ "$line" == 'kind: Namespace' ]]; then + in_ns=1 + elif [[ "$line" == '---' ]]; then + in_ns=0 + fi + if [[ "$in_ns" == 1 && "$line" == *'hook-delete-policy'*'before-hook-creation'* ]]; then + printf 'Namespace must not use before-hook-creation (cascade-deletes contents on re-run)\n' >&2 + return 1 + fi + done < "$file" +} +assert_namespace_hook_safe "$default_rendered" + +# With the migration hook disabled, the Namespace is a plain manifest. +nohook_rendered="$tmp_dir/nohook.yaml" +helm template openwork-ee "$chart_dir" --set migrations.hook=false > "$nohook_rendered" +assert_count "$nohook_rendered" 'kind: Namespace' 1 +assert_count "$nohook_rendered" 'helm.sh/hook-weight": "-11"' 0 + +# createNamespace=false skips the Namespace object (out-of-band provisioning). +no_nsdef_rendered="$tmp_dir/no-nsdef.yaml" +helm template openwork-ee "$chart_dir" --set createNamespace=false > "$no_nsdef_rendered" +assert_count "$no_nsdef_rendered" 'kind: Namespace' 0 + +# Full render (ingress + inference enabled): Namespace + 11 namespaced resources. full_rendered="$tmp_dir/full.yaml" helm template openwork-ee "$chart_dir" \ --set ingress.enabled=true --set inference.enabled=true > "$full_rendered" +assert_count "$full_rendered" 'kind: Namespace' 1 assert_count "$full_rendered" ' namespace: "openwork"' 11 # Explicit override wins on every resource. @@ -42,6 +92,12 @@ fallback_rendered="$tmp_dir/fallback.yaml" helm template openwork-ee "$chart_dir" --namespace rel-ns --set namespace= > "$fallback_rendered" assert_count "$fallback_rendered" ' namespace: "rel-ns"' 8 +# The Namespace object name follows the namespace value. +nsdef_override_rendered="$tmp_dir/nsdef-override.yaml" +helm template openwork-ee "$chart_dir" --set namespace=platform > "$nsdef_override_rendered" +assert_count "$nsdef_override_rendered" 'kind: Namespace' 1 +assert_contains "$nsdef_override_rendered" 'name: "platform"' + # Numeric and YAML-keyword overrides stay quoted strings: --set types these as # number/bool, and metadata.namespace must render as a quoted string. numeric_rendered="$tmp_dir/numeric.yaml" diff --git a/packaging/helm/openwork-ee/values.yaml b/packaging/helm/openwork-ee/values.yaml index 8ca0a7e519..95f6c4c997 100644 --- a/packaging/helm/openwork-ee/values.yaml +++ b/packaging/helm/openwork-ee/values.yaml @@ -8,6 +8,12 @@ fullnameOverride: "" # `helm install`, keep this aligned with `--namespace` / the release namespace. namespace: openwork +# Render a Namespace object for the namespace above, so GitOps flows (ArgoCD +# runs helm template and does not create namespaces unless CreateNamespace=true) +# manage it as a resource from git. Set false only when the platform team +# provisions namespaces out of band. +createNamespace: true + image: tag: latest pullPolicy: IfNotPresent @@ -164,7 +170,26 @@ redis: allowInsecureInternal: false secret: + # How this deployment manages secrets. One of: + # inline: the chart renders a v1/Secret from secret.values. + # Local evaluation only — the values live wherever this + # values file lives, so never commit real credentials. + # existingSecret: workloads consume a pre-created Secret named by + # secret.existingSecret; the chart renders nothing. + # externalSecrets: the chart renders an ExternalSecret (ESO) that + # materializes the workload Secret from an external provider + # — the GitOps/ArgoCD-safe path; git holds only store + # references and remote key paths. + secretsMode: inline + # Must be true when secretsMode: inline and false in every other mode; the + # render fails otherwise. Exception: legacy values files upgraded from + # before secretsMode that set create: false with the untouched placeholder + # values below are auto-corrected to existingSecret mode (using the + # chart-created secret name when existingSecret is blank) — real values with + # create: false still fail the render. create: true + # REQUIRED when secretsMode: existingSecret — must be populated, not blank + # or empty (the render fails otherwise). Must stay empty in other modes. existingSecret: "" keys: databaseUrl: DATABASE_URL @@ -187,6 +212,10 @@ secret: githubConnectorAppWebhookSecret: GITHUB_CONNECTOR_APP_WEBHOOK_SECRET connectLinkPrivateKey: DEN_CONNECT_LINK_PRIVATE_KEY initialAdminBootstrapCode: DEN_INITIAL_ADMIN_BOOTSTRAP_CODE + # Used ONLY when secretsMode: inline; ignored otherwise. REQUIRED then: + # databaseUrl, betterAuthSecret, denDbEncryptionKey (the workloads cannot + # start without them). Never commit real credentials here — use + # externalSecrets mode for anything beyond local evaluation. values: databaseUrl: mysql://openwork:change-me@mysql:3306/openwork_den databaseRedisUrl: "" @@ -209,6 +238,44 @@ secret: connectLinkPrivateKey: "" initialAdminBootstrapCode: "" +# External Secrets Operator (ESO) configuration. Active only when +# secretsMode: externalSecrets; the chart renders an ExternalSecret that +# materializes the workload Secret from the provider. Requires ESO and a +# SecretStore/ClusterSecretStore on the destination cluster. +externalSecrets: + # ATTENTION: every secret.keys.* value the workloads consume must exist as + # a JSON property in your provider under pathPrefix — the chart pulls each + # key by name from / and cannot invent missing ones. + # How often ESO re-syncs the Secret from the provider. + refreshInterval: 1h + # Store holding the remote keys. For AWS Secrets Manager the store's + # spec.provider.aws is configured on the (Cluster)SecretStore itself; the + # chart only references it by name. + secretStoreRef: + # REQUIRED when secretsMode: externalSecrets — must be populated, not + # blank or empty (the render fails otherwise), and must match the name of + # your SecretStore/ClusterSecretStore. Without it ESO has no provider to + # read from. + name: "external-secrets" + kind: ClusterSecretStore + target: + # Owner deletes the Secret with the ExternalSecret; Merge keeps it. + creationPolicy: Owner + # Retain keeps the Secret if the ExternalSecret is deleted. + deletionPolicy: Retain + # Remote path trunk: every secret.keys.* value (DATABASE_URL, + # BETTER_AUTH_SECRET, ...) is pulled from /. The + # chart renders spec.data from secret.keys, so the key list here can never + # drift from what the workloads consume. + # REQUIRED when secretsMode: externalSecrets (render fails if blank). + # Example (AWS Secrets Manager): pathPrefix: "eks/openwork/prod/den" + pathPrefix: "" + # Strategy fields applied uniformly to every pulled key (ESO defaults, + # matching the external-secrets.io/v1 schema). + conversionStrategy: Default + decodingStrategy: None + metadataPolicy: None + installerArtifacts: enabled: false existingClaim: "" @@ -308,6 +375,12 @@ migrations: activeDeadlineSeconds: 600 podAnnotations: {} resources: {} + # Pinned kubectl image for the wait-for-secret init container (non-inline + # secretsModes). Keep this at a specific patch version (or digest); avoid + # floating tags so installs are reproducible. + kubectlImage: + repository: bitnami/kubectl + tag: "1.33.4" command: - node args: