diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml new file mode 100644 index 0000000..5ea662c --- /dev/null +++ b/.github/workflows/test.yml @@ -0,0 +1,63 @@ +name: Test + +on: + pull_request: + branches: [main] + push: + branches: [main] + +permissions: + contents: read + +jobs: + lint: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - uses: azure/setup-helm@v4 + with: + version: v3.16.4 + + - name: Add chart repos + run: | + helm repo add cnpg https://cloudnative-pg.github.io/charts + helm repo add cerbos https://download.cerbos.dev/helm-charts + helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts + helm repo update + + - name: Build dependencies + run: | + helm dependency build . + # helm 3 leaves sub-charts as .tgz; some helm versions / chart + # configurations error with "missing in charts/ directory" until the + # archives are extracted. Unpack so `helm lint` and `helm template` + # see the dirs. + for tgz in charts/*.tgz; do + tar -xzf "$tgz" -C charts/ + done + ls -la charts/ + + - name: Lint + run: helm lint . + + - name: Template (dev profile) + run: helm template plinth . --values values/dev.values.yaml > /tmp/dev.yaml + + - name: Template (defaults) + run: helm template plinth . > /tmp/defaults.yaml + + - name: Smoke — assert key resources are rendered + run: | + set -euo pipefail + for needle in \ + "kind: Cluster" \ + "name: plinth-cerbos-policies" \ + "kind: Deployment" \ + "kind: CustomResourceDefinition" ; do + if ! grep -q "$needle" /tmp/dev.yaml; then + echo "missing: $needle" >&2 + exit 1 + fi + done + echo "all expected resources present" diff --git a/.gitignore b/.gitignore index 07011a7..8c8a212 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,8 @@ -# Helm +# Helm — sub-chart deps are downloaded by `helm dependency build`, not committed *.tgz charts/*.tgz -charts/*/charts/ +charts/*/ +Chart.lock # Local values overrides values.local.yaml diff --git a/.helmignore b/.helmignore new file mode 100644 index 0000000..2c96ce9 --- /dev/null +++ b/.helmignore @@ -0,0 +1,15 @@ +.DS_Store +.git/ +.gitignore +.idea/ +.vscode/ +.github/ +*.swp +*.swo +*.bak +*.tmp +*.tgz +charts/*.tgz +README.md.tmpl +values/staging.values.yaml +values/prod.values.yaml diff --git a/Chart.yaml b/Chart.yaml new file mode 100644 index 0000000..1d95c7b --- /dev/null +++ b/Chart.yaml @@ -0,0 +1,50 @@ +apiVersion: v2 +name: plinth +description: >- + The Plinth substrate — an opinionated Kubernetes baseline that the + Plinth starters target. v0.1.0 ships a walking skeleton: CloudNativePG + for Postgres, Cerbos for authorisation, and the OpenTelemetry Collector. + The full reference architecture (Vault, Authentik, SigNoz, Wazuh, + Falco, Trivy, Argo CD, Backstage, etc.) lands incrementally — see the + roadmap in the README. + +type: application +version: 0.1.0 +appVersion: "0.1.0" + +home: https://plinth.run +sources: + - https://github.com/plinth-dev/platform +maintainers: + - name: plinth-dev + url: https://github.com/plinth-dev + +icon: https://plinth.run/manuscript-favicon.png + +keywords: + - plinth + - internal-tools + - postgres + - authorisation + - opentelemetry + - bank-grade + +# Sub-chart dependencies are pulled from upstream chart repos via +# `helm dependency update`. Pinned to specific minor versions so a +# release is reproducible — bump the pin deliberately, never floatingly. +dependencies: + - name: cloudnative-pg + version: 0.28.0 + repository: https://cloudnative-pg.github.io/charts + + - name: cerbos + version: 0.52.1 + repository: https://download.cerbos.dev/helm-charts + + - name: opentelemetry-collector + version: 0.153.0 + repository: https://open-telemetry.github.io/opentelemetry-helm-charts + +annotations: + category: Platform + licenses: MIT diff --git a/README.md b/README.md index c3cadff..6de8e84 100644 --- a/README.md +++ b/README.md @@ -1,50 +1,105 @@ # Plinth — Substrate -> **Status: not yet released — Phase D begins after Phase B/C.** -> No chart is published to `oci://ghcr.io/plinth-dev/platform` yet; the install command below is the **target** flow. Track progress on the [roadmap](https://github.com/plinth-dev/.github/blob/main/ROADMAP.md). Architecture reference (already complete): [plinth.run/architecture](https://plinth.run/architecture/). +The Helm umbrella chart that bootstraps the Plinth substrate on a Kubernetes cluster. v0.1.0 is a walking skeleton: CloudNativePG for Postgres, Cerbos for authorisation, and the OpenTelemetry Collector. The full reference architecture lands incrementally — see [Roadmap](#roadmap) below. -The Helm umbrella chart and Talos cluster bootstrap that will bring up the entire Plinth reference architecture on a fresh Kubernetes cluster. +## Install (dev profile) ```bash -# Target — Phase D -helm install plinth oci://ghcr.io/plinth-dev/platform --values dev.values.yaml +helm repo add cnpg https://cloudnative-pg.github.io/charts +helm repo add cerbos https://download.cerbos.dev/helm-charts +helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts + +git clone https://github.com/plinth-dev/platform && cd platform +helm dependency build . +helm install plinth . --namespace plinth --create-namespace --values values/dev.values.yaml ``` -## What it will contain (target) +Once the install settles, you'll have: -One umbrella chart, sub-charts per concern. Upstream Helm releases are used where they exist; vendored only when version pinning matters. +- A 1-instance CloudNativePG `Cluster` named `plinth-postgres` exposing Service `plinth-postgres-rw:5432` +- A 1-replica Cerbos PDP at `plinth-cerbos:3592` (HTTP) and `:3593` (gRPC), with a placeholder `items` policy +- A 1-replica OpenTelemetry Collector at `plinth-opentelemetry-collector:4317` (gRPC) and `:4318` (HTTP), exporting traces/metrics/logs to stdout -| Concern | Components | -| --- | --- | -| Identity | Vault (HA Raft), Authentik, Ory Oathkeeper, cert-manager | -| Authorization | Cerbos PDP | -| Data | CloudNativePG, MinIO, NATS JetStream, Redis Sentinel, OpenSearch | -| Observability | SigNoz (ClickHouse-backed), OpenTelemetry Collector | -| Security | Wazuh, Falco, Trivy Operator, Kyverno | -| GitOps + DevX | Argo CD, Argo Rollouts, Backstage | +Pointing the [`starter-api`](https://github.com/plinth-dev/starter-api) at this substrate is one env block: -Optional sub-charts (default off): Temporal, kube-prometheus-stack (LGTM-style), GitLab CE. +```bash +DATABASE_URL=postgresql://plinth@plinth-postgres-rw:5432/plinth +CERBOS_ADDR=plinth-cerbos:3593 +OTEL_EXPORTER_OTLP_ENDPOINT=http://plinth-opentelemetry-collector:4318 +``` + +## What v0.1.0 ships + +| Concern | Component | Sub-chart | Version | +|---|---|---|---| +| Data | CloudNativePG operator + `Cluster` CR | `cnpg/cloudnative-pg` | 0.28.0 | +| Authorisation | Cerbos PDP | `cerbos/cerbos` | 0.52.1 | +| Observability | OpenTelemetry Collector | `open-telemetry/opentelemetry-collector` | 0.153.0 | + +The umbrella adds two Plinth-specific resources: a `Cluster` CR (bootstrapped database + role) and a `plinth-cerbos-policies` ConfigMap that's mounted into the Cerbos pod at `/policies`. ## Profiles -| Profile | Shape | -| --- | --- | -| `dev` | Single node, no HA, all defaults turned on | -| `staging` | 3 nodes, no DR, full feature set | -| `prod` | Full HA, DR site replication, hardened defaults | +| Profile | Status | Shape | +|---|---|---| +| `dev` | shipped | Single node, no HA, all defaults turned on | +| `staging` | stub — see `values/staging.values.yaml` | 3 nodes, no DR, full feature set | +| `prod` | stub — see `values/prod.values.yaml` | Full HA, DR site replication, hardened defaults | -## Layout +`values/staging.values.yaml` and `values/prod.values.yaml` carry comments describing the intended shape; the actual values land in subsequent chart versions. +## Customise + +Plinth-specific knobs live under the top-level `plinth` key in `values.yaml`: + +```yaml +plinth: + name: plinth + postgres: + enabled: true + instances: 1 + storage: + size: 5Gi + database: plinth + owner: plinth + cerbosPoliciesEnabled: true + cerbosPolicies: + items.yaml: | + apiVersion: api.cerbos.dev/v1 + resourcePolicy: + version: default + resource: items + rules: # ... ``` -. -├── Chart.yaml # Umbrella chart with sub-chart deps -├── values.yaml # Defaults -├── values/ # Profile values: dev / staging / prod -├── charts/ # Vendored sub-charts where pinning matters -├── bootstrap/ # Argo app-of-apps + initial Cerbos / Wazuh seed -└── cluster-bootstrap/ # talosctl manifests, Omni link, walkthrough + +Sub-chart values (e.g. `cerbos.replicaCount`, `opentelemetry-collector.config.exporters`) are forwarded directly to upstream charts — see each upstream chart's README for the full schema. + +## Roadmap + +Tracked in the umbrella's GitHub Issues. Planned for subsequent versions: + +| Concern | Components | +|---|---| +| Identity | Vault (HA Raft), Authentik, Ory Oathkeeper, cert-manager | +| Data | MinIO, NATS JetStream, Redis Sentinel, OpenSearch | +| Observability | SigNoz (ClickHouse-backed), kube-prometheus-stack | +| Security | Wazuh, Falco, Trivy Operator, Kyverno | +| GitOps + DevX | Argo CD, Argo Rollouts, Backstage | +| Bootstrap | Talos manifests + Omni link, Argo app-of-apps | +| Release | OCI publish to `oci://ghcr.io/plinth-dev/platform` | + +Optional sub-charts (default off): Temporal, GitLab CE. + +## Develop + +```bash +helm dependency build . +helm lint . +helm template plinth . --values values/dev.values.yaml | less ``` +CI runs the same three steps plus a smoke check that asserts the rendered output contains the expected `kind: Cluster`, `kind: CustomResourceDefinition`, and Plinth-specific ConfigMap. + ## Related - [`plinth.run`](https://plinth.run) — full architecture reference, tutorials, ADRs. diff --git a/templates/NOTES.txt b/templates/NOTES.txt new file mode 100644 index 0000000..6c7cfa3 --- /dev/null +++ b/templates/NOTES.txt @@ -0,0 +1,30 @@ +Plinth substrate v{{ .Chart.Version }} installed as release "{{ .Release.Name }}" in namespace "{{ .Release.Namespace }}". + +What just came up: + +{{- if .Values.plinth.postgres.enabled }} + * CloudNativePG operator + a {{ .Values.plinth.postgres.instances }}-instance cluster + named "{{ include "plinth.fullname" . }}-postgres". The default database is + "{{ .Values.plinth.postgres.database }}" owned by role "{{ .Values.plinth.postgres.owner }}". + Connection details land in two secrets: + kubectl get secret {{ include "plinth.fullname" . }}-postgres-app -n {{ .Release.Namespace }} + kubectl get secret {{ include "plinth.fullname" . }}-postgres-superuser -n {{ .Release.Namespace }} +{{- end }} + + * Cerbos PDP listening on :3592 (HTTP) and :3593 (gRPC). Policies are mounted + from the "plinth-cerbos-policies" ConfigMap; tweak via `plinth.cerbosPolicies` + in values. + + * OpenTelemetry Collector accepting OTLP/HTTP on :4318 and OTLP/gRPC on :4317. + Dev profile exports to logs only — wire the exporter to your tracing + backend via the `opentelemetry-collector.config.exporters` block and the + matching pipeline `exporters: [...]` lists. + +Wire the starter-api environment to these services: + DATABASE_URL=postgresql://{{ .Values.plinth.postgres.owner }}@{{ include "plinth.fullname" . }}-postgres-rw:5432/{{ .Values.plinth.postgres.database }} + CERBOS_ADDR={{ include "plinth.fullname" . }}-cerbos:3593 + OTEL_EXPORTER_OTLP_ENDPOINT=http://{{ include "plinth.fullname" . }}-opentelemetry-collector:4318 + +Roadmap: Vault, Authentik, SigNoz, Wazuh, Falco, Trivy, Argo CD, Backstage, +and the Talos cluster bootstrap land in subsequent versions — +https://github.com/plinth-dev/platform. diff --git a/templates/_helpers.tpl b/templates/_helpers.tpl new file mode 100644 index 0000000..93f453f --- /dev/null +++ b/templates/_helpers.tpl @@ -0,0 +1,34 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "plinth.name" -}} +{{- default "plinth" .Values.plinth.name | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{/* +Fully-qualified resource name: -. +Used so multiple installs of the umbrella into the same namespace don't +collide. Truncated to 63 chars (Kubernetes label limit). +*/}} +{{- define "plinth.fullname" -}} +{{- $name := default .Chart.Name .Values.plinth.name -}} +{{- if contains $name .Release.Name -}} +{{- .Release.Name | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} +{{- end -}} +{{- end -}} + +{{/* +Standard label set applied to every resource we own. +*/}} +{{- define "plinth.labels" -}} +helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +app.kubernetes.io/name: {{ include "plinth.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +app.kubernetes.io/part-of: plinth +{{- end -}} diff --git a/templates/cerbos-policies-configmap.yaml b/templates/cerbos-policies-configmap.yaml new file mode 100644 index 0000000..71317a7 --- /dev/null +++ b/templates/cerbos-policies-configmap.yaml @@ -0,0 +1,45 @@ +{{- if .Values.plinth.cerbosPoliciesEnabled -}} +# ConfigMap of Cerbos policies mounted into the Cerbos pod at /policies. +# `plinth.cerbosPolicies` is a map of `: `; entries +# end up as files in the mount, which the Cerbos config below points at as +# its policy directory. +# +# The default carries an example "items" resource policy matching what the +# starter-api scaffold ships. Override entirely from your install values +# once you've defined your own resources. +apiVersion: v1 +kind: ConfigMap +metadata: + name: plinth-cerbos-policies + labels: + {{- include "plinth.labels" . | nindent 4 }} +data: +{{- if .Values.plinth.cerbosPolicies }} + {{- range $name, $body := .Values.plinth.cerbosPolicies }} + {{ $name }}: | +{{ $body | indent 4 }} + {{- end }} +{{- else }} + items.yaml: | + apiVersion: api.cerbos.dev/v1 + resourcePolicy: + version: default + resource: items + rules: + - actions: ["read", "list"] + effect: EFFECT_ALLOW + roles: + - viewer + - editor + - admin + - actions: ["create", "update", "delete"] + effect: EFFECT_ALLOW + roles: + - editor + - admin + - actions: ["delete"] + effect: EFFECT_ALLOW + roles: + - admin +{{- end }} +{{- end }} diff --git a/templates/postgres-cluster.yaml b/templates/postgres-cluster.yaml new file mode 100644 index 0000000..b7cccdd --- /dev/null +++ b/templates/postgres-cluster.yaml @@ -0,0 +1,43 @@ +{{- if .Values.plinth.postgres.enabled -}} +# CloudNativePG Cluster CR. Created after the operator is up — the operator's +# CRD must exist before this resource applies. Helm install handles ordering +# via wave-style hooks; if you see "no kind matches", run +# `kubectl wait --for condition=Established crd/clusters.postgresql.cnpg.io` +# and re-run. The cnpg sub-chart's pre-install hooks normally make this a +# non-issue; the wait is for fresh clusters where the API server hasn't +# discovered the CRD yet. +apiVersion: postgresql.cnpg.io/v1 +kind: Cluster +metadata: + name: {{ include "plinth.fullname" . }}-postgres + labels: + {{- include "plinth.labels" . | nindent 4 }} +spec: + instances: {{ .Values.plinth.postgres.instances }} + + # Default Postgres image — bump in lockstep with cnpg releases. + imageName: ghcr.io/cloudnative-pg/postgresql:17.2 + + bootstrap: + initdb: + database: {{ .Values.plinth.postgres.database | quote }} + owner: {{ .Values.plinth.postgres.owner | quote }} + + storage: + size: {{ .Values.plinth.postgres.storage.size | quote }} + {{- with .Values.plinth.postgres.storage.storageClass }} + storageClass: {{ . | quote }} + {{- end }} + + # Conservative resource floor for dev. Override via values for staging/prod. + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + cpu: 500m + memory: 512Mi + + # Backups, monitoring, and replication knobs land in subsequent chart + # versions — see the roadmap. +{{- end }} diff --git a/values.yaml b/values.yaml new file mode 100644 index 0000000..fe37c6e --- /dev/null +++ b/values.yaml @@ -0,0 +1,113 @@ +# Default values for the Plinth substrate. +# +# v0.1.0 boots a single-namespace dev profile: CloudNativePG operator + a +# 1-instance Cluster, Cerbos with a placeholder policy, and the OpenTelemetry +# Collector with an OTLP/HTTP receiver. Override per environment via +# `helm install -f values/.values.yaml`. +# +# Sub-chart values are shaped by their upstream schemas (cloudnative-pg, +# cerbos, opentelemetry-collector). Plinth-specific knobs live under the +# top-level `plinth` key. + +plinth: + # Logical name baked into resource names + labels. Override per install + # so multiple Plinth instances can co-exist in the same cluster. + name: plinth + + postgres: + # Toggle for the Cluster CR rendered by templates/postgres-cluster.yaml. + # The cnpg operator itself is always installed via the cloudnative-pg + # sub-chart — disabling here only skips the Cluster CR. + enabled: true + instances: 1 + storage: + size: 5Gi + storageClass: "" + database: plinth + owner: plinth + + # Toggle for the templates/cerbos-policies-configmap.yaml ConfigMap. + cerbosPoliciesEnabled: true + + # Cerbos policies bundled into a ConfigMap mounted by the cerbos sub-chart. + # Each entry is `policies.: `. Empty by default — + # the template ships a sensible "items" placeholder. + cerbosPolicies: {} + + # Backstage / Argo / Vault / Authentik / Wazuh / Falco / Trivy / Kyverno / + # SigNoz are tracked in the roadmap; toggles will appear here in subsequent + # versions. + +# ---------------------------------------------------------------------------- +# Sub-chart values — forwarded to the upstream charts at install time. See +# each chart's README for the full schema. +# ---------------------------------------------------------------------------- + +cloudnative-pg: + replicaCount: 1 + resources: + requests: + cpu: 50m + memory: 100Mi + limits: + cpu: 200m + memory: 200Mi + +cerbos: + replicaCount: 1 + cerbos: + logLevel: INFO + httpPort: 3592 + grpcPort: 3593 + volumes: + - name: plinth-policies + configMap: + name: plinth-cerbos-policies + volumeMounts: + - name: plinth-policies + mountPath: /policies + readOnly: true + +opentelemetry-collector: + mode: deployment + replicaCount: 1 + image: + repository: otel/opentelemetry-collector-contrib + presets: + kubernetesAttributes: + enabled: true + config: + receivers: + otlp: + protocols: + grpc: + endpoint: 0.0.0.0:4317 + http: + endpoint: 0.0.0.0:4318 + processors: + batch: + send_batch_size: 1024 + timeout: 5s + memory_limiter: + check_interval: 1s + limit_percentage: 75 + spike_limit_percentage: 25 + exporters: + # Dev profile dumps to logs by default — swap for OTLP→Tempo / Jaeger / + # SigNoz / etc. via `values/.values.yaml`. + debug: + verbosity: basic + service: + pipelines: + traces: + receivers: [otlp] + processors: [memory_limiter, batch] + exporters: [debug] + metrics: + receivers: [otlp] + processors: [memory_limiter, batch] + exporters: [debug] + logs: + receivers: [otlp] + processors: [memory_limiter, batch] + exporters: [debug] diff --git a/values/dev.values.yaml b/values/dev.values.yaml new file mode 100644 index 0000000..c36dd39 --- /dev/null +++ b/values/dev.values.yaml @@ -0,0 +1,30 @@ +# Dev profile — single node, no HA, all defaults turned on. +# +# Use against kind / minikube / k3d / Docker Desktop. Resource floors are +# intentionally low so the chart fits on a laptop. +# +# helm install plinth . --values values/dev.values.yaml + +plinth: + postgres: + instances: 1 + storage: + size: 5Gi + +cnpg: + resources: + requests: + cpu: 50m + memory: 100Mi + limits: + cpu: 200m + memory: 200Mi + +cerbos: + replicaCount: 1 + +otelCollector: + mode: deployment + replicaCount: 1 + # Dev exporter: dump traces/metrics/logs to stdout. Switch to OTLP for any + # real backend (Tempo, Jaeger, SigNoz, Honeycomb, Datadog). diff --git a/values/prod.values.yaml b/values/prod.values.yaml new file mode 100644 index 0000000..fae8d3d --- /dev/null +++ b/values/prod.values.yaml @@ -0,0 +1,12 @@ +# Prod profile — TODO. v0.1.0 ships dev only; prod lands in a subsequent +# chart version. The intended shape (see plinth.run/architecture): +# +# - Full HA: Postgres 3+ instances with synchronous replication +# - DR: cross-region replication to a standby Plinth substrate +# - Hardened defaults: PSP / Pod Security Admission "restricted", +# NetworkPolicy default-deny, OPA/Kyverno admission gates +# - All observability sub-charts on (SigNoz HA, Wazuh, Falco, Trivy) +# - Vault HA Raft with auto-unseal via cloud KMS +# - Argo CD app-of-apps for GitOps-driven sub-chart management +# +# Track progress: https://github.com/plinth-dev/platform/issues diff --git a/values/staging.values.yaml b/values/staging.values.yaml new file mode 100644 index 0000000..001aa98 --- /dev/null +++ b/values/staging.values.yaml @@ -0,0 +1,10 @@ +# Staging profile — TODO. v0.1.0 ships dev only; staging lands in a +# subsequent chart version. The intended shape (see plinth.run/architecture): +# +# - 3-node Postgres cluster, no DR +# - Cerbos: 2 replicas behind a Service with a PodDisruptionBudget +# - OTel Collector: deployment + DaemonSet (host-level logs) +# - Vault, Authentik, SigNoz toggled on +# - cert-manager + ingress-nginx wired in +# +# Track progress: https://github.com/plinth-dev/platform/issues