From 33a02f641f6026222ad9f26a1c866d035c724afa Mon Sep 17 00:00:00 2001 From: Smana Date: Mon, 24 Aug 2026 13:45:44 +0200 Subject: [PATCH 1/5] feat(gcp): scoped Crossplane IAM for slice 5, replacing roles/editor Restores the binding removed in #1818 -- which had no consumer then and does now -- with least privilege rather than breadth. BEFORE: roles/editor, project-wide. Thousands of permissions across every service, granted to a principal nothing created. AFTER: one binding, roles/resourcemanager.projectIamAdmin, conditioned so it may grant only an allowlist: api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []) .hasOnly(['roles/dns.admin']) WHY THE CONDITION IS THE POINT projectIamAdmin alone would be a large improvement on editor and STILL a privilege-escalation path: setIamPolicy can grant any role to any principal, including granting Crossplane itself roles/owner. The condition closes that. It is also the honest GCP analogue of the AWS side's `xplane-*` scoping. AWS restricts Crossplane by resource NAME; GCP cannot for project IAM, because the resource IS the project -- so it restricts by grantable ROLE instead. Same intent, different axis, and worth stating because the earlier conclusion in this repo was that GCP simply had no equivalent. TWO THINGS MEASURED, NOT READ 1. GCP IAM conditions run a RESTRICTED CEL dialect. The first attempt used `.all(r, r in [...] || r.startsWith(...))` to allow both predefined roles and xplane-prefixed custom ones. GCP rejected it at apply: Condition expression compilation failed: undeclared reference to '@not_strictly_false' The `.all()` macro is unavailable; `hasOnly()` is the supported form and matches exact strings only. tofu validate passes on the broken version -- only the API knows. 2. That forced a tighter design, which is the better one. Exact names mean dynamically-named custom roles cannot be allowlisted, so roles/iam.roleAdmin is NOT granted at all: GCPWorkloadIdentity's optional customRole.permissions is deferred rather than half-enabled. Slice 5's actual need -- criterion 21, external-dns records and cert-manager's DNS-01 challenge -- is served entirely by roles/dns.admin. The file records what re-enabling customRole would require, and that widening the condition is not it. Both traps from the removed version are carried forward: the NUMBER/ID split in the principal string (reversed, the API accepts it and it never matches), and the missing graph edge to module.gke that fails only a FRESH apply. Verified on the live project: `gcloud projects get-iam-policy` shows exactly one crossplane binding, with the condition attached and the member in the precise principal://.../ns/crossplane-system/sa/crossplane form design criterion 19 requires. No roles/iam.roleAdmin binding exists. --- opentofu/gcp/gke/init/iam.tf | 155 ++++++++++++++++++++++++----------- 1 file changed, 106 insertions(+), 49 deletions(-) diff --git a/opentofu/gcp/gke/init/iam.tf b/opentofu/gcp/gke/init/iam.tf index a29a6edef..027af3328 100644 --- a/opentofu/gcp/gke/init/iam.tf +++ b/opentofu/gcp/gke/init/iam.tf @@ -1,49 +1,106 @@ -# NO project IAM bindings here, deliberately. -# -# This file used to grant roles/editor project-wide to the Workload Identity -# principal -# -# principal://iam.googleapis.com/projects//locations/global -# /workloadIdentityPools/.svc.id.goog -# /subject/ns/crossplane-system/sa/crossplane -# -# as "bootstrap-only breadth", to be narrowed in slice 5 (GCPWorkloadIdentity). -# It was removed instead, for a reason that only became clear once the GCP -# cluster was actually built: CROSSPLANE IS NOT DEPLOYED ON GCP. The cluster's -# Flux tree is crds, flux and namespaces -- no Crossplane, no providers, no -# compositions. Nothing created that ServiceAccount and nothing used the grant. -# -# So it was not "a grant that is wider than what Crossplane needs". It was a -# project-wide editor grant with NO consumer, and it was reachable: namespaces/ -# base is shared between clouds and creates crossplane-system on GCP too, so -# anyone able to create a ServiceAccount named `crossplane` in that existing -# namespace inherited editor on the whole project. The binding is also scoped to -# the project-wide identity pool rather than to this cluster, so every future -# GKE cluster in the project -- including a throwaway one -- would have inherited -# it as well. -# -# Narrowing was rejected as the fix because any role set chosen now would be a -# guess about compositions that do not exist yet. Note also that GCP IAM -# conditions would NOT reproduce the AWS `xplane-*` scoping: resource-level -# conditions are unsupported for most services Crossplane would touch, so the -# AWS parity is not directly available here. -# -# SLICE 5 MUST create its own binding alongside the GCP compositions that define -# what is actually needed, and must NOT restore this one. Two traps that cost -# real time when it existed, worth keeping when it comes back: -# -# 1. `projects/` takes the project NUMBER, `workloadIdentityPools/` takes the -# project ID. Reversed, the API accepts the binding and it simply never -# matches -- a permission error that points nowhere. Derive the number from -# data.google_project.this.number, never hand-copy it. -# 2. The principal string is built from variables, so OpenTofu sees no -# reference to module.gke and schedules the binding in PARALLEL with the -# cluster -- but the pool `.svc.id.goog` does not exist until a -# cluster with workload_pool has been created. Without an explicit -# depends_on = [module.gke] a FRESH apply fails with -# Error 400: Identity Pool does not exist (ogenki-435905.svc.id.goog) -# and does not reproduce on re-apply, because by then the pool exists. -# Measured on the first real deploy, 2026-08-23. -# -# data.google_project.this stays in data.tf: output "project_number" feeds the -# Flux postBuild substitution ConfigMap and is unrelated to any of the above. +# Crossplane's GCP identity, for slice 5 (GCPWorkloadIdentity). +# +# This file previously granted roles/editor project-wide and was REMOVED in +# #1818, because Crossplane was not deployed on GCP at all and the grant had no +# consumer. Slice 5 gives it one, so it comes back — deliberately scoped this +# time, with the two traps that cost real time recorded below. +# +# ── WHAT IT REPLACES ──────────────────────────────────────────────────────── +# +# roles/editor is thousands of permissions across every service. What +# GCPWorkloadIdentity renders is ProjectIAMMember, one per requested role, so the +# capability actually needed is setIamPolicy on the project: +# +# roles/resourcemanager.projectIamAdmin +# +# ── WHY THAT ALONE IS NOT ENOUGH ──────────────────────────────────────────── +# +# projectIamAdmin is a PRIVILEGE-ESCALATION PATH: setIamPolicy can grant any role +# to any principal, including granting Crossplane itself roles/owner. Narrowing +# editor to it would be a large improvement and still leave that open. +# +# The mitigation is an IAM Condition on `modifiedGrantsByRole`, limiting WHICH +# roles this binding may grant. That is GCP's analogue of the AWS side's +# `xplane-*` scoping (platform constitution): AWS restricts Crossplane by +# resource NAME; GCP cannot for project IAM, because the resource IS the project, +# so it restricts by grantable ROLE instead. +# +# Adding a role to the allowlist is therefore a deliberate act, which is the +# point. A workload needing something outside it fails with a permission error +# naming the role, rather than Crossplane quietly having had it all along. +# +locals { + # Predefined roles Crossplane may grant. Keep tight; grow on evidence. + crossplane_grantable_roles = [ + "roles/dns.admin", # external-dns records + cert-manager DNS-01 (criterion 21) + ] + + # TRAP 1, and it is silent: `projects/` takes the project NUMBER while + # `workloadIdentityPools/` takes the project ID. Reversed, the API ACCEPTS the + # binding and it simply never matches — a permission error that points nowhere. + # Derived from data.google_project rather than hand-copied for that reason. + crossplane_principal = join("", [ + "principal://iam.googleapis.com/projects/${data.google_project.this.number}", + "/locations/global/workloadIdentityPools/${var.project_id}.svc.id.goog", + "/subject/ns/crossplane-system/sa/crossplane", + ]) + + # Grants are limited to the allowlist above, and nothing else. + # + # `hasOnly` is the ONLY usable form here. GCP IAM conditions run a restricted + # CEL dialect: the `.all()` macro is rejected at apply time with + # undeclared reference to '@not_strictly_false' + # so an expression mixing exact matches with a `startsWith` prefix cannot be + # written. Measured 2026-08-24, not read in docs. + # + # The consequence is deliberate and recorded below: exact role names only, + # therefore no support for dynamically-named custom roles. + crossplane_grant_condition = join("", [ + "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([", + join(",", [for r in local.crossplane_grantable_roles : "'${r}'"]), + "])", + ]) +} + +# Additive binding, and it must stay that way. +# +# NEVER use google_project_iam_policy or google_project_iam_binding here: both +# are AUTHORITATIVE for the roles they manage and would delete every binding +# they do not know about — other workloads, and break-glass human access. The +# failure is silent until something unrelated loses permission. +resource "google_project_iam_member" "crossplane_iam_admin" { + project = var.project_id + role = "roles/resourcemanager.projectIamAdmin" + member = local.crossplane_principal + + condition { + title = "xplane-scoped-grants-only" + description = "Crossplane may grant only the allowlisted predefined roles. Without this, projectIamAdmin can grant itself roles/owner." + expression = local.crossplane_grant_condition + } + + # TRAP 2, fresh-apply only. The principal string is built from variables, so + # OpenTofu sees no reference to module.gke and schedules this in PARALLEL with + # the cluster — but the pool `.svc.id.goog` does not exist until a + # cluster with workload_pool has been created. Without this a FRESH apply fails + # with `Error 400: Identity Pool does not exist`, and it does NOT reproduce on + # re-apply, because by then the pool exists. Measured 2026-08-23. + depends_on = [module.gke] +} + +# NO roles/iam.roleAdmin BINDING, deliberately. +# +# It would only be needed for GCPWorkloadIdentity's optional +# `customRole.permissions`, and that feature CANNOT be granted safely today: the +# condition above uses `hasOnly`, which matches exact role names, and a custom +# role's name is chosen by the composition at render time. Allowing it would mean +# either dropping the condition — restoring the escalation path this file exists +# to close — or enumerating names that do not exist yet. +# +# So the capability is deferred rather than half-granted. Slice 5's actual need +# (criterion 21: external-dns records and cert-manager's DNS-01 challenge) is +# served entirely by roles/dns.admin. +# +# To enable customRole later: add roles/iam.roleAdmin, and either accept an +# unconditioned projectIamAdmin or pre-create the custom roles in OpenTofu so +# their names can be named in the allowlist. Do NOT simply widen the condition. From f208984216e6029062eaf57c04775607479d155c Mon Sep 17 00:00:00 2001 From: Smana Date: Mon, 24 Aug 2026 13:52:46 +0200 Subject: [PATCH 2/5] fix(gcp): bind Workload Identity to sa/provider-gcp, not sa/crossplane The binding committed in 33a02f64 could never have matched. It named `ns/crossplane-system/sa/crossplane`, but Crossplane core never talks to GCP -- the PROVIDER pod makes the cloud API calls, under its own ServiceAccount. The AWS side has said so all along: opentofu/aws/eks/init/iam.tf:56-57 binds its Pod Identity to crossplane-system/provider-aws, not to crossplane. I found this by checking what AWS actually binds while writing the GCP provider manifests, not from any error -- and there would not have been a useful one. This is TRAP 1 from the same file, in its other form. A wrong subject is ACCEPTED by the API and simply never matches: no validation failure, no Crossplane error at install, just permission denials at first use that point at the workload rather than at the binding. The name is now tied to its source in a comment: it must equal the serviceAccountTemplate in the DeploymentRuntimeConfig that slice 5's provider tree will carry. The two are set in different repositories' worth of context and nothing checks that they agree. Verified on the live project: the stale sa/crossplane binding is destroyed and exactly one remains -- roles/resourcemanager.projectIamAdmin ns/crossplane-system/sa/provider-gcp condition: modifiedGrantsByRole.hasOnly(['roles/dns.admin']) which is the precise principal:// form design criterion 19 requires. --- opentofu/gcp/gke/init/iam.tf | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/opentofu/gcp/gke/init/iam.tf b/opentofu/gcp/gke/init/iam.tf index 027af3328..0e42b14bf 100644 --- a/opentofu/gcp/gke/init/iam.tf +++ b/opentofu/gcp/gke/init/iam.tf @@ -42,7 +42,16 @@ locals { crossplane_principal = join("", [ "principal://iam.googleapis.com/projects/${data.google_project.this.number}", "/locations/global/workloadIdentityPools/${var.project_id}.svc.id.goog", - "/subject/ns/crossplane-system/sa/crossplane", + # sa/provider-gcp, NOT sa/crossplane. The PROVIDER pod makes the cloud API + # calls; Crossplane core never talks to GCP. The name is set by the + # DeploymentRuntimeConfig's serviceAccountTemplate in + # infrastructure/gcp-mycluster-0/crossplane/providers/ and the two MUST agree + # -- the AWS side binds crossplane-system/provider-aws for the same reason + # (opentofu/aws/eks/init/iam.tf). + # + # Getting this wrong fails exactly like TRAP 1: the binding is accepted, it + # simply never matches, and the error points nowhere. + "/subject/ns/crossplane-system/sa/provider-gcp", ]) # Grants are limited to the allowlist above, and nothing else. From 1b952701246b1ccd77a549b3ec1530d5390cabff Mon Sep 17 00:00:00 2001 From: Smana Date: Mon, 24 Aug 2026 14:17:43 +0200 Subject: [PATCH 3/5] docs(gcp): decide how private certificates are served on GCP Records a decision that workstream 11 would otherwise have had to make under pressure, and separates two workstreams that are easy to conflate. THE SPLIT, which was the first thing to get straight: 10 PUBLIC certs -- cert-manager clouddns DNS-01 against Let's Encrypt. Depends on slice 5. 11 PRIVATE certs -- OpenBao's own PKI, the GCP counterpart to what bao.priv.aws.ogenki.io serves. Depends only on workstream 1, so it is NOT blocked by slice 5. Only 11 needs OpenBao running. Both rows now cross-reference the new section instead of leaving the reader to infer which is which. DECISION: two OpenBaos, two roots. Rejected -- a shared OpenBao reached over the tailnet: zero new infrastructure and one trust anchor, but it makes GCP certificate issuance hard-depend on AWS and on the tailnet, in a platform whose stated point is that each cloud stands alone. The design already flags that same coupling as undesirable for the Flux GitHub App secret. Rejected -- two OpenBaos sharing one root: one trust anchor AND independent operation, which looks like the best of both. It is not, operationally. It needs either the root PRIVATE KEY copied into GCP Secret Manager, doubling exposure of the most sensitive material the platform holds, or GCP's intermediate cross-signed at bootstrap -- a manual ceremony on EVERY REBUILD of a platform whose lifecycle is build-validate-destroy. The deciding evidence is from this session rather than from theory. The private domain rename forced a new OpenBao server certificate, and re-issuing it under the existing chain turned out to be impossible: the intermediate that had signed it had no private key stored anywhere, because OpenBao issued it and the key never left OpenBao. A fresh CA was the only way forward. That is precisely the failure option C institutionalises, and it fires at rebuild time -- the worst moment. Two independent roots cost one extra trust anchor and delete the whole class. Consequences carried into workstream 11 rather than left implicit: tailnet clients must trust both roots; GCP needs its own Secret Manager entries for the root token and cert-manager AppRole, mirroring AWS Secrets Manager and following the pattern flux-github-app already set; Cloud KMS auto-unseal is what makes an unattended rebuild possible; and the per-cloud private domains from ADR-0017 make the split clean, since no name resolves to either CA ambiguously. Nothing here reaches application manifests. Workloads request a cert-manager Certificate, which is already cloud-neutral -- the issuer differs per cloud, the developer-facing API does not, which is ADR-0007's split by audience. Also added to the resume plan, since workstream 11 can start independently and whoever picks it up will look there first. Verified: ./scripts/validate-links.sh -> all relative links resolve. --- .../plans/2026-08-23-gcp-foundation-resume.md | 23 ++++++ .../specs/2026-08-18-gcp-support-design.md | 74 ++++++++++++++++++- 2 files changed, 95 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md b/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md index 6f7d27a34..e8187b6de 100644 --- a/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md +++ b/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md @@ -97,6 +97,29 @@ flux reconcile source git flux-system failure mode, `depends_on` gaps, and the `clusters/gcp-mycluster-0/` Flux wiring. Two review agents produced nothing across repeated asks. +### Private certificates need their own OpenBao on GCP + +Decided 2026-08-24, recorded in the design under *Private certificates on GCP*: +each cloud runs its own OpenBao with its **own root CA** (option B), rather than +sharing AWS's over the tailnet or cross-signing a common root. + +Two workstreams, often conflated and worth keeping apart: + +- **10** — public certs, cert-manager clouddns DNS-01. Depends on slice 5. +- **11** — private certs, OpenBao on GCP (MIG + internal LB + Cloud KMS + auto-unseal). Depends only on workstream 1, so it is **not blocked by slice 5** + and can start whenever. + +The deciding argument came from this very rebuild: the domain rename forced a new +server certificate, and re-issuing under the old chain was impossible because the +intermediate that signed it had no private key stored anywhere — OpenBao had +issued it. Cross-cloud PKI ceremony is the first thing to break on a platform +rebuilt daily, and it breaks at rebuild time. + +Carry into workstream 11: tailnet clients must trust both roots; GCP needs its +own Secret Manager entries for the root token and cert-manager AppRole; Cloud KMS +auto-unseal is what makes an unattended rebuild possible. + ### GPU quota — blocks slice 4's last criterion `GPUS_ALL_REGIONS` on project `ogenki-435905` is **0**, so no GPU node can be diff --git a/docs/superpowers/specs/2026-08-18-gcp-support-design.md b/docs/superpowers/specs/2026-08-18-gcp-support-design.md index 8d0dfe4a4..ce10e01f9 100644 --- a/docs/superpowers/specs/2026-08-18-gcp-support-design.md +++ b/docs/superpowers/specs/2026-08-18-gcp-support-design.md @@ -213,8 +213,8 @@ known rather than predicted. | 7 | Extract the Crossplane Configuration packages, OCI-released | 3 | **DONE 2026-08-19 (v0.1.0)** | | 8 | `objectStore` API migration + `App`/`SQLInstance` branching | 5, 7 | unblocked by 7 | | 9 | Object-storage call sites: Harbor (GCS driver), `openbao-snapshot` (GCS + Cloud KMS), CNPG barman (GCS) | 5, 8 | | -| 10 | DNS + PKI: `external-dns` google provider, cert-manager clouddns DNS-01 | 5 | | -| 11 | OpenBao on GCP: MIG + internal LB + Cloud KMS auto-unseal | 1 | | +| 10 | DNS + PKI: `external-dns` google provider, cert-manager clouddns DNS-01 (**public** certs) — see [Private certificates on GCP](#private-certificates-on-gcp) | 5 | | +| 11 | OpenBao on GCP: MIG + internal LB + Cloud KMS auto-unseal (**private** certs) — see [Private certificates on GCP](#private-certificates-on-gcp) | 1 | | | 12 | Gateway/LB: GCP public-LB annotations, drop `aws-load-balancer-controller` | 3 | | | 13 | Storage: `gp3` → `pd-balanced`/hyperdisk, EFS CSI → Filestore CSI | 3 | | | 14 | GPU + LLM platform: GPU `ComputeClass`, GCS Fuse weights, no `runtimeclass-nvidia` | 4, 9 | | @@ -351,6 +351,76 @@ Falsifiable, verified against a live cluster. 11. A written monthly run-rate estimate exists (cluster fee, static pool, Cloud NAT, Cloud DNS, Tailscale instance) stating the zonal-vs-regional choice and its price delta. +### Private certificates on GCP + +*Added 2026-08-24, while scoping slice 5. Workstreams 10 and 11 both touch +certificates and are easy to conflate; they are separable and only one needs +OpenBao.* + +**The split.** Workstream 10 gives **public** certificates — cert-manager's +clouddns DNS-01 solver against Let's Encrypt, for names under +`priv.gcp.ogenki.io`. Workstream 11 gives **private** certificates from OpenBao's +own PKI, the GCP counterpart to what `bao.priv.aws.ogenki.io` serves today. Only +11 requires running OpenBao, and it depends only on workstream 1 (network), so it +is not blocked by slice 5. + +**Why GCP needs its own OpenBao at all.** Per the 2026-08-23 correction above: +OpenBao *the product* is cloud-agnostic, our OpenBao *stacks* are not. +`openbao/management` configures both the `vault` and `aws` providers and reads +its root token, the cert-manager AppRole and the operator password from AWS +Secrets Manager; `openbao/cluster` is ASG, ELB, KMS and Route53 throughout. + +#### Options considered + +**A — Shared OpenBao.** GCP's cert-manager reaches `bao.priv.aws.ogenki.io` over +the tailnet. No new infrastructure, one CA, one trust anchor. Rejected: it makes +GCP certificate issuance hard-depend on AWS *and* on the tailnet, in a platform +whose stated point is that each cloud stands alone. It is the same coupling this +design already flags as undesirable for the Flux GitHub App secret. + +**B — Two OpenBaos, two roots. CHOSEN.** Each cloud runs its own OpenBao with its +own root CA. Fully independent; matches [ADR-0007](../../../website/content/docs/decisions/0007-cloud-abstraction-boundaries.md)'s +rule that platform-facing infrastructure stays cloud-shaped. Costs one extra +trust anchor for tailnet clients. + +**C — Two OpenBaos, one shared root.** Each cloud holds its own intermediate, +both signed by a common root: one trust anchor, independent operation. Rejected +on operational grounds — it requires either copying the root PRIVATE KEY into GCP +Secret Manager, doubling exposure of the most sensitive material the platform +holds, or cross-signing GCP's intermediate at bootstrap, which is a manual +ceremony **on every rebuild** of a platform whose lifecycle is +build-validate-destroy. + +#### Why B, concretely + +The 2026-08-24 rebuild made the argument better than theory could. The private +domain rename forced a new server certificate, and re-issuing it under the +existing chain proved impossible: the intermediate that had signed it had **no +private key stored anywhere**, because OpenBao issued it and the key never left +OpenBao. A fresh CA was the only way forward. + +That is exactly the failure mode option C institutionalises. Cross-cloud PKI +ceremony is the first thing to break on a platform rebuilt this often, and it +breaks at rebuild time, which is the worst moment. Two independent roots cost one +extra trust anchor and remove the entire class. + +#### Consequences to carry into workstream 11 + +- Tailnet clients must trust **both** roots. That is the accepted cost; it is a + one-line addition wherever the AWS root is already distributed. +- GCP needs its own secret store for the root token and the cert-manager AppRole + — **GCP Secret Manager**, mirroring what AWS Secrets Manager does today. The + `flux-github-app` secret already establishes that pattern on GCP. +- **Cloud KMS auto-unseal** is the GCP analogue of the AWS KMS unseal, already + named in the workstream row. It is what makes an unattended rebuild possible. +- The GCP PKI issues for `*.priv.gcp.ogenki.io` only; the AWS one keeps + `*.priv.aws.ogenki.io`. The per-cloud private domains from + [ADR-0017](../../../website/content/docs/decisions/0017-multi-cloud-dns-naming.md) + make that split clean — there is no name a client could resolve to either CA. +- Nothing about this reaches application manifests: workloads request a + cert-manager `Certificate`, which is already cloud-neutral. The issuer differs + per cloud, the developer-facing API does not — ADR-0007's split by audience. + **Slice 4 (autoscaling)** — *results recorded 2026-08-24, measured on gcp-mycluster-0. Four PASS, one partial, one blocked on a GCP quota. Each is annotated below.* From 22eca740dc4834142bd92d425bb5f69b25bd7a97 Mon Sep 17 00:00:00 2001 From: Smana Date: Mon, 24 Aug 2026 14:23:47 +0200 Subject: [PATCH 4/5] =?UTF-8?q?fix(gcp):=20apply=20the=20slice=205=20revie?= =?UTF-8?q?w=20=E2=80=94=20one=20contradiction=20hid=20a=20real=20gap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Seven findings applied. The first was a wording error that turned out to be covering a hole in the plan. THE CONTRADICTION, AND WHAT WAS BEHIND IT The new PKI section said workstream 10 issues PUBLIC certificates "for names under priv.gcp.ogenki.io" -- while the same section, and ADR-0017, say priv.gcp.ogenki.io is the PRIVATE zone and public stays cloud.ogenki.io. A section written to stop workstreams 10 and 11 being conflated opened by conflating them. Correcting the domain exposed the actual problem: WORKSTREAM 10's DNS-01 HAS NOTHING TO SOLVE AGAINST. opentofu/gcp/network/dns.tf creates a PRIVATE Cloud DNS zone and nothing else, and Let's Encrypt must resolve the _acme-challenge TXT record publicly. cloud.ogenki.io is a Route53 zone this repository does not even manage -- it appears only as a data lookup. So "cert-manager clouddns DNS-01" was never a complete plan, and asserting the wrong domain hid that. Three ways out are now recorded, none chosen: solve DNS-01 against Route53 from GCP (works today, reintroduces the cross-cloud dependency option A was rejected for); delegate a public subdomain to a new public Cloud DNS zone; or serve no public certificates from GCP and keep public ingress on AWS, which is coherent while GCP has no public endpoints. Worth settling before workstream 10 starts rather than during it. Workstream 11 is unaffected. THE DEAD REFERENCE iam.tf pointed at "the DeploymentRuntimeConfig's serviceAccountTemplate in infrastructure/gcp-mycluster-0/crossplane/providers/" in the present tense. That directory does not exist, and `provider-gcp` appears nowhere else in the repo -- a reader greps, finds nothing, and cannot tell whether the binding or the path is wrong. Restated as what it is: an OBLIGATION on slice 5, with the warning that nothing checks the two agree. DUPLICATION, the pattern that keeps recurring here Four restatements removed: a verbatim re-telling of TRAP 1 twelve lines below the original; a forward-pointer to a block in the same file; a second copy of the dns.admin rationale; and a re-summary of the 2026-08-23 OpenBao correction that already sits earlier in the same document. The resume plan also re-told the design's whole rationale after naming it as the source -- now reduced to the sequencing facts that are the plan's actual job, with a link for the reasoning. The measured-behaviour comments are untouched: the CEL .all() rejection, the NUMBER/ID split, the provider-SA trap, additive-vs-authoritative. Those are the file's value; the findings were about copies of them. THE REVIEWER'S CLOSING CHALLENGE, ACCEPTED This binding again lands ahead of its consumer -- exactly the situation #1818 removed the old one over. The header no longer glosses that. It states the caveat, why it is acceptable here where roles/editor was not (blast radius is one grantable role, not owner; and the provider cannot authenticate without it, so it must precede the deployment), and instructs removing it again if slice 5 stalls. Verified: tofu validate and tofu fmt clean; ./scripts/validate-links.sh -> all relative links resolve. --- .../plans/2026-08-23-gcp-foundation-resume.md | 29 ++++++------- .../specs/2026-08-18-gcp-support-design.md | 38 ++++++++++++----- opentofu/gcp/gke/init/iam.tf | 41 ++++++++++++------- 3 files changed, 66 insertions(+), 42 deletions(-) diff --git a/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md b/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md index e8187b6de..1c80ed6ff 100644 --- a/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md +++ b/docs/superpowers/plans/2026-08-23-gcp-foundation-resume.md @@ -99,26 +99,21 @@ flux reconcile source git flux-system ### Private certificates need their own OpenBao on GCP -Decided 2026-08-24, recorded in the design under *Private certificates on GCP*: -each cloud runs its own OpenBao with its **own root CA** (option B), rather than -sharing AWS's over the tailnet or cross-signing a common root. +**Decided 2026-08-24. Rationale and options live in the design under *Private +certificates on GCP*** — not repeated here, because this file is the one edited +on resume and the two would drift. -Two workstreams, often conflated and worth keeping apart: +What matters for sequencing: -- **10** — public certs, cert-manager clouddns DNS-01. Depends on slice 5. -- **11** — private certs, OpenBao on GCP (MIG + internal LB + Cloud KMS - auto-unseal). Depends only on workstream 1, so it is **not blocked by slice 5** +- **Workstream 11** (private certs, OpenBao on GCP: MIG + internal LB + Cloud KMS + auto-unseal) depends only on workstream 1, so it is **not blocked by slice 5** and can start whenever. - -The deciding argument came from this very rebuild: the domain rename forced a new -server certificate, and re-issuing under the old chain was impossible because the -intermediate that signed it had no private key stored anywhere — OpenBao had -issued it. Cross-cloud PKI ceremony is the first thing to break on a platform -rebuilt daily, and it breaks at rebuild time. - -Carry into workstream 11: tailnet clients must trust both roots; GCP needs its -own Secret Manager entries for the root token and cert-manager AppRole; Cloud KMS -auto-unseal is what makes an unattended rebuild possible. +- **Workstream 10** (external-dns + public certs) is blocked on an open question + the design records: GCP has only a *private* Cloud DNS zone, so DNS-01 has + nothing publicly resolvable to solve against. Settle that before starting it. +- The decision is **two OpenBaos, two roots** — so workstream 11 needs its own + GCP Secret Manager entries for the root token and cert-manager AppRole, and + tailnet clients end up trusting both roots. ### GPU quota — blocks slice 4's last criterion diff --git a/docs/superpowers/specs/2026-08-18-gcp-support-design.md b/docs/superpowers/specs/2026-08-18-gcp-support-design.md index ce10e01f9..936105834 100644 --- a/docs/superpowers/specs/2026-08-18-gcp-support-design.md +++ b/docs/superpowers/specs/2026-08-18-gcp-support-design.md @@ -357,18 +357,34 @@ Falsifiable, verified against a live cluster. certificates and are easy to conflate; they are separable and only one needs OpenBao.* -**The split.** Workstream 10 gives **public** certificates — cert-manager's -clouddns DNS-01 solver against Let's Encrypt, for names under -`priv.gcp.ogenki.io`. Workstream 11 gives **private** certificates from OpenBao's -own PKI, the GCP counterpart to what `bao.priv.aws.ogenki.io` serves today. Only -11 requires running OpenBao, and it depends only on workstream 1 (network), so it -is not blocked by slice 5. - -**Why GCP needs its own OpenBao at all.** Per the 2026-08-23 correction above: +**The split.** Workstream 11 gives **private** certificates from OpenBao's own +PKI, the GCP counterpart to what `bao.priv.aws.ogenki.io` serves today, for names +under `priv.gcp.ogenki.io`. Workstream 10 covers `external-dns` plus **public** +certificates, which are for `cloud.ogenki.io` — [ADR-0017](../../../website/content/docs/decisions/0017-multi-cloud-dns-naming.md) +keeps the public zone cloud-agnostic and `priv..ogenki.io` private. + +Only 11 requires running OpenBao, and it depends only on workstream 1 (network), +so it is not blocked by slice 5. + +> **Open question, surfaced 2026-08-24 while writing this section: workstream 10's +> DNS-01 has nothing to solve against on GCP.** `opentofu/gcp/network/dns.tf` +> creates a **private** Cloud DNS zone and nothing else, while Let's Encrypt must +> resolve the `_acme-challenge` TXT record **publicly**. `cloud.ogenki.io` is a +> Route53 zone that this repository does not even manage — it appears only as a +> `data` lookup. So "cert-manager clouddns DNS-01" is not yet a complete plan. +> +> Three ways out, none chosen: solve DNS-01 against **Route53** from the GCP +> cluster (works today, but reintroduces exactly the cross-cloud dependency +> option A was rejected for); **delegate** a public subdomain to a new public +> Cloud DNS zone (clean, needs a registrar change and a public zone this repo +> would then own); or serve **no public certificates from GCP at all** and keep +> public ingress on AWS — which is coherent while GCP has no public endpoints. +> +> Worth settling before workstream 10 starts, not during it. Nothing about it +> affects workstream 11, which is self-contained. + +**Why GCP needs its own OpenBao at all.** See the 2026-08-23 correction above: OpenBao *the product* is cloud-agnostic, our OpenBao *stacks* are not. -`openbao/management` configures both the `vault` and `aws` providers and reads -its root token, the cert-manager AppRole and the operator password from AWS -Secrets Manager; `openbao/cluster` is ASG, ELB, KMS and Route53 throughout. #### Options considered diff --git a/opentofu/gcp/gke/init/iam.tf b/opentofu/gcp/gke/init/iam.tf index 0e42b14bf..12c5b003a 100644 --- a/opentofu/gcp/gke/init/iam.tf +++ b/opentofu/gcp/gke/init/iam.tf @@ -2,8 +2,24 @@ # # This file previously granted roles/editor project-wide and was REMOVED in # #1818, because Crossplane was not deployed on GCP at all and the grant had no -# consumer. Slice 5 gives it one, so it comes back — deliberately scoped this -# time, with the two traps that cost real time recorded below. +# consumer. It comes back for slice 5 — deliberately scoped this time, with +# everything that was measured rather than assumed recorded below. +# +# HONEST CAVEAT: the consumer still does not exist. Crossplane is not yet +# deployed on GCP, so this binding again lands ahead of the workload it is for, +# which is the situation #1818 removed the old one over. Two things make that +# acceptable where roles/editor was not: +# +# - The blast radius is one role. Anyone able to create a ServiceAccount named +# `provider-gcp` in crossplane-system could assume this identity, but all it +# can then do is grant roles/dns.admin -- not grant itself owner, which is +# what editor allowed. +# - It is a prerequisite, not a leftover: the GCP provider cannot authenticate +# without it, so it has to precede the deployment rather than follow it. +# +# If slice 5 stalls, REMOVE THIS AGAIN rather than letting it sit. The reasoning +# that justified deleting the last one applies to a narrow grant too, just more +# slowly. # # ── WHAT IT REPLACES ──────────────────────────────────────────────────────── # @@ -43,14 +59,14 @@ locals { "principal://iam.googleapis.com/projects/${data.google_project.this.number}", "/locations/global/workloadIdentityPools/${var.project_id}.svc.id.goog", # sa/provider-gcp, NOT sa/crossplane. The PROVIDER pod makes the cloud API - # calls; Crossplane core never talks to GCP. The name is set by the - # DeploymentRuntimeConfig's serviceAccountTemplate in - # infrastructure/gcp-mycluster-0/crossplane/providers/ and the two MUST agree - # -- the AWS side binds crossplane-system/provider-aws for the same reason - # (opentofu/aws/eks/init/iam.tf). + # calls; Crossplane core never talks to GCP -- the AWS side binds + # crossplane-system/provider-aws for the same reason + # (opentofu/aws/eks/init/iam.tf:56-57). # - # Getting this wrong fails exactly like TRAP 1: the binding is accepted, it - # simply never matches, and the error points nowhere. + # OBLIGATION ON SLICE 5, not a description of something that exists: when the + # GCP provider tree is written, its DeploymentRuntimeConfig MUST name this + # ServiceAccount `provider-gcp`. Nothing checks that the two agree, and a + # mismatch fails in the same silent way as TRAP 1. "/subject/ns/crossplane-system/sa/provider-gcp", ]) @@ -62,8 +78,6 @@ locals { # so an expression mixing exact matches with a `startsWith` prefix cannot be # written. Measured 2026-08-24, not read in docs. # - # The consequence is deliberate and recorded below: exact role names only, - # therefore no support for dynamically-named custom roles. crossplane_grant_condition = join("", [ "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([", join(",", [for r in local.crossplane_grantable_roles : "'${r}'"]), @@ -106,9 +120,8 @@ resource "google_project_iam_member" "crossplane_iam_admin" { # either dropping the condition — restoring the escalation path this file exists # to close — or enumerating names that do not exist yet. # -# So the capability is deferred rather than half-granted. Slice 5's actual need -# (criterion 21: external-dns records and cert-manager's DNS-01 challenge) is -# served entirely by roles/dns.admin. +# So the capability is deferred rather than half-granted; the allowlist above +# covers what slice 5 actually needs. # # To enable customRole later: add roles/iam.roleAdmin, and either accept an # unconditioned projectIamAdmin or pre-create the custom roles in OpenTofu so From 09b4a26270601e7481630205c52c7187ed2f1de4 Mon Sep 17 00:00:00 2001 From: Smana Date: Mon, 24 Aug 2026 14:32:43 +0200 Subject: [PATCH 5/5] fix(gcp): replace roles/dns.admin with a scoped custom role MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Security review of the slice 5 Crossplane binding found that roles/dns.admin carries far more than external-dns and cert-manager need: - dns.managedZones.delete contradicts the platform constitution's "no deletion permissions for stateful services (S3, IAM, Route53)". The AWS side honours that rule; this did not. - The dns.responsePolicies.* / dns.policies.* family lets a compromised provider-gcp bind a response policy to the cluster VPC, overriding metadata.google.internal or *.googleapis.com to redirect in-cluster traffic and harvest credentials — invisible to external-dns. Replace it with a pre-created xplane_dns_editor custom role holding only record-set management, transactional changes, and read-only zone lookup. Pre-creating it in OpenTofu also gives it a deterministic name, which is what makes it allowlistable at all: the IAM condition matches exact names via hasOnly, so a composition-rendered role never could be. Also grant a read-only xplane_role_reader. projectIamAdmin's 9 permissions do not include iam.roles.get, and referencing a custom role in setIamPolicy can require reading it — a constraint the predefined-role draft would never have hit. Whether GCP enforces it here is unverified (needs a live provider pod), so it is granted pre-emptively: the downside is asymmetric and reading role definitions confers nothing. Record three gaps the condition cannot close rather than leaving them assumed safe: it gates 1 of the role's 9 permissions (modifiedGrantsByRole is undefined for non-setIamPolicy verbs, making hasOnly vacuously true — latent until the org adopts Principal Access Boundaries); it constrains the role and never the member; and it is scoped to the project-wide workload identity pool, so every future cluster in the project inherits it. The last was documented in #1818 and lost in the rewrite. Confirmed while reviewing: modifiedGrantsByRole covers revocations as well as grants, so the condition also stops Crossplane removing bindings it did not create, including break-glass human access. Update the design's example claim, which asked for roles/dns.admin and would now be refused by the condition, and mark customRole.permissions unusable. --- .../specs/2026-08-18-gcp-support-design.md | 20 ++- opentofu/gcp/gke/init/iam.tf | 170 +++++++++++++++--- 2 files changed, 161 insertions(+), 29 deletions(-) diff --git a/docs/superpowers/specs/2026-08-18-gcp-support-design.md b/docs/superpowers/specs/2026-08-18-gcp-support-design.md index 936105834..f8bd66e3e 100644 --- a/docs/superpowers/specs/2026-08-18-gcp-support-design.md +++ b/docs/superpowers/specs/2026-08-18-gcp-support-design.md @@ -299,14 +299,22 @@ kind: GCPWorkloadIdentity metadata: { name: external-dns, namespace: infrastructure } spec: serviceAccount: { name: external-dns, namespace: infrastructure } - roles: [roles/dns.admin] # predefined roles, named as GCP names them - customRole: # optional — creates + binds a custom role - permissions: [dns.resourceRecordSets.create] + roles: [projects/ogenki-435905/roles/xplane_dns_editor] ``` -Renders one **`ProjectIAMMember`** per role, member = the `principal://` KSA string, plus a -`ProjectIAMCustomRole` when `customRole.permissions` is set. No Google service account, no -annotation. `xplane-*` prefix owned by the composition. +Renders one **`ProjectIAMMember`** per role, member = the `principal://` KSA string. No Google +service account, no annotation. `xplane-*` prefix owned by the composition. + +> **`roles:` may only name roles the OpenTofu allowlist permits**, and as of 2026-08-24 that is the +> single pre-created `xplane_dns_editor` — *not* `roles/dns.admin`, which was the first draft and was +> dropped for carrying zone deletion and response policies (see `opentofu/gcp/gke/init/iam.tf` for +> the full reasoning). A claim naming anything else is refused by the IAM condition, not by the +> composition, so the error surfaces at the provider and names the role. +> +> **`customRole.permissions` is unusable on this platform** and should not appear in claims. The +> condition matches exact role names via `hasOnly`, and a role the composition names at render time +> cannot be allowlisted in advance. Adding a capability means adding a +> `google_project_iam_custom_role` in OpenTofu and referencing it — the same two-step every time. **Two hard constraints, both silent-failure classes:** diff --git a/opentofu/gcp/gke/init/iam.tf b/opentofu/gcp/gke/init/iam.tf index 12c5b003a..82b43bdcf 100644 --- a/opentofu/gcp/gke/init/iam.tf +++ b/opentofu/gcp/gke/init/iam.tf @@ -10,10 +10,10 @@ # which is the situation #1818 removed the old one over. Two things make that # acceptable where roles/editor was not: # -# - The blast radius is one role. Anyone able to create a ServiceAccount named -# `provider-gcp` in crossplane-system could assume this identity, but all it -# can then do is grant roles/dns.admin -- not grant itself owner, which is -# what editor allowed. +# - The blast radius is one narrow role. Anyone able to create a ServiceAccount +# named `provider-gcp` in crossplane-system could assume this identity, but +# all it can then do is grant DNS record-set management — not grant itself +# owner, which is what editor allowed. # - It is a prerequisite, not a leftover: the GCP provider cannot authenticate # without it, so it has to precede the deployment rather than follow it. # @@ -36,19 +36,75 @@ # editor to it would be a large improvement and still leave that open. # # The mitigation is an IAM Condition on `modifiedGrantsByRole`, limiting WHICH -# roles this binding may grant. That is GCP's analogue of the AWS side's +# roles this binding may modify. That is GCP's analogue of the AWS side's # `xplane-*` scoping (platform constitution): AWS restricts Crossplane by # resource NAME; GCP cannot for project IAM, because the resource IS the project, -# so it restricts by grantable ROLE instead. +# so it restricts by role instead. +# +# The attribute covers GRANTS AND REVOCATIONS both — Google documents it as "role +# names from the role bindings that the request modifies", and the reference +# table is headed "Granted/revoked roles". So the condition also stops Crossplane +# REMOVING bindings it did not create, including break-glass human access. +# Verified against the attribute reference 2026-08-24. # # Adding a role to the allowlist is therefore a deliberate act, which is the # point. A workload needing something outside it fails with a permission error # naming the role, rather than Crossplane quietly having had it all along. -# + +# The DNS role Crossplane is permitted to grant. +# +# Pre-created HERE rather than by the composition, and that is what makes it +# grantable at all: the condition below uses `hasOnly`, which matches exact role +# names, so a role whose name the composition invents at render time could never +# be allowlisted. A role created in OpenTofu has a DETERMINISTIC name, so it can. +# +# NOT roles/dns.admin, which was the first draft. That predefined role carries +# `dns.managedZones.delete` and the whole `dns.responsePolicies.*` / +# `dns.policies.*` family, both beyond anything slice 5 needs: +# +# - Zone deletion contradicts the platform constitution outright — "no deletion +# permissions for stateful services (S3, IAM, Route53)". The AWS side honours +# that; granting dns.admin here would not have. +# - Response policies are the sharper one. A compromised provider-gcp could +# bind a response policy to the cluster's VPC overriding +# `metadata.google.internal` or `*.googleapis.com`, redirecting in-cluster +# traffic and harvesting credentials — invisible to external-dns, which only +# ever looks at record sets. +# +# `role_id` may not contain dashes, so the platform's `xplane-` convention is +# spelled `xplane_` here. +resource "google_project_iam_custom_role" "crossplane_dns" { + project = var.project_id + role_id = "xplane_dns_editor" + title = "Crossplane DNS editor" + description = "Record-set management for external-dns and cert-manager DNS-01. Deliberately excludes zone deletion and response policies; see opentofu/gcp/gke/init/iam.tf." + + permissions = [ + # Record sets: the actual job. Delete IS included — external-dns removes + # records when a route goes away, and cert-manager cleans up its + # _acme-challenge TXT records after validation. + "dns.resourceRecordSets.create", + "dns.resourceRecordSets.delete", + "dns.resourceRecordSets.get", + "dns.resourceRecordSets.list", + "dns.resourceRecordSets.update", + + # Changes are how Cloud DNS applies record-set edits transactionally. + "dns.changes.create", + "dns.changes.get", + "dns.changes.list", + + # Read-only on zones. external-dns must discover which zone owns a name; it + # must never create or destroy one. + "dns.managedZones.get", + "dns.managedZones.list", + ] +} + locals { - # Predefined roles Crossplane may grant. Keep tight; grow on evidence. + # Roles Crossplane may grant. Keep tight; grow on evidence. crossplane_grantable_roles = [ - "roles/dns.admin", # external-dns records + cert-manager DNS-01 (criterion 21) + google_project_iam_custom_role.crossplane_dns.name, ] # TRAP 1, and it is silent: `projects/` takes the project NUMBER while @@ -77,7 +133,6 @@ locals { # undeclared reference to '@not_strictly_false' # so an expression mixing exact matches with a `startsWith` prefix cannot be # written. Measured 2026-08-24, not read in docs. - # crossplane_grant_condition = join("", [ "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([", join(",", [for r in local.crossplane_grantable_roles : "'${r}'"]), @@ -98,7 +153,7 @@ resource "google_project_iam_member" "crossplane_iam_admin" { condition { title = "xplane-scoped-grants-only" - description = "Crossplane may grant only the allowlisted predefined roles. Without this, projectIamAdmin can grant itself roles/owner." + description = "Crossplane may modify only the allowlisted role bindings. Without this, projectIamAdmin can grant itself roles/owner." expression = local.crossplane_grant_condition } @@ -111,18 +166,87 @@ resource "google_project_iam_member" "crossplane_iam_admin" { depends_on = [module.gke] } +# Read-only access to role definitions, so the grant above can actually be made. +# +# `gcloud iam roles describe roles/resourcemanager.projectIamAdmin` returns 9 +# permissions and `iam.roles.get` is NOT among them (checked 2026-08-24). +# Referencing a CUSTOM role in a setIamPolicy call can require the caller to read +# that role's definition — a constraint that does not exist for predefined roles, +# and therefore one the previous roles/dns.admin draft would never have hit. +# +# UNVERIFIED whether GCP enforces it on this path: confirming needs a live +# provider-gcp pod attempting the grant, and GCP is torn down. Granted +# pre-emptively because the downside is asymmetric — without it slice 5 fails at +# runtime with an error naming the wrong thing, and with it the identity gains +# only the ability to READ role definitions, which are not secret and confer +# nothing. Unconditioned deliberately: `modifiedGrantsByRole` is undefined for +# read requests, so a condition here would be vacuous anyway (see gap 1 below). +resource "google_project_iam_custom_role" "crossplane_role_reader" { + project = var.project_id + role_id = "xplane_role_reader" + title = "Crossplane role reader" + description = "Read-only on IAM role definitions, so ProjectIAMMember can reference the custom DNS role. Confers no grant capability." + + permissions = [ + "iam.roles.get", + "iam.roles.list", + ] +} + +resource "google_project_iam_member" "crossplane_role_reader" { + project = var.project_id + role = google_project_iam_custom_role.crossplane_role_reader.name + member = local.crossplane_principal + + # Same fresh-apply race as the binding above — see TRAP 2. + depends_on = [module.gke] +} + +# ── WHAT THIS BINDING STILL DOES NOT CLOSE ────────────────────────────────── +# +# Written down because both are inherent to the condition mechanism, neither has +# a fix available in the restricted CEL dialect, and an undocumented gap becomes +# an assumed-safe one. Reviewed 2026-08-24. +# +# 1. The condition gates 1 of the role's 9 permissions. +# `modifiedGrantsByRole` is populated only for setIamPolicy-shaped requests. +# Google, verbatim: "For other types of requests, the attribute is not +# defined." Undefined falls back to the mandated `[]` default, and +# `[].hasOnly(...)` is vacuously TRUE — so projectIamAdmin's other verbs +# (resourcemanager.projects.{create,update,delete,search}PolicyBinding and +# iam.policybindings.{get,list}) are granted UNCONDITIONED. +# Concretely: deletePolicyBinding could remove a Principal Access Boundary +# binding constraining this very workload. Latent today — the project uses no +# PAB policies, and creating one is separately blocked by the missing +# iam.principalaccessboundarypolicies.bind — but it goes live silently the day +# the org adopts PAB. Re-examine this binding then. +# +# 2. The condition constrains the ROLE, never the MEMBER. +# Nothing stops Crossplane granting the allowlisted role to a principal other +# than itself — e.g. any pod identity in the cluster. Two things bound it: +# the org policy `constraints/iam.allowedPolicyMemberDomains` is enforced +# (allowedValues: C01fvyerd), so allUsers/allAuthenticatedUsers and external +# Google accounts are already refused at the perimeter; and the allowlisted +# role is narrow by construction, so the worst in-org outcome is DNS +# record-set management, not project control. Shrinking the role shrinks this. +# +# 3. The binding is scoped to the PROJECT-WIDE workload identity pool +# `.svc.id.goog`, not to this cluster — GKE workload-identity +# subjects carry no cluster dimension, so there is no way to write it +# otherwise. Every future GKE cluster in this project, including a throwaway +# one, therefore inherits it: a `crossplane-system/provider-gcp` ServiceAccount +# in ANY cluster here matches. Unavoidable, so it is recorded rather than +# fixed; it is a further reason to keep the allowlist narrow. + # NO roles/iam.roleAdmin BINDING, deliberately. # -# It would only be needed for GCPWorkloadIdentity's optional -# `customRole.permissions`, and that feature CANNOT be granted safely today: the -# condition above uses `hasOnly`, which matches exact role names, and a custom -# role's name is chosen by the composition at render time. Allowing it would mean -# either dropping the condition — restoring the escalation path this file exists -# to close — or enumerating names that do not exist yet. -# -# So the capability is deferred rather than half-granted; the allowlist above -# covers what slice 5 actually needs. +# GCPWorkloadIdentity's optional `customRole.permissions` renders a role whose +# name the composition chooses at render time, and `hasOnly` cannot allowlist a +# name that does not exist yet. Granting roleAdmin would mean either dropping the +# condition — restoring the escalation path this file exists to close — or +# accepting arbitrary role creation. # -# To enable customRole later: add roles/iam.roleAdmin, and either accept an -# unconditioned projectIamAdmin or pre-create the custom roles in OpenTofu so -# their names can be named in the allowlist. Do NOT simply widen the condition. +# The pre-created role above is the supported alternative and covers slice 5's +# actual need. Extend the SAME pattern for anything further: add a +# google_project_iam_custom_role here, reference it in the allowlist. Do NOT +# widen the condition.