Summary
On a fresh NiCo site, the vault-pki-config job in nico-prereqs creates the Vault Kubernetes-auth role (and its SA binding) for the NiCo API under the legacy name carbide-api, but nico-api authenticates to Vault requesting role nico-api bound to SA nico-api. The names don't match, so nico-api can't log in and CrashLoopBackOffs. A carbide→nico rename straggler.
Environment
- Reproduced on a dev site (2.0.0+ NiCo)
- nico-api: a recent 2.1.x pre-release
- Component: nico-prereqs helm chart (
vault-pki-config job)
Evidence
vault-pki-config job log (writes the role under the OLD name despite logging "for nico-api"):
Creating K8s auth role for nico-api...
Success! Data written to: auth/kubernetes/role/carbide-api
nico-api log:
login to Vault requesting role nico-api → 400 invalid role name "nico-api" → CrashLoopBackOff
A second facet of the same rename gap: even once the role name is fixed, the role is bound to SA carbide-api, so nico-api then hits 403 service account name not authorized until the binding is also updated to SA nico-api.
Root cause
The vault-pki-config job template in nico-prereqs still names both the k8s-auth role and its bound service account carbide-api instead of nico-api.
Impact
Every new 2.0.0+ NiCo site hits this unless it adds a values override. Sites on 2.0.0+ currently work around it with a per-site override; reducing these overrides is desirable since they tend to bite later.
Workaround (in use)
Manually create the correctly-named role bound to the right SA, then restart nico-api:
vault write auth/kubernetes/role/nico-api \
bound_service_account_names=nico-api \
<remaining params duplicated from auth/kubernetes/role/carbide-api>
kubectl rollout restart deploy/nico-api -n <ns>
Requested fix
nico-prereqs vault-pki-config should name the k8s-auth role nico-api and bind it to SA nico-api, so no per-site override is required.
Summary
On a fresh NiCo site, the
vault-pki-configjob in nico-prereqs creates the Vault Kubernetes-auth role (and its SA binding) for the NiCo API under the legacy namecarbide-api, butnico-apiauthenticates to Vault requesting rolenico-apibound to SAnico-api. The names don't match, so nico-api can't log in and CrashLoopBackOffs. A carbide→nico rename straggler.Environment
vault-pki-configjob)Evidence
vault-pki-configjob log (writes the role under the OLD name despite logging "for nico-api"):nico-apilog:A second facet of the same rename gap: even once the role name is fixed, the role is bound to SA
carbide-api, so nico-api then hits403 service account name not authorizeduntil the binding is also updated to SAnico-api.Root cause
The
vault-pki-configjob template in nico-prereqs still names both the k8s-auth role and its bound service accountcarbide-apiinstead ofnico-api.Impact
Every new 2.0.0+ NiCo site hits this unless it adds a values override. Sites on 2.0.0+ currently work around it with a per-site override; reducing these overrides is desirable since they tend to bite later.
Workaround (in use)
Manually create the correctly-named role bound to the right SA, then restart nico-api:
Requested fix
nico-prereqs
vault-pki-configshould name the k8s-auth rolenico-apiand bind it to SAnico-api, so no per-site override is required.