Skip to content

flux-operator Helm release is owned by both OpenTofu and Flux #1775

Description

@Smana

Summary

The flux-operator Helm release is managed by two systems at once. OpenTofu bootstraps it at a pinned version with no values; Flux then adopts the same release name/namespace and floats it from an OCIRepository with a much richer values set. Every tofu apply on eks/configure therefore fights Flux for ownership.

Evidence

Observed on mycluster-0, 2026-08-19:

$ helm list -n flux-system
NAME           REVISION  CHART                              APP VERSION
flux           1         flux-instance-0.55.0               v0.55.0
flux-operator  11        flux-operator-0.58.1+b93eb18c9dfb  v0.58.1

$ kubectl get helmrelease -n flux-system
NAME           READY  STATUS
flux-operator  True   Helm upgrade succeeded for release flux-system/flux-operator.v11
                      with chart flux-operator@0.58.1+b93eb18c9dfb

The two owners:

Owner Source Version Values
OpenTofu opentofu/eks/configure/main.tfhelm_release.flux_operator pinned var.flux_operator_version (0.55.0) none
Flux flux/operator/helmrelease.yaml + flux/sources/ocirepo-flux-operator.yaml floats on semver: ">=0.43.0 <1.0.0" → 0.58.1 web UI, Zitadel OIDC, networkPolicy.create: false

tofu plan against live state:

~ resource "helm_release" "flux_operator" {
    ~ version = "0.58.1+b93eb18c9dfb" -> "0.55.0"
  }

flux_instance is not affected — still revision 1, cleanly OpenTofu-owned.

Impact

Any terramate script run deploy (or any apply on eks/configure that is non-empty for another reason) downgrades the running operator to var.flux_operator_version and strips the Flux-managed values — the Flux web UI loses its OIDC config — until Flux reconciles it back, up to 10 minutes later (interval: 10m).

Low severity, self-healing, but surprising, and it makes eks/configure plans permanently non-empty, which hides real drift.

lifecycle.ignore_changes does NOT work

Worth recording, because it is the obvious first thing to reach for. Both variants make planning fail outright:

lifecycle {
  ignore_changes = [version, values, set]   # also tried: ignore_changes = all
}
Error: Error locating chart

  with helm_release.flux_operator,
  on main.tf line 127:

Unable to locate chart oci://ghcr.io/controlplaneio-fluxcd/charts/flux-operator:
failed to perform "FetchReference" on source:
ghcr.io/controlplaneio-fluxcd/charts/flux-operator:0.58.1_b93eb18c9dfb: not found

Ignoring version makes OpenTofu plan against the state value 0.58.1+b93eb18c9dfb. That +buildmeta suffix is what Flux writes when it resolves the OCI digest — it is not a published tag — and the Helm provider maps +_ when looking one up. The 404 kills the plan before anything else runs.

Options considered

Option Trade-off
Accept the flap Self-healing within 10m, but eks/configure plans stay permanently dirty
Track var.flux_operator_version toward the floating version Shrinks the window; manual upkeep, always lagging
tofu state rm helm_release.flux_operator after bootstrap OpenTofu stops tracking what Flux owns — but re-deploys then hit cannot re-use a name that is still in use again (the bug fixed in #1773) unless paired with upgrade_install = true, which reintroduces the version fight
Pin the OCIRepository to an exact version Kills the intentional auto-update on the operator

No option is obviously correct — it is a design call about who owns the operator after bootstrap.

Context

Surfaced while fixing #1773 (the orphaned-failed-release wedge). Not caused by that PR; that PR's plan is simply the first non-empty one in a while, which is how this became visible.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions