Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion bundle/manifests/oadp-operator.clusterserviceversion.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ metadata:
Cloud Provider,Developer Tools,Modernization & Migration,OpenShift Optional,Storage
certified: "false"
containerImage: quay.io/konveyor/oadp-operator:latest
createdAt: "2025-02-28T20:03:54Z"
createdAt: "2026-07-16T15:56:44Z"
description: OADP (OpenShift API for Data Protection) operator sets up and installs
Data Protection Applications on the OpenShift platform.
features.operators.openshift.io/cnf: "false"
Expand Down Expand Up @@ -1169,6 +1169,15 @@ spec:
- patch
- update
- watch
- apiGroups:
- ""
resources:
- resourcequotas
verbs:
- create
- get
- list
- watch
- apiGroups:
- apps
resources:
Expand Down
5,591 changes: 2,944 additions & 2,647 deletions bundle/manifests/oadp.openshift.io_dataprotectionapplications.yaml

Large diffs are not rendered by default.

5,591 changes: 2,944 additions & 2,647 deletions config/crd/bases/oadp.openshift.io_dataprotectionapplications.yaml

Large diffs are not rendered by default.

19 changes: 19 additions & 0 deletions config/quota/resource_quota.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Example ResourceQuota matching operator defaults (create-if-absent).
# The operator creates this object in the DPA namespace when missing; it is not
# shipped as a CSV-owned bundle manifest so OLM upgrades cannot reset admin edits.
# Do not add this file to config/default kustomization (keeps it out of the OLM bundle).
apiVersion: v1
kind: ResourceQuota
metadata:
name: oadp-resource-quota
namespace: openshift-adp
labels:
app.kubernetes.io/name: oadp-operator
app.kubernetes.io/managed-by: oadp-operator
annotations:
oadp.openshift.io/quota-policy: create-if-absent
spec:
hard:
pods: "200"
requests.cpu: "50"
requests.memory: 64Gi
9 changes: 9 additions & 0 deletions config/rbac/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,15 @@ rules:
- patch
- update
- watch
- apiGroups:
- ""
resources:
- resourcequotas
verbs:
- create
- get
- list
- watch
- apiGroups:
- apps
resources:
Expand Down
47 changes: 47 additions & 0 deletions docs/config/resource_quota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# ResourceQuota for the OADP namespace

When a DataProtectionApplication (DPA) is reconciled, the OADP operator ensures a
namespace-scoped `ResourceQuota` named `oadp-resource-quota` exists in the DPA
namespace (typically `openshift-adp`).

## Defaults

| Hard resource | Default |
|---------------|---------|
| `pods` | `200` |
| `requests.cpu` | `50` |
| `requests.memory` | `64Gi` |

These values are a mid-size cluster ceiling (operator, Velero, node-agent DaemonSet,
optional helpers, and some concurrent backup/restore pods). The default quota does
**not** set `limits.cpu` or `limits.memory` because default Velero and node-agent
pods specify requests only; tracking limits in the quota can cause admission to
reject those pods.

## Create-if-absent

- If the quota is **missing**, the operator creates it with the defaults above.
- If the quota **exists**, the operator leaves it unchanged (admin edits survive
reconcile and upgrades).
- If an admin **deletes** the quota, the next DPA reconcile recreates the defaults.

## Inspect and adjust

```bash
oc describe resourcequota oadp-resource-quota -n openshift-adp

oc edit resourcequota oadp-resource-quota -n openshift-adp

oc patch resourcequota oadp-resource-quota -n openshift-adp --type merge \
-p '{"spec":{"hard":{"pods":"400","requests.cpu":"100","requests.memory":"128Gi"}}}'
```

Node-agent is a DaemonSet: large clusters may need higher `pods` / `requests.cpu` /
`requests.memory`. If pods are Pending because of quota, raise the hard limits.

Admins who set container limits on all workloads may optionally add `limits.cpu` /
`limits.memory` to the quota via `oc edit`.

## Design

See [docs/design/2026-07-16-resourcequota-design.md](../design/2026-07-16-resourcequota-design.md).
168 changes: 168 additions & 0 deletions docs/design/2026-07-16-resourcequota-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Design proposal: ResourceQuota for the OADP namespace

## Abstract

Ship a default namespace-scoped `ResourceQuota` for the OADP install namespace (typically `openshift-adp`) so compliance checks that require a ResourceQuota pass on install, while allowing cluster admins to adjust hard limits without the operator overwriting their changes on reconcile or upgrade.

## Background

Compliance scans commonly flag pods running in namespaces without a `ResourceQuota` applied. OADP installs into a suggested namespace (`openshift-adp` via `operatorframework.io/suggested-namespace`) and also supports other namespaces (for example ACM’s `open-cluster-management-backup`).

Today OADP sets per-container resource requests/limits (operator Deployment, Velero/node-agent defaults, DPA `resourceAllocations`) but does **not** create a namespace-level `ResourceQuota`. There is no existing ResourceQuota manifest in `config/` or the OLM bundle.

A fixed quota is sensitive to cluster size because node-agent is a DaemonSet: pod count and aggregate CPU/memory scale with node count. Defaults must therefore be a generous ceiling, not a tight right-size.

## Goals

- Clear the compliance finding: the OADP namespace has a `ResourceQuota` applied after install / when a DPA is reconciled.
- Ship product defaults (not docs-only).
- Allow admins to adjust the quota (`oc edit` / `oc patch`) with changes preserved across operator reconciles and upgrades (create-if-absent).
- Apply in whatever namespace the DPA lives in (not hardcoded only to `openshift-adp`).

## Non Goals

- `LimitRange` (separate finding / follow-up if required).
- `ClusterResourceQuota`.
- DPA API fields to configure quota (admins edit the ResourceQuota object directly).
- Automatic quota scaling based on node count.
- Per-cloud or per-profile default matrices.
- Deleting or resetting admin-modified quotas.

## Decision summary

| Choice | Decision |
|--------|----------|
| Delivery | Ship defaults with the product; operator ensures object exists |
| Admin adjustability | Create-if-absent; never update `.spec` if the named quota exists |
| Default sizing | Generous mid-size ceiling (node-agent + Velero + operator + helpers + some backup/restore pods) |
| Hard resources | `pods`, `requests.cpu`, `requests.memory` only — **no** `limits.*` (see below) |
| Configuration surface | Direct edit of `ResourceQuota`; no new DPA fields |
| Object name | `oadp-resource-quota` |

## High-level design

### Shipped object

A single namespaced ResourceQuota:

```yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: oadp-resource-quota
namespace: openshift-adp # or DPA namespace
labels:
app.kubernetes.io/name: oadp-operator
app.kubernetes.io/managed-by: oadp-operator
annotations:
oadp.openshift.io/quota-policy: create-if-absent
spec:
hard:
pods: "200"
requests.cpu: "50"
requests.memory: 64Gi
```

Defaults are a **ceiling** for a typical mid-size cluster. Large clusters or heavy concurrent data-mover workloads may need higher hard limits; small clusters may lower them.

Canonical default values live in operator code (or a single shared constant/source used by the operator). A matching YAML under `config/` documents the same content for review and examples.

### Comparison to established OADP pod resources

Default per-container resources today:

| Component | Requests | Limits |
|-----------|----------|--------|
| Velero | 500m / 128Mi | *none* |
| Node-agent (per pod) | 500m / 128Mi | *none* |
| Operator manager | 500m / 128Mi | 1 / 512Mi |
| VM file restore / kubevirt datamover | 10m / 64Mi | 500m / 128Mi |
| CLI download | 50m / 32Mi | 100m / 64Mi |

**Quota vs defaults:** The namespace hard limits are aggregate ceilings, not “barely above one Velero pod.”

- Steady-state (operator + Velero, no node-agent): ~1 CPU / ~256Mi requests → well under `requests.cpu: 50` / `requests.memory: 64Gi`.
- With node-agent (DaemonSet): each node adds 500m / 128Mi requests. Rough headroom before hitting the ceiling is on the order of ~90+ node-agent pods for `requests.cpu: 50`, or ~190+ nodes for `pods: 200` (plus operator/Velero/helpers).

**Why `limits.*` are omitted from the default quota:** Default Velero and node-agent pods set **requests only** (no limits). If a ResourceQuota tracks `limits.cpu` / `limits.memory`, admission can reject pods that omit limits (unless a LimitRange supplies defaults). Shipping `limits.*` would risk breaking the default Velero/node-agent stack. Admins who set container limits everywhere may add `limits.*` to the quota themselves via `oc edit`.

### Lifecycle (create-if-absent)

1. On DPA reconcile, look up `ResourceQuota/oadp-resource-quota` in the DPA namespace.
2. If **missing** → create with shipped defaults.
3. If **present** → do nothing (no patch of `.spec`).
4. Quota is **not** a DPA-owned child for deletion purposes; namespace deletion removes it. DPA teardown does not delete the quota.
5. If an admin **deletes** the quota, the next DPA reconcile recreates defaults.

### OLM / bundle interaction

Pure CSV-owned bundle objects are reconciled by OLM and can reset admin edits on upgrade. To honor create-if-absent:

- The **operator** is authoritative: it creates `oadp-resource-quota` if absent and never updates `.spec` when present.
- Keep a matching example under `config/` (and optional docs sample) for review; do **not** add the ResourceQuota as a CSV-owned bundle manifest, so OLM cannot clobber admin edits on upgrade.
- Practical day-1 timing: the quota appears when a DPA is reconciled (normal install path creates a DPA shortly after operator install). That is sufficient to clear the compliance finding for pods that run because of the DPA.

### RBAC

Extend manager permissions (kubebuilder markers → `make manifests` / bundle):

```text
//+kubebuilder:rbac:groups="",resources=resourcequotas,verbs=get;list;watch;create
```

No `update`, `patch`, or `delete` on `resourcequotas`, so the operator cannot overwrite or remove admin edits even if a bug attempts Update.

### Admin workflow

```bash
# Inspect
oc describe resourcequota oadp-resource-quota -n openshift-adp

# Adjust (survives reconcile/upgrade)
oc edit resourcequota oadp-resource-quota -n openshift-adp
# or
oc patch resourcequota oadp-resource-quota -n openshift-adp --type merge -p '{"spec":{"hard":{"pods":"400"}}}'
```

Document that node-agent is a DaemonSet and large clusters may need higher `pods` / `requests.cpu` / `requests.memory`. If pods are Pending due to quota, raise hard limits. Note that default Velero/node-agent have no container limits, so the shipped quota intentionally omits `limits.*`.

## Edge cases

| Case | Behavior |
|------|----------|
| Admin deletes the quota | Next DPA reconcile recreates defaults |
| Admin creates an additional ResourceQuota | Both apply; effective limit is the intersection (most restrictive). Operator only manages `oadp-resource-quota` |
| ACM / custom install namespace | Same ensure logic in the DPA’s namespace |
| Multiple DPAs in one namespace | Idempotent ensure; still one `oadp-resource-quota` |
| Quota too tight → pods Pending | Docs/support: raise hard limits; no auto-scale |
| No DPA yet | Quota is not ensured until a DPA is reconciled (expected shortly after install) |

## Testing

- **Unit:** missing → created with defaults; existing custom `.spec` → unchanged (operator does not patch).
- **E2E (light/optional):** after deploy + DPA, `ResourceQuota/oadp-resource-quota` exists in the test namespace.

## Documentation

Add a short admin note under `docs/config/` covering defaults, inspect/adjust commands, recreate-on-delete behavior, and node-agent / large-cluster sizing guidance.

## Alternatives considered

1. **Bundle-only OLM-owned ResourceQuota** — Simple, but upgrades reset admin edits; rejected for adjustability requirement.
2. **Docs/sample only** — No upgrade conflict, but does not ship an applied quota; rejected.
3. **Second “override” ResourceQuota** — Cannot loosen a tight default (intersection); rejected as the primary adjust mechanism.
4. **DPA API for quota** — Extra API surface; admins can already edit ResourceQuota; deferred/out of scope.
5. **Include `limits.cpu` / `limits.memory` in default hard list** — Rejected for v1 because default Velero/node-agent pods lack container limits and would risk admission failure; admins can add these fields manually if their workloads all set limits.

## Implementation sketch (for planning)

1. Add default constants + `ensureResourceQuota` helper (get → create if not found).
2. Call ensure from DPA reconcile in the DPA namespace.
3. Add kubebuilder RBAC; regenerate manifests/bundle.
4. Add `config/` example YAML matching defaults (not CSV-owned in the bundle).
5. Unit tests for create-if-absent.
6. Admin docs under `docs/config/`.

## Open questions

None for v1 of this design. Default hard-limit numbers may be tuned during implementation review if QE/support have preferred ceilings.
3 changes: 3 additions & 0 deletions internal/controller/dataprotectionapplication_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,9 @@ var debugMode = os.Getenv("DEBUG") == "true"
//+kubebuilder:rbac:groups=security.openshift.io,resources=securitycontextconstraints,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=security.openshift.io,resources=securitycontextconstraints,verbs=use,resourceNames=privileged
//+kubebuilder:rbac:groups="",resources=secrets;configmaps;pods;services;serviceaccounts;endpoints;persistentvolumeclaims;events,verbs=get;list;watch;create;update;patch;delete;deletecollection
//+kubebuilder:rbac:groups="",resources=resourcequotas,verbs=get;list;watch;create
//+kubebuilder:rbac:groups="",resources=namespaces,verbs=get;list;watch;create;update;patch

//+kubebuilder:rbac:groups=apps,resources=deployments;daemonsets,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=route.openshift.io,resources=routes,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=console.openshift.io,resources=consoleclidownloads,verbs=get;list;watch;create;update;patch;delete
Expand Down Expand Up @@ -98,6 +100,7 @@ func (r *DataProtectionApplicationReconciler) Reconcile(ctx context.Context, req

_, err := ReconcileBatch(r.Log,
r.ValidateDataProtectionCR,
r.ReconcileResourceQuota,
r.ReconcileFsRestoreHelperConfig,
r.ReconcileBackupStorageLocations,
r.ReconcileRegistrySecrets,
Expand Down
72 changes: 72 additions & 0 deletions internal/controller/resourcequota.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
package controller

import (
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
)

const (
oadpResourceQuotaName = "oadp-resource-quota"
oadpResourceQuotaPolicyAnnotation = "oadp.openshift.io/quota-policy"
oadpResourceQuotaPolicyValue = "create-if-absent"
)

// defaultOADPResourceQuotaHard is a generous mid-size ceiling for the OADP
// namespace. It intentionally omits limits.* because default Velero and
// node-agent pods set requests only; tracking limits in the quota can reject
// those pods at admission.
func defaultOADPResourceQuotaHard() corev1.ResourceList {
return corev1.ResourceList{
corev1.ResourcePods: resource.MustParse("200"),
corev1.ResourceRequestsCPU: resource.MustParse("50"),
corev1.ResourceRequestsMemory: resource.MustParse("64Gi"),
}
}

// ReconcileResourceQuota ensures ResourceQuota/oadp-resource-quota exists in
// the DPA namespace. Create-if-absent: never updates an existing quota's spec.
func (r *DataProtectionApplicationReconciler) ReconcileResourceQuota(log logr.Logger) (bool, error) {
log = log.WithValues("resourcequota", oadpResourceQuotaName)

ns := r.dpa.Namespace
existing := &corev1.ResourceQuota{}
err := r.Get(r.Context, types.NamespacedName{Name: oadpResourceQuotaName, Namespace: ns}, existing)
if err == nil {
log.V(1).Info("ResourceQuota already exists; leaving unchanged")
return true, nil
}
if !apierrors.IsNotFound(err) {
return false, err
}

quota := &corev1.ResourceQuota{
ObjectMeta: metav1.ObjectMeta{
Name: oadpResourceQuotaName,
Namespace: ns,
Labels: map[string]string{
"app.kubernetes.io/name": "oadp-operator",
"app.kubernetes.io/managed-by": "oadp-operator",
},
Annotations: map[string]string{
oadpResourceQuotaPolicyAnnotation: oadpResourceQuotaPolicyValue,
},
},
Spec: corev1.ResourceQuotaSpec{
Hard: defaultOADPResourceQuotaHard(),
},
}

log.Info("Creating default ResourceQuota")
if err := r.Create(r.Context, quota); err != nil {
// Another reconcile may have created it concurrently.
if apierrors.IsAlreadyExists(err) {
return true, nil
}
return false, err
}
return true, nil
}
Loading