Skip to content

OAPE-878: Add OpenSpec harness-evals docs and coding guidelines #465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
arun717 wants to merge 3 commits into
base: master
Choose a base branch
from
Open

OAPE-878: Add OpenSpec harness-evals docs and coding guidelines #465

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5d36c30
Add agentic docs and coding guidelines for AI-assisted development.
arun717 Jul 31, 2026
fe9e3fa
OAPE-878: Restructure agentic docs into harness-evals for OpenSpec.
arun717 Aug 4, 2026
3ba4b59
Resolved coderabbit PR review comments
arun717 Aug 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Cert Manager Operator - Agentic Documentation

**Component**: Cert Manager Operator for OpenShift
**Repository**: [openshift/cert-manager-operator](https://github.com/openshift/cert-manager-operator)
**Default branch**: `master` | **Go**: 1.26.0

> **Retrieval-first**: Prefer `harness-evals/harness-docs/` for architecture, ADRs, and coding playbooks. Platform hub: [openshift/enhancements/ai-docs](https://github.com/openshift/enhancements/tree/master/ai-docs).

## What is Cert Manager Operator?

Deploys/configures upstream **cert-manager** (and optionally **istio-csr** / **trust-manager**) on OpenShift. Operator NS `cert-manager-operator`; operand NS `cert-manager` (hardcoded).

**Key Principle**: Dual-stack — library-go for CertManager; controller-runtime for IstioCSR/TrustManager. **Apply methods differ** (do not assume SSA everywhere).

## Core Components

| CR | Scope / name | Stack | Gate |
|----|--------------|-------|------|
| [CertManager](harness-evals/harness-docs/domain/certmanager.md) | Cluster / `cluster` | library-go `resourceapply` | Always on |
| [IstioCSR](harness-evals/harness-docs/domain/istiocsr.md) | Namespaced / `default` | Create+Update | `IstioCSR` GA default true |
| [TrustManager](harness-evals/harness-docs/domain/trustmanager.md) | Cluster / `cluster` | **SSA** | `TrustManager` TP default false |

**Operands**: Certificate/Issuer/… → cert-manager; Bundle → trust-manager.
**Quick Start**: `oc get certmanager cluster -o yaml` | `make local-run`

## Critical Patterns

1. **Never assume uniform SSA** — CertManager library-go; IstioCSR Create+`UpdateWithRetry`; TrustManager only SSA + field owner `trust-manager-controller`. [components.md](harness-evals/harness-docs/architecture/components.md)
2. **Never hand-edit generated assets** — bindata.go, clients, regenerated `bindata/` / `bundle/` (`make generate`, `update-manifests`, `bundle`, `verify-bindata`).
3. **Greenfield = TrustManager** — copy SSA + `HandleReconcileResult`/`FromClientError` (30s requeue); not IstioCSR. [ADR-0002](harness-evals/harness-docs/decisions/adr-0002-apply-strategies.md)
4. **Feature gates** — `--unsupported-addon-features`; five touchpoints; no cluster FeatureSet for TrustManager (CM-1141). [ADR-0003](harness-evals/harness-docs/decisions/adr-0003-feature-gates.md)
5. **Ignore** `certmanager_controller.go` placeholder — RBAC markers only, never started.
6. **RELATED_IMAGE triple-sync** — manager.yaml env ↔ controller constants ↔ CSV `relatedImages` + `make bundle`.
7. **Cache** — no label-filtered ConfigMap/Issuer caches; use predicates when unlabeled watches are required.
8. **TLS 1.3** — strip cipher args via `StripArgsByKeys`; nil profile = Intermediate; Infrastructure discovery gates TLS + cloud-cred hooks (APIServer shares that factory, not discovered separately).

## Key Paths

| Area | Path |
|------|------|
| Startup | `pkg/operator/starter.go` → CertManager set → optional ctrl-runtime manager |
| Shared | `pkg/controller/common` (`CtrlClient`, errors, TLS/validation) |
| Features | `api/operator/v1alpha1/features.go`, `pkg/features` |
| Images | `RELATED_IMAGE_CERT_MANAGER_{CONTROLLER,WEBHOOK,CA_INJECTOR,ACMESOLVER,ISTIOCSR,TRUST_MANAGER}` |
| ADRs | [0001](harness-evals/harness-docs/decisions/adr-0001-dual-controller-frameworks.md) · [0002](harness-evals/harness-docs/decisions/adr-0002-apply-strategies.md) · [0003](harness-evals/harness-docs/decisions/adr-0003-feature-gates.md) |

## Documentation Map

```text
harness-evals/
harness-docs/ # architecture, domain, ADRs, *-guidelines, DEVELOPMENT, TESTING
evals/ # OpenSpec stage eval gates (repo-assessment/plan/tasks/code-generation)
docs/{proxy,cloud_credentials,operand_metrics}.md # human product docs
```

| Need | Start here |
|------|------------|
| Spec / CEL / NetworkPolicy enum | `harness-evals/harness-docs/domain/*.md` |
| Errors, cache, TLS, cloud, FIPS | `harness-evals/harness-docs/architecture/components.md` |
| Controllers / apply / gates | `harness-evals/harness-docs/operator-controllers-guidelines.md` |
| OLM / relatedImages | `harness-evals/harness-docs/olm-packaging-guidelines.md` + DEVELOPMENT |
| Unit assert Patch vs Update | `harness-evals/harness-docs/CERT_MANAGER_OPERATOR_TESTING.md` |
| FIPS build rules | `harness-evals/harness-docs/fips-guidelines.md` |

**AI Agent Path**: domain → components.md → matching `*-guidelines.md` in harness-docs → DEVELOPMENT/TESTING

**Also**: [DEVELOPMENT](harness-evals/harness-docs/CERT_MANAGER_OPERATOR_DEVELOPMENT.md) · [TESTING](harness-evals/harness-docs/CERT_MANAGER_OPERATOR_TESTING.md) · [enhancements](harness-evals/harness-docs/references/enhancements.md) · [ecosystem](harness-evals/harness-docs/references/ecosystem.md)

**Guideline index**: `harness-evals/harness-docs/{operator-controllers,error-handling,api-contracts,olm-packaging,integration,security,testing,fips}-guidelines.md`

**Platform**: [hub](https://github.com/openshift/enhancements/tree/master/ai-docs) · [operator-patterns](https://github.com/openshift/enhancements/tree/master/ai-docs/platform/operator-patterns) · [status-conditions](https://github.com/openshift/enhancements/blob/master/ai-docs/platform/operator-patterns/status-conditions.md)

## External References

- [Product docs](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html)
- [Upstream cert-manager](https://cert-manager.io/docs/)
- [README](README.md) · [docs/proxy.md](docs/proxy.md) · [docs/cloud_credentials.md](docs/cloud_credentials.md) · [docs/operand_metrics.md](docs/operand_metrics.md)

---

**Platform Documentation**: [openshift/enhancements/ai-docs](https://github.com/openshift/enhancements/tree/master/ai-docs)
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ test: manifests generate vet test-apis test-unit ## Run tests.

.PHONY: test-unit
test-unit: ## Run unit tests.
go test $$(go list ./... | grep -vE 'test/[e2e|apis|utils]') -coverprofile cover.out
go test $$(go list ./... | grep -vE 'test/(e2e|apis|utils)(/|$$)') -coverprofile cover.out

# Utilize controller-runtime provided envtest for API integration test
.PHONY: test-apis ## Run only the api integration tests.
Expand Down
12 changes: 12 additions & 0 deletions harness-evals/evals/code-generation_eval.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
stage: code-generation
artifact: fork working copy (code + tests)
version: 1
eval_count: 0
oape_commands:
- api-generate
- api-generate-tests
- api-implement
- e2e-generate
- manual
- any
evals: []
3 changes: 3 additions & 0 deletions harness-evals/evals/plan_eval.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
stage: plan
template: templates/plan-template.md

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

for spec in \
  harness-evals/evals/plan_eval.yaml \
  harness-evals/evals/repo-assessment_eval.yaml \
  harness-evals/evals/tasks_eval.yaml
do
  template="$(awk -F': ' '$1 == "template" { print $2; exit }' "$spec")"
  config_dir="$(dirname "$spec")"
  candidate="$config_dir/$template"

  if [[ -f "$candidate" ]]; then
    printf 'FOUND: %s -> %s\n' "$spec" "$candidate"
  else
    printf 'MISSING relative to config: %s -> %s\n' "$spec" "$candidate"
  fi
done

printf '\nTemplate files:\n'
fd -t f -a '(plan-template|repo-assessment-template|tasks-template)\.md$' .

printf '\nResolver references:\n'
rg -n -C 3 'template:|plan_eval|repo-assessment_eval|tasks_eval|templates/' .

Repository: openshift/cert-manager-operator

Length of output: 50389

Create the missing evaluation template files.

The three evaluation stages cannot load without their template files. All three templates are missing from the repository:

  • harness-evals/evals/templates/plan-template.md is missing (referenced by plan_eval.yaml)
  • harness-evals/evals/templates/repo-assessment-template.md is missing (referenced by repo-assessment_eval.yaml)
  • harness-evals/evals/templates/tasks-template.md is missing (referenced by tasks_eval.yaml)

Create these files in the harness-evals/evals/templates/ directory with the content each evaluation stage expects.

📍 Affects 3 files
  • harness-evals/evals/plan_eval.yaml#L2-L2 (this comment)
  • harness-evals/evals/repo-assessment_eval.yaml#L2-L2
  • harness-evals/evals/tasks_eval.yaml#L2-L2
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@harness-evals/evals/plan_eval.yaml` at line 2, Create the three missing
evaluation template files: harness-evals/evals/templates/plan-template.md for
plan_eval.yaml, harness-evals/evals/templates/repo-assessment-template.md for
repo-assessment_eval.yaml, and harness-evals/evals/templates/tasks-template.md
for tasks_eval.yaml. Populate each with the content expected by its
corresponding evaluation stage so all three template references resolve
successfully.

evals: []
3 changes: 3 additions & 0 deletions harness-evals/evals/repo-assessment_eval.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
stage: repo-assessment
template: templates/repo-assessment-template.md
evals: []
3 changes: 3 additions & 0 deletions harness-evals/evals/tasks_eval.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
stage: tasks
template: templates/tasks-template.md
evals: []
81 changes: 81 additions & 0 deletions harness-evals/harness-docs/CERT_MANAGER_OPERATOR_DEVELOPMENT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Cert Manager Operator - Development Guide

> Layout / controller comparison: [architecture/components.md](./architecture/components.md).
> Agent playbooks: [*-guidelines.md](.) (operator-controllers, olm-packaging, fips, integration, …).

## Quick Start

- Go **1.26.0+** (`go.mod`), OpenShift + `KUBECONFIG`, container engine

```bash
make deploy
oc scale --replicas=0 deploy --all -n cert-manager-operator
make local-run # RELATED_IMAGE_* + versions from Makefile
make build && make generate && make update-manifests && make verify
```

FIPS: `make build` sources `hack/go-fips.sh`. WARN (non-FIPS) builds are **local-only** — never CI/prod/image push. Keep `go.mod` replace `openshift/jetstack-cert-manager` lockstep with `CERT_MANAGER_VERSION`. See [fips-guidelines.md](./fips-guidelines.md).

## Common Tasks

### CertManager deployment overrides

1. Spec change → `certmanager_types.go` → `make manifests generate`
2. Wire in `deployment_*.go` / `deployment_overrides.go` (library-go — **not** TrustManager SSA)

### New controller-runtime operand (greenfield)

1. Copy **TrustManager** (SSA `Patch` + unique `FieldOwner`), not IstioCSR Create+Update
2. Feature gate — all five touchpoints ([ADR-0003](./decisions/adr-0003-feature-gates.md))
3. Image triple-sync (below) + bindata via `hack/update-*-manifests.sh` + `make update-manifests` / `verify-bindata`
4. RBAC via `+kubebuilder:rbac` → `make manifests` → `make bundle` (never hand-edit CSV RBAC)
5. Reuse `common.HandleReconcileResult` / `FromClientError` / validation helpers (`defaultRequeueTime=30s`)
6. E2E + Ginkgo labels under `test/e2e/`

### RELATED_IMAGE / relatedImages sync

| CSV `relatedImages.name` | Env | Consumer |
|--------------------------|-----|----------|
| `cert-manager-controller` / webhook / ca-injector / acmesolver | `RELATED_IMAGE_CERT_MANAGER_*` | `related_images.go` |
| `cert-manager-istiocsr` | `RELATED_IMAGE_CERT_MANAGER_ISTIOCSR` | `istiocsr/constants.go` |
| `cert-manager-trust-manager` | `RELATED_IMAGE_CERT_MANAGER_TRUST_MANAGER` | `trustmanager/constants.go` |

`RELATED_IMAGE_*` / `*_OPERAND_IMAGE_VERSION` literals in **`config/manager/manager.yaml` are hand-maintained** — bumping Makefile `CERT_*_VERSION` alone does **not** update them. Edit `manager.yaml` to match, then `make bundle` (auto-fills CSV `relatedImages` from `RELATED_IMAGE_*` env). New operand image also needs controller constants/map. See [olm-packaging-guidelines.md](./olm-packaging-guidelines.md).

### Bump operand versions

1. Makefile `CERT_MANAGER_VERSION` / `ISTIO_CSR_VERSION` / `TRUST_MANAGER_VERSION` (+ bundle version)
2. `make update-manifests` + keep jetstack replace version lockstep
3. **Manually** update matching `RELATED_IMAGE_*` / `*_OPERAND_IMAGE_VERSION` in `config/manager/manager.yaml`
4. `make bundle`; refresh CSV description links / RBAC if upstream changed; run `hack/verify-crds*.sh` directly if CRDs changed (not in `make verify-scripts`)

### Enable TrustManager locally

`--unsupported-addon-features=TrustManager=true`; create `TrustManager` named `cluster`.

## Common Mistakes

1. Hand-edit bindata / generated clients / generated `bundle/` CSV
2. Assume SSA for IstioCSR or CertManager
3. Logic in `certmanager_controller.go` placeholder
4. Label-filtered cache for ConfigMaps or Issuer/ClusterIssuer
5. Expect IstioCSR/TrustManager delete to GC operands (warn-only TODO)
6. TLS 1.3 cipher-suite args (use `StripArgsByKeys`)
7. Create CredentialsRequest in-operator (mount-only; controller Deployment only)
8. Point cert-manager replace at upstream or ship non-FIPS image

## Component-Specific Notes

| Topic | Detail |
|-------|--------|
| Namespaces | Operator `cert-manager-operator`; operands `cert-manager` |
| Cloud creds | AWS `/.aws` + `AWS_SDK_LOAD_CONFIG=1`; GCP ADC path — `../../docs/cloud_credentials.md` |
| Trusted CA | Fixed mount path; missing CM = retryable — `integration-guidelines.md` |
| Uninstall | Manual operand cleanup; `console.openshift.io/disable-operand-delete: "true"` |

## See Also

- [CERT_MANAGER_OPERATOR_TESTING.md](./CERT_MANAGER_OPERATOR_TESTING.md)
- [architecture/components.md](./architecture/components.md)
- [operator-controllers-guidelines.md](./operator-controllers-guidelines.md)
- [olm-packaging-guidelines.md](./olm-packaging-guidelines.md)
59 changes: 59 additions & 0 deletions harness-evals/harness-docs/CERT_MANAGER_OPERATOR_TESTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Cert Manager Operator - Testing Guide

> **Generic Testing Practices**: See [Platform ai-docs](https://github.com/openshift/enhancements/tree/master/ai-docs). Deep playbook: [testing-guidelines.md](./testing-guidelines.md).

## Test Layout

| Suite | Location | How to run |
|-------|----------|------------|
| Unit | `pkg/**/*_test.go` | `make test-unit` |
| API / envtest | `test/apis`, `api/operator/v1alpha1/tests/**` | `make test-apis` |
| E2E | `test/e2e/` (build tag `e2e`, Ginkgo) | `make test-e2e` |

`make test` = manifests + generate + vet + test-apis + test-unit — **not** e2e.

## Unit Tests — assert the real apply path

| Controller | Assert | Fake |
|------------|--------|------|
| CertManager | library-go `resourceapply` / expected generation / rollout | library-go fake clientset — **no** `FakeCtrlClient` |
| IstioCSR | `CreateCallCount` / `UpdateWithRetryCallCount` | `fakes.FakeCtrlClient` — **not** `PatchCallCount` |
| TrustManager | `PatchCallCount` with `client.Apply` + FieldOwner `trust-manager-controller` | `fakes.FakeCtrlClient` — **not** Update/Create |

A TrustManager test asserting `UpdateCallCount` (or IstioCSR asserting `PatchCallCount`) is the wrong path even if green. Regenerate fakes with `make generate-fakes` after changing `CtrlClient` — never hand-edit `pkg/controller/common/fakes/`.

```bash
make test-unit
go test -count=1 ./pkg/controller/trustmanager/...
go test -count=1 ./pkg/controller/istiocsr/...
go test -count=1 ./pkg/controller/certmanager/...
```

## API Tests (`make test-apis`)

envtest + Ginkgo against CEL/`XValidation` fixtures under `api/operator/v1alpha1/tests/{certmanagers,istiocsrs,trustmanagers}.operator.openshift.io/`. Required after Spec validation changes — unit tests cannot enforce CEL.

## E2E Tests

```bash
make test-e2e
# Default label filter (Makefile) — always single-quote overrides:
# Platform: isSubsetOf {AWS,Generic} && CredentialsMode: isSubsetOf {Mint} && !Feature:ServiceMesh
make test-e2e E2E_GINKGO_LABEL_FILTER='Feature:TrustManager && !TechPreview:Inverted'
```

| Area | Labels |
|------|--------|
| TrustManager | `Feature:TrustManager`, `TechPreview` |
| Gate-disabled TrustManager | `TechPreview:Inverted` (do **not** merge with enabled suite) |
| TLS profile | `Feature:TLSProfile`, `TechPreview` |
| IstioCSR | `Feature:IstioCSR` |
| ServiceMesh smoke | `Feature:IstioCSR-ServiceMesh` (excluded by default `!Feature:ServiceMesh`) |

Tips: singleton CR names (`cluster` / `default`); operands in `cert-manager` NS; `make test-e2e-debug-cluster` on failure.

## See Also

- [testing-guidelines.md](./testing-guidelines.md)
- [CERT_MANAGER_OPERATOR_DEVELOPMENT.md](./CERT_MANAGER_OPERATOR_DEVELOPMENT.md)
- [architecture/components.md](./architecture/components.md)
Loading