Skip to content

local catalog and inherit ref#3144

Open
gazarenkov wants to merge 4 commits into
redhat-developer:mainfrom
gazarenkov:local-test-plugins-catalog
Open

local catalog and inherit ref#3144
gazarenkov wants to merge 4 commits into
redhat-developer:mainfrom
gazarenkov:local-test-plugins-catalog

Conversation

@gazarenkov

Copy link
Copy Markdown
Member

Description

  • For local test (make run, make test,...) download plugin catalog and make it used as default plugins configuration
  • For compatibility: support {{inherit}} dynamic-plugins reference

Which issue(s) does this PR fix or relate to

https://redhat.atlassian.net/browse/RHIDP-15008
https://redhat.atlassian.net/browse/RHIDP-15015
https://redhat.atlassian.net/browse/RHIDP-15016

PR acceptance criteria

  • Tests
  • Documentation

How to test changes / Special notes to the reviewer

Happy path:
Run make local-dynamic-plugins
Run make install run and create any Backstage CR.
It should correctly work (but sending the whole

Building Container Images for Testing

Need to test container images from this PR?

For Maintainers: To trigger a test image build, review the code and comment /build-images.
This always builds the HEAD of the PR branch.

For Contributors: Ask a maintainer to run /build-images.

Images will be built and pushed to Quay with links posted in comments.

@gazarenkov gazarenkov requested a review from a team as a code owner July 6, 2026 10:04
Comment thread hack/create-local-dynamic-plugins.sh Fixed
Comment thread hack/create-local-dynamic-plugins.sh Fixed
Comment thread hack/create-local-dynamic-plugins.sh Fixed
@rhdh-qodo-merge

Copy link
Copy Markdown

Code Review by Qodo

Grey Divider

Sorry, something went wrong

We weren't able to complete the code review on our side. Please try again

Grey Divider

Qodo Logo

@rhdh-qodo-merge

Copy link
Copy Markdown

PR Summary by Qodo

Local catalog plugin setup and {{inherit}} reference resolution

✨ Enhancement 🧪 Tests 📝 Documentation 🕐 20-40 Minutes

Grey Divider

AI Description

• Adds make local-dynamic-plugins target that extracts dynamic-plugins.default.yaml from the
 catalog-index OCI image via skopeo and writes it to a git-ignored
 config/profile/rhdh/local-test/ directory for local development use.
• Replaces inline mkdir/cp commands in make run, make test, and make integration-test with a
 shared hack/copy-local-dynamic-plugins.sh script that also overlays the local-test
 dynamic-plugins.yaml for the rhdh profile.
• Implements {{inherit}} reference resolution in pkg/model/dynamic-plugins-reference.go: during
 ConfigMap merge, user plugin URLs containing :{{inherit}} are replaced with the matching
 version/digest from the default plugin catalog, with an error returned if no match is found.
• Adds DynaPlugin.BaseURL() helper to strip tag/digest/plugin-path suffixes from OCI URLs for
 lookup purposes.
• Documents the new local setup workflow and the {{inherit}} URL reference syntax in
 docs/developer.md and docs/dynamic-plugins.md.
Diagram

graph TD
    A(["make local-dynamic-plugins"]) --> B["hack/create-local-dynamic-plugins.sh"]
    B -->|skopeo pull| C[("catalog-index OCI image")]
    C -->|extract dynamic-plugins.default.yaml| D["config/profile/rhdh/local-test/dynamic-plugins.yaml"]

    E(["make run / test / integration-test"]) --> F["hack/copy-local-dynamic-plugins.sh"]
    F -->|copy default-config| G["bin/default-config/"]
    D -->|overlay rhdh profile| G

    H["MergePluginsData"] --> I["resolveReferences"]
    I --> J["resolveInheritReference"]
    J -->|lookup base URL| K["buildBaseURLMap"]
    K -->|DynaPlugin.BaseURL| L["Default plugins catalog"]
    I -->|resolved plugins| H

    subgraph Legend
      direction LR
      _make(["Make target"]) ~~~ _script["Shell script"] ~~~ _img[("OCI image")] ~~~ _go["Go function"]
    end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. docker run to extract plugin YAML
  • ➕ Simpler script — no layer iteration needed
  • ➕ Handles multi-arch automatically
  • ➖ Requires a running Docker daemon
  • ➖ Pulls the full image (slower)
  • ➖ Not suitable for rootless/CI environments
2. Commit a snapshot of dynamic-plugins.yaml to the repo
  • ➕ No tooling dependency (skopeo)
  • ➕ Works offline
  • ➖ Snapshot goes stale quickly
  • ➖ Pollutes git history with large YAML files
  • ➖ Defeats the purpose of pulling from the authoritative catalog image

Recommendation: The PR's approach is sound for both concerns. For local setup, using skopeo to pull and extract from the OCI image is the right tool — it avoids running a container and works without Docker. An alternative would be docker run --rm to extract the file, which is simpler but requires a running Docker daemon and pulls the full image. The current approach is preferable for CI/CD and rootless environments. For the {{inherit}} resolution, integrating it directly into MergePluginsData is the correct place — it keeps resolution transparent to callers. The only notable gap is that ref:// is stubbed and will return an error if used, which should be clearly communicated to users.

Files changed (13) +615 / -11

Enhancement (2) +140 / -0
dynamic-plugins-reference.goAdd {{inherit}} reference resolution for dynamic plugin package URLs +133/-0

Add {{inherit}} reference resolution for dynamic plugin package URLs

• Introduces 'resolveReferences', 'resolveInheritReference', 'buildBaseURLMap', and 'DynaPlugin.BaseURL()'. When a user plugin URL contains ':{{inherit}}', the operator resolves it to the matching version/digest from the default plugin catalog during ConfigMap merge. Stubs out 'ref://' resolution for future implementation.

pkg/model/dynamic-plugins-reference.go

dynamic-plugins.goInvoke resolveReferences during ConfigMap merge in MergePluginsData +7/-0

Invoke resolveReferences during ConfigMap merge in MergePluginsData

• Calls 'resolveReferences' on the user-supplied plugin list (second config) using the default plugins (first config) as the base before performing the merge. Returns an error if any reference cannot be resolved.

pkg/model/dynamic-plugins.go

Refactor (5) +5 / -5
zz_generated.deepcopy.goFix import alias style in generated deepcopy file +1/-1

Fix import alias style in generated deepcopy file

• Changes the import from a quoted path '"k8s.io/apimachinery/pkg/apis/meta/v1"' to an aliased form 'v1 "k8s.io/apimachinery/pkg/apis/meta/v1"' to match goimports formatting.

api/v1alpha1/zz_generated.deepcopy.go

zz_generated.deepcopy.goFix import alias style in generated deepcopy file (v1alpha2) +1/-1

Fix import alias style in generated deepcopy file (v1alpha2)

• Applies the same goimports-driven alias fix as v1alpha1 for the apiextensions/v1 import.

api/v1alpha2/zz_generated.deepcopy.go

zz_generated.deepcopy.goFix import alias style in generated deepcopy file (v1alpha3) +1/-1

Fix import alias style in generated deepcopy file (v1alpha3)

• Applies the same goimports-driven alias fix as v1alpha2.

api/v1alpha3/zz_generated.deepcopy.go

zz_generated.deepcopy.goFix import alias style in generated deepcopy file (v1alpha4) +1/-1

Fix import alias style in generated deepcopy file (v1alpha4)

• Applies the same goimports-driven alias fix as v1alpha2.

api/v1alpha4/zz_generated.deepcopy.go

zz_generated.deepcopy.goFix import alias style in generated deepcopy file (v1alpha5) +1/-1

Fix import alias style in generated deepcopy file (v1alpha5)

• Applies the same goimports-driven alias fix as v1alpha2.

api/v1alpha5/zz_generated.deepcopy.go

Tests (1) +249 / -0
dynamic-plugins-reference_test.goTests for BaseURL, resolveInheritReference, resolveReferences, and MergePluginsData with inherit +249/-0

Tests for BaseURL, resolveInheritReference, resolveReferences, and MergePluginsData with inherit

• Comprehensive table-driven tests covering OCI URL base extraction, inherit reference resolution (with and without plugin paths), mixed plugin lists, and error cases for unresolvable references. Also includes integration-style tests through 'MergePluginsData'.

pkg/model/dynamic-plugins-reference_test.go

Documentation (2) +40 / -0
developer.mdDocument local dynamic plugins setup workflow for rhdh profile +26/-0

Document local dynamic plugins setup workflow for rhdh profile

• Adds a new section explaining how to run 'make local-dynamic-plugins' to generate the git-ignored 'local-test' directory, including requirements (skopeo), optional image override, and a note that 'make run/test/integration-test' require this directory to exist.

docs/developer.md

dynamic-plugins.mdDocument Plugin URL References and {{inherit}} syntax +14/-0

Document Plugin URL References and {{inherit}} syntax

• Adds a new 'Plugin URL References' section describing the operator's reference resolution behavior during ConfigMap merge and linking to the upstream OCI Package Version Inheritance documentation for the ':{{inherit}}' syntax.

docs/dynamic-plugins.md

Other (3) +181 / -6
MakefileConsolidate local config setup into shared script; add local-dynamic-plugins target +8/-6

Consolidate local config setup into shared script; add local-dynamic-plugins target

• Replaces repeated inline 'mkdir/cp' commands in 'run', 'test', and 'integration-test' targets with a single call to 'hack/copy-local-dynamic-plugins.sh'. Adds a new 'local-dynamic-plugins' target that invokes 'hack/create-local-dynamic-plugins.sh' to extract plugin config from the catalog-index image.

Makefile

copy-local-dynamic-plugins.shNew script: copy default-config and overlay local-test dynamic-plugins for rhdh profile +56/-0

New script: copy default-config and overlay local-test dynamic-plugins for rhdh profile

• Copies 'default-config' and 'plugin-deps' from the profile directory to LOCALBIN. For the 'rhdh' profile, additionally overlays 'dynamic-plugins.yaml' from the git-ignored 'local-test' directory, failing with a helpful error if the directory is missing.

hack/copy-local-dynamic-plugins.sh

create-local-dynamic-plugins.shNew script: extract dynamic-plugins.default.yaml from catalog-index OCI image +117/-0

New script: extract dynamic-plugins.default.yaml from catalog-index OCI image

• Uses 'skopeo' to pull the catalog-index image and extract 'dynamic-plugins.default.yaml' from its layers. Wraps the extracted YAML in a Kubernetes ConfigMap and writes it to 'config/profile/rhdh/local-test/dynamic-plugins.yaml' for local testing.

hack/create-local-dynamic-plugins.sh

@rhdh-qodo-merge rhdh-qodo-merge Bot added documentation Improvements or additions to documentation enhancement New feature or request Tests labels Jul 6, 2026
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.16393% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.28%. Comparing base (d6b77e9) to head (f8dceec).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
pkg/model/dynamic-plugins-reference.go 89.28% 5 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3144      +/-   ##
==========================================
+ Coverage   61.51%   62.28%   +0.76%     
==========================================
  Files          37       38       +1     
  Lines        2162     2222      +60     
==========================================
+ Hits         1330     1384      +54     
- Misses        691      696       +5     
- Partials      141      142       +1     
Flag Coverage Δ
nightly ?
unittests 62.28% <90.16%> (+0.76%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pkg/model/dynamic-plugins.go 78.10% <100.00%> (+0.65%) ⬆️
pkg/model/dynamic-plugins-reference.go 89.28% <89.28%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sonarqubecloud

sonarqubecloud Bot commented Jul 6, 2026

Copy link
Copy Markdown

@rm3l

rm3l commented Jul 7, 2026

Copy link
Copy Markdown
Member

/build-images

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

PR images built successfully!

Images are available for testing:

  1. Operator: quay.io/rhdh-community/operator:0.11.0-pr-3144-f8dceec
  2. Bundle: quay.io/rhdh-community/operator-bundle:0.11.0-pr-3144-f8dceec
  3. Catalog: quay.io/rhdh-community/operator-catalog:0.11.0-pr-3144-f8dceec

Also available with PR number tag:

  • quay.io/rhdh-community/operator:0.11.0-pr-3144
  • quay.io/rhdh-community/operator-bundle:0.11.0-pr-3144
  • quay.io/rhdh-community/operator-catalog:0.11.0-pr-3144

Triggered by @rm3l

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants