ci: validate loaded plugins after RHDH startup

[rm3l]

Description

The CI smoke tests only checked for HTTP 200 from the homepage, which meant some configuration changes could silently break plugin loading without CI catching it.

This checks the plugins that are loaded once RHDH is started and compares against the expected configuration.

Which issue(s) does this PR fix or relate to

• Fixes RHDHBUGS-3440

PR acceptance criteria

• Tests updated and passing
• Documentation updated
• Built-in TechDocs updated if needed. Note that TechDocs changes may need to be reviewed by a Product Manager and/or Architect to ensure content accuracy, clarity, and alignment with user needs.

How to test changes / Special notes to the reviewer

The checks on this PR are passing with the changes here, e.g.: https://github.com/redhat-developer/rhdh-local/actions/runs/28782801578/job/85342325860?pr=262#step:3:2001

To verify locally:

1. Start RHDH locally: docker compose up -d
2. Wait for it to be ready, then run: ./tests/validate-loaded-plugins.sh
3. All configured plugins should show [PASS]
4. The disabled global-floating-action-button plugin should also show [PASS] (confirmed not loaded)

For orchestrator config, pass the extra config: ./tests/validate-loaded-plugins.sh orchestrator/configs/dynamic-plugins/dynamic-plugins.yaml

Assisted-by: Claude

[rhdh-qodo-merge]

PR Summary by Qodo

CI: validate configured dynamic plugins are loaded after RHDH startup

🧪 Tests ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Add CI validation that configured dynamic plugins are actually loaded at runtime.
• Query RHDH loaded-plugins API and compare against dynamic-plugins YAML configuration.
• Ensure orchestrator compose tests load orchestrator plugins via dynamic-plugins.override.yaml.

Diagram

graph TD
  A["GitHub Action: compose test"] --> B["RHDH container (7007)"]
  A --> C["validate-loaded-plugins.sh"] --> D["dynamic-plugins*.yaml"]
  C --> E["/api/auth/guest/refresh"] --> F["/api/extensions/loaded-plugins"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Fail fast in RHDH on plugin load mismatches

• ➕ Catches misconfiguration in all environments, not just CI
• ➕ Eliminates dependency on CI-side parsing/normalization logic
• ➖ Requires product/runtime change and careful compatibility handling
• ➖ Harder to scope/roll out than a CI-only guardrail

2. Add a dedicated CI test harness (Node/Playwright) for runtime assertions

• ➕ Richer assertions and reporting than bash
• ➕ Better long-term maintainability as checks grow
• ➖ Heavier dependencies and more moving parts for a small validation
• ➖ Longer setup time and higher CI execution cost

3. Validate plugin loading via logs/health checks instead of loaded-plugins API

• ➕ Avoids coupling to an API contract if it changes
• ➕ Can catch load errors even when endpoint is unavailable
• ➖ Log scraping is brittle and environment-dependent
• ➖ Harder to reliably map expected config to runtime outcomes

Recommendation: Keep the current approach: querying /api/extensions/loaded-plugins is a direct, low-overhead signal that matches the goal (detect silent plugin load regressions). The chosen CI-side script is an appropriate scope for RHDHBUGS-3440; consider migrating to a richer harness only if assertions expand beyond plugin presence/absence.

Files changed (2) +200 / -0

Tests (1) +183 / -0

validate-loaded-plugins.shNew smoke test to compare configured vs loaded dynamic plugins +183/-0

New smoke test to compare configured vs loaded dynamic plugins

• Introduces a bash script that reads the effective dynamic-plugins config (override preferred) plus optional extra configs, then authenticates via guest refresh and queries /api/extensions/loaded-plugins. It normalizes plugin naming differences (OCI image vs npm scoped names) and asserts enabled plugins are present while disabled plugins are absent.

tests/validate-loaded-plugins.sh

Other (1) +17 / -0

action.yamlAppend orchestrator plugins to override and run plugin-load validation +17/-0

Append orchestrator plugins to override and run plugin-load validation

• Adds a step for orchestrator compose configs to ensure orchestrator plugins are included in configs/dynamic-plugins/dynamic-plugins.override.yaml (copying the base config if missing). Adds a post-start CI step to run tests/validate-loaded-plugins.sh after RHDH becomes ready.

.github/actions/rhdh-local-compose-test/action.yaml

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials

1. yq errors skip validation ✓ Resolved 🐞 Bug ≡ Correctness

Description

tests/validate-loaded-plugins.sh suppresses yq errors and treats “no plugins parsed” as success
(exit 0), so missing/incompatible yq or parse failures can silently bypass the new CI check even
when the config has plugins.

Code

tests/validate-loaded-plugins.sh[R64-89]

+# Extract plugin names that should be enabled (not explicitly disabled).
+expected_plugins=()
+for cfg in "${config_files[@]}"; do
+    while IFS= read -r raw_pkg; do
+        [[ -z "$raw_pkg" ]] && continue
+        raw_pkg="${raw_pkg//\'/}"
+        raw_pkg="${raw_pkg//\"/}"
+        expected_plugins+=("$(extract_plugin_name "$raw_pkg")")
+    done < <(yq '.plugins[] | select(.disabled != true and .enabled != false) | .package' "$cfg" 2>/dev/null)
+done
+
+# Extract plugin names that should NOT be loaded (explicitly disabled).
+disabled_plugins=()
+for cfg in "${config_files[@]}"; do
+    while IFS= read -r raw_pkg; do
+        [[ -z "$raw_pkg" ]] && continue
+        raw_pkg="${raw_pkg//\'/}"
+        raw_pkg="${raw_pkg//\"/}"
+        disabled_plugins+=("$(extract_plugin_name "$raw_pkg")")
+    done < <(yq '.plugins[] | select(.disabled == true or .enabled == false) | .package' "$cfg" 2>/dev/null)
+done
+
+if [[ ${#expected_plugins[@]} -eq 0 ]] && [[ ${#disabled_plugins[@]} -eq 0 ]]; then
+    echo "WARNING: No plugins found in config files. Nothing to validate."
+    exit 0
+fi

Relevance

⭐⭐⭐ High

Team previously fixed “yq: command not found” and prefers fail-fast scripts; suppressing yq errors
undermines CI.

PR-#47
PR-#78

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The script discards yq errors and then exits successfully when nothing is parsed, which means the
validation can be skipped even though the repo’s dynamic plugins config clearly contains plugin
entries.

tests/validate-loaded-plugins.sh[64-89]
configs/dynamic-plugins/dynamic-plugins.yaml[7-27]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`tests/validate-loaded-plugins.sh` redirects `yq` stderr to `/dev/null` and never checks `yq`’s exit status. If `yq` is missing, incompatible, or the expression fails, both `expected_plugins` and `disabled_plugins` remain empty and the script exits 0 with a warning, silently skipping validation.

## Issue Context
This script is now executed in CI from the composite action, so a silent success means CI will not actually validate plugin loading.

## Fix Focus Areas
- tests/validate-loaded-plugins.sh[16-89]

## What to change
- Add explicit preflight checks:
 - `command -v yq` and `command -v jq` must succeed or exit 1.
- Stop hiding `yq` errors (`2>/dev/null`) and/or use `yq -e` / `set -o pipefail` patterns that fail the script when parsing fails.
- Use raw output to avoid manual quote stripping (for mikefarah/yq v4): `yq -r '...'`.
- Differentiate between:
 - a genuinely empty plugins list (acceptable to exit 0), and
 - a parse/tooling failure (must exit 1). For example, first verify the config has a `.plugins` array, then extract; if extraction fails, treat as error.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Hardcoded temp file path ✓ Resolved 🐞 Bug ☼ Reliability

Description

tests/validate-loaded-plugins.sh always writes the loaded-plugins response to
/tmp/loaded-plugins.json without cleanup, which is less robust if the script is run multiple times
in the same environment and makes debugging harder when stale files linger.

Code

tests/validate-loaded-plugins.sh[R104-121]

+echo "Fetching loaded plugins from ${RHDH_URL}/api/extensions/loaded-plugins ..."
+http_code=$(curl -sS -o /tmp/loaded-plugins.json -w "%{http_code}" \
+    -H "Authorization: Bearer ${RHDH_TOKEN}" \
+    "${RHDH_URL}/api/extensions/loaded-plugins") || true
+
+if [[ "$http_code" != "200" ]]; then
+    echo "ERROR: Could not reach loaded-plugins endpoint (HTTP $http_code)." >&2
+    echo "Response body:" >&2
+    cat /tmp/loaded-plugins.json >&2 2>/dev/null || true
+    exit 1
+fi
+
+loaded_names=$(jq -r '.[].name' /tmp/loaded-plugins.json 2>/dev/null)
+if [[ -z "$loaded_names" ]]; then
+    echo "ERROR: Loaded plugins response was empty or could not be parsed." >&2
+    echo "Response body:" >&2
+    cat /tmp/loaded-plugins.json >&2 2>/dev/null || true
+    exit 1

Relevance

⭐⭐ Medium

Repo already uses hardcoded /tmp files (accepted), but no clear precedent on requiring
mktemp/cleanup for scripts.

PR-#177

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The script writes and reads a fixed filename in /tmp and does not remove it.

tests/validate-loaded-plugins.sh[104-121]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The script uses a fixed path `/tmp/loaded-plugins.json` and never deletes it.

## Issue Context
In CI this is usually fine, but it’s easy to make more robust (and safer for repeated local runs) with a unique temp file and cleanup.

## Fix Focus Areas
- tests/validate-loaded-plugins.sh[104-122]

## What to change
- Replace the hard-coded temp path with `tmpfile=$(mktemp)`.
- Add `trap 'rm -f "$tmpfile"' EXIT`.
- Use `$tmpfile` for all reads/writes.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud