feat(agentready): add project-level filtering to Grafana dashboards by CryptoRodeo · Pull Request #94 · konflux-ci/devlake

CryptoRodeo · 2026-06-02T20:36:58Z

⚠️ Pre Checklist

Please complete ALL items in this checklist, and remove before submitting

I have read through the Contributing Documentation.
I have added relevant tests.
I have added relevant documentation.
I will add labels to the PR, such as pr-type/bug-fix, pr-type/feature-development, etc.

Summary

What does this PR do?

Add ${project} template variable and project_mapping JOINs to all three dashboards (fleet-overview, findings-analysis, repo-detail) so panels scope data to the selected DevLake project.

Does this close any open issues?

Closes KFLUXDP-1008

Screenshots

project-filter-test.mp4

Other Information

Any other information that is important to this PR.

Assisted-By: Claude Opus 4.6

Add ${project} template variable and project_mapping JOINs to all three dashboards (fleet-overview, findings-analysis, repo-detail) so panels scope data to the selected DevLake project. Assisted-By: Claude Opus 4.6 Signed-off-by: Bryan Ramos <bramos@redhat.com>

Dannyb48 · 2026-06-02T20:43:30Z

/review

gbenhaim · 2026-06-03T07:07:38Z

/review

gbenhaim · 2026-06-03T07:11:00Z

/agentic_review

qodo-app-for-konflux-ci · 2026-06-03T07:14:50Z

Code Review by Qodo

Sorry, something went wrong

We weren't able to complete the code review on our side. Please try again

Dannyb48 · 2026-06-04T13:40:34Z

/agentic_review

qodo-app-for-konflux-ci · 2026-06-04T13:40:43Z

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (1) 🔗 Cross-repo conflicts (0)

Context used

✅ Compliance rules (platform): 110 rules

1. Changes under plugins/agentready 📘 Rule violation § Compliance

Description

This PR modifies files under backend/plugins/agentready/, which is outside the allowed owned
plugin directories for this change. This violates the restriction that only
backend/plugins/aireview/, backend/plugins/codecov/, and backend/plugins/testregistry/ may be
touched.

Code

backend/plugins/agentready/grafana/findings-analysis.json[24]

+          "rawSql": "SELECT\n  f.attribute_name,\n  COUNT(*) AS failure_count\nFROM _tool_agentready_findings f\nJOIN project_mapping pm ON f.repo_id = pm.row_id AND pm.`table` = 'repos'\nWHERE f.status = 'fail' AND pm.project_name in (${project})\nGROUP BY f.attribute_id, f.attribute_name\nORDER BY failure_count DESC\nLIMIT 15",

Relevance

⭐⭐⭐ High
PR #86 documents strict owned-plugin boundaries; changes outside allowed dirs are treated as
non-compliant.
PR-#86
PR-#96

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 984 restricts all changed file paths to the three owned plugin directories listed in
the rule. The diff shows modifications to three dashboard JSON files under
backend/plugins/agentready/grafana/, which is not an allowed directory prefix.
Rule 984: Restrict code changes to owned plugin directories only
backend/plugins/agentready/grafana/findings-analysis.json[24-130]
backend/plugins/agentready/grafana/fleet-overview.json[29-213]
backend/plugins/agentready/grafana/repo-detail.json[196-221]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The PR updates files under `backend/plugins/agentready/`, but compliance requires that changes be restricted to owned plugin directories only.

## Issue Context
Allowed directories per compliance are:
- `backend/plugins/aireview/`
- `backend/plugins/codecov/`
- `backend/plugins/testregistry/`

Any modifications outside these paths violate the rule.

## Fix Focus Areas
- backend/plugins/agentready/grafana/findings-analysis.json[24-130]
- backend/plugins/agentready/grafana/fleet-overview.json[29-213]
- backend/plugins/agentready/grafana/repo-detail.json[196-221]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. Project join inflates aggregates 🐞 Bug ≡ Correctness

Description

Several panel queries join project_mapping and then use COUNT(*)/AVG(...), which duplicates
rows when a repo appears in more than one of the selected projects (the project variable is
multi-select and defaults to All). This inflates counts (e.g., certification distribution) and
weights averages (e.g., avg score / pass rates) by “number of selected projects a repo belongs to”
instead of “per repo”.

Code

backend/plugins/agentready/grafana/fleet-overview.json[71]

+          "rawSql": "SELECT\n  a.certification_level AS metric,\n  COUNT(*) AS value\nFROM _tool_agentready_assessments a\nJOIN project_mapping pm ON a.repo_id = pm.row_id AND pm.`table` = 'repos'\nINNER JOIN (\n  SELECT aa.repo_id, MAX(aa.assessed_at) as max_assessed\n  FROM _tool_agentready_assessments aa\n  JOIN project_mapping pm2 ON aa.repo_id = pm2.row_id AND pm2.`table` = 'repos'\n  WHERE aa.id != '' AND pm2.project_name in (${project})\n  GROUP BY aa.repo_id\n) latest ON a.repo_id = latest.repo_id AND a.assessed_at = latest.max_assessed\nWHERE pm.project_name in (${project})\nGROUP BY a.certification_level",

Relevance

⭐⭐⭐ High
Team previously fixed inflated AgentReady dashboard aggregates via dedup before AVG (PR #87), so
will accept join-dup fix.
PR-#87

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The dashboard offers multi-project selection, and the schema allows the same repo (row_id) to belong
to multiple projects. With JOIN project_mapping ... pm.project_name in (${project}), a single
assessment/finding row can match multiple mapping rows, duplicating it before COUNT(*)/AVG(...)
aggregates are computed.
backend/plugins/agentready/grafana/fleet-overview.json[65-74]
backend/plugins/agentready/grafana/fleet-overview.json[186-213]
backend/plugins/agentready/grafana/findings-analysis.json[16-46]
backend/core/models/migrationscripts/archived/project_mapping.go[20-25]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Queries that `JOIN project_mapping` and then aggregate with `COUNT(*)` or `AVG(...)` will over-count/over-weight when the same repo is mapped to multiple selected projects. This is especially problematic because the `project` variable is multi-select and defaults to `All`.

### Issue Context
`project_mapping` is keyed by `(project_name, table, row_id)`, so the same `row_id` can legitimately map to multiple `project_name` values. Under `pm.project_name IN (${project})`, a single assessment/finding row can match multiple mapping rows and be duplicated.

### Fix
Prefer filtering via `EXISTS` (or a join to a `SELECT DISTINCT row_id ...` derived table) so each underlying fact row is counted once. For example:
- Replace `JOIN project_mapping pm ... WHERE pm.project_name in (${project})` with:
 `WHERE EXISTS (SELECT 1 FROM project_mapping pm WHERE pm.row_id = a.repo_id AND pm.\`table\`='repos' AND pm.project_name IN (${project}))`
- Where a join is necessary, join against a de-duplicated mapping subquery:
 `JOIN (SELECT DISTINCT row_id FROM project_mapping WHERE \`table\`='repos' AND project_name IN (${project})) pm ON pm.row_id = a.repo_id`

### Fix Focus Areas
- backend/plugins/agentready/grafana/fleet-overview.json[29-29]
- backend/plugins/agentready/grafana/fleet-overview.json[71-71]
- backend/plugins/agentready/grafana/findings-analysis.json[24-24]
- backend/plugins/agentready/grafana/findings-analysis.json[45-45]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. Project var wrong datasource 🐞 Bug ≡ Correctness

Description

The new project template variable is hard-coded to the Grafana datasource named mysql, while the
dashboards otherwise use the selectable ${DS_DEVLAKE} datasource. In environments where the
DevLake datasource is not literally named mysql, the project dropdown query fails and panels using
${project} won’t render correctly.

Code

backend/plugins/agentready/grafana/findings-analysis.json[R116-124]

+        "datasource": "mysql",
+        "definition": "select distinct name from projects",
+        "hide": 0,
+        "includeAll": true,
+        "label": "Project",
+        "multi": true,
+        "name": "project",
+        "options": [],
+        "query": "select distinct name from projects",

Relevance

⭐⭐ Medium
Agentready dashboards already assume mysql datasource (PR #85); no prior reviews enforcing DS
variable for templating queries.
PR-#85

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
AgentReady dashboards use a datasource selector (DS_DEVLAKE) for panels, but the new project
variable queries projects via a fixed datasource name (mysql). This mismatch breaks the project
dropdown (and therefore panel filtering) when the selected/installed DevLake datasource isn’t named
mysql.
backend/plugins/agentready/grafana/findings-analysis.json[21-46]
backend/plugins/agentready/grafana/findings-analysis.json[103-130]
backend/plugins/agentready/grafana/repo-detail.json[189-224]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The `project` template variable uses a hard-coded datasource (`"mysql"`) instead of the dashboard’s datasource variable (`${DS_DEVLAKE}`), causing the variable query (and dependent panels) to fail when the datasource name differs.

### Issue Context
These dashboards already define `DS_DEVLAKE` as a datasource template variable and use `${DS_DEVLAKE}` for panel queries (and for `repo_id` in repo-detail). The `project` variable should query `projects` from the same selected datasource.

### Fix
Update the `project` variable in all three dashboards to use `${DS_DEVLAKE}` as its datasource (consistent with panels / other variables).

### Fix Focus Areas
- backend/plugins/agentready/grafana/findings-analysis.json[110-129]
- backend/plugins/agentready/grafana/fleet-overview.json[192-212]
- backend/plugins/agentready/grafana/repo-detail.json[196-216]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-app-for-konflux-ci · 2026-06-04T13:48:54Z

      "targets": [
        {
-          "rawSql": "SELECT\n  f.attribute_name,\n  COUNT(*) AS failure_count\nFROM _tool_agentready_findings f\nWHERE f.status = 'fail'\nGROUP BY f.attribute_id, f.attribute_name\nORDER BY failure_count DESC\nLIMIT 15",
+          "rawSql": "SELECT\n  f.attribute_name,\n  COUNT(*) AS failure_count\nFROM _tool_agentready_findings f\nJOIN project_mapping pm ON f.repo_id = pm.row_id AND pm.`table` = 'repos'\nWHERE f.status = 'fail' AND pm.project_name in (${project})\nGROUP BY f.attribute_id, f.attribute_name\nORDER BY failure_count DESC\nLIMIT 15",


1. Changes under plugins/agentready 📘 Rule violation § Compliance

This PR modifies files under backend/plugins/agentready/, which is outside the allowed owned plugin directories for this change. This violates the restriction that only backend/plugins/aireview/, backend/plugins/codecov/, and backend/plugins/testregistry/ may be touched.

Agent Prompt

## Issue description The PR updates files under `backend/plugins/agentready/`, but compliance requires that changes be restricted to owned plugin directories only. ## Issue Context Allowed directories per compliance are: - `backend/plugins/aireview/` - `backend/plugins/codecov/` - `backend/plugins/testregistry/` Any modifications outside these paths violate the rule. ## Fix Focus Areas - backend/plugins/agentready/grafana/findings-analysis.json[24-130] - backend/plugins/agentready/grafana/fleet-overview.json[29-213] - backend/plugins/agentready/grafana/repo-detail.json[196-221]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

This PR modifies files under backend/plugins/agentready/, which is outside the allowed owned
plugin directories for this change.

Is there a rule I can add so the agentready plugin can be modified? 👀

@CryptoRodeo it looks like when we made this repo agentready lol.

Looks like we need to update the root AGENTS.md file. Qodo seems to be pulling the rules from there. And then probably need to add skills/AGENTs.md for this plugin as well

https://github.com/konflux-ci/devlake/pull/86/changes#top

That's it, I'll create a another PR to update that after, thanks!

qodo-app-for-konflux-ci · 2026-06-04T13:48:55Z

+        "datasource": "mysql",
+        "definition": "select distinct name from projects",
+        "hide": 0,
+        "includeAll": true,
+        "label": "Project",
+        "multi": true,
+        "name": "project",
+        "options": [],
+        "query": "select distinct name from projects",


2. Project var wrong datasource 🐞 Bug ≡ Correctness

The new project template variable is hard-coded to the Grafana datasource named mysql, while the dashboards otherwise use the selectable ${DS_DEVLAKE} datasource. In environments where the DevLake datasource is not literally named mysql, the project dropdown query fails and panels using ${project} won’t render correctly.

Agent Prompt

### Issue description The `project` template variable uses a hard-coded datasource (`"mysql"`) instead of the dashboard’s datasource variable (`${DS_DEVLAKE}`), causing the variable query (and dependent panels) to fail when the datasource name differs. ### Issue Context These dashboards already define `DS_DEVLAKE` as a datasource template variable and use `${DS_DEVLAKE}` for panel queries (and for `repo_id` in repo-detail). The `project` variable should query `projects` from the same selected datasource. ### Fix Update the `project` variable in all three dashboards to use `${DS_DEVLAKE}` as its datasource (consistent with panels / other variables). ### Fix Focus Areas - backend/plugins/agentready/grafana/findings-analysis.json[110-129] - backend/plugins/agentready/grafana/fleet-overview.json[192-212] - backend/plugins/agentready/grafana/repo-detail.json[196-216]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Updated, switched to a format that follows the other plugins.

qodo-app-for-konflux-ci · 2026-06-04T13:48:55Z

      "targets": [
        {
-          "rawSql": "SELECT\n  a.certification_level AS metric,\n  COUNT(*) AS value\nFROM _tool_agentready_assessments a\nINNER JOIN (\n  SELECT repo_id, MAX(assessed_at) as max_assessed\n  FROM _tool_agentready_assessments WHERE id != ''\n  GROUP BY repo_id\n) latest ON a.repo_id = latest.repo_id AND a.assessed_at = latest.max_assessed\nGROUP BY a.certification_level",
+          "rawSql": "SELECT\n  a.certification_level AS metric,\n  COUNT(*) AS value\nFROM _tool_agentready_assessments a\nJOIN project_mapping pm ON a.repo_id = pm.row_id AND pm.`table` = 'repos'\nINNER JOIN (\n  SELECT aa.repo_id, MAX(aa.assessed_at) as max_assessed\n  FROM _tool_agentready_assessments aa\n  JOIN project_mapping pm2 ON aa.repo_id = pm2.row_id AND pm2.`table` = 'repos'\n  WHERE aa.id != '' AND pm2.project_name in (${project})\n  GROUP BY aa.repo_id\n) latest ON a.repo_id = latest.repo_id AND a.assessed_at = latest.max_assessed\nWHERE pm.project_name in (${project})\nGROUP BY a.certification_level",


3. Project join inflates aggregates 🐞 Bug ≡ Correctness

Several panel queries join project_mapping and then use COUNT(*)/AVG(...), which duplicates rows when a repo appears in more than one of the selected projects (the project variable is multi-select and defaults to All). This inflates counts (e.g., certification distribution) and weights averages (e.g., avg score / pass rates) by “number of selected projects a repo belongs to” instead of “per repo”.

Agent Prompt

### Issue description Queries that `JOIN project_mapping` and then aggregate with `COUNT(*)` or `AVG(...)` will over-count/over-weight when the same repo is mapped to multiple selected projects. This is especially problematic because the `project` variable is multi-select and defaults to `All`. ### Issue Context `project_mapping` is keyed by `(project_name, table, row_id)`, so the same `row_id` can legitimately map to multiple `project_name` values. Under `pm.project_name IN (${project})`, a single assessment/finding row can match multiple mapping rows and be duplicated. ### Fix Prefer filtering via `EXISTS` (or a join to a `SELECT DISTINCT row_id ...` derived table) so each underlying fact row is counted once. For example: - Replace `JOIN project_mapping pm ... WHERE pm.project_name in (${project})` with: `WHERE EXISTS (SELECT 1 FROM project_mapping pm WHERE pm.row_id = a.repo_id AND pm.\`table\`='repos' AND pm.project_name IN (${project}))` - Where a join is necessary, join against a de-duplicated mapping subquery: `JOIN (SELECT DISTINCT row_id FROM project_mapping WHERE \`table\`='repos' AND project_name IN (${project})) pm ON pm.row_id = a.repo_id` ### Fix Focus Areas - backend/plugins/agentready/grafana/fleet-overview.json[29-29] - backend/plugins/agentready/grafana/fleet-overview.json[71-71] - backend/plugins/agentready/grafana/findings-analysis.json[24-24] - backend/plugins/agentready/grafana/findings-analysis.json[45-45]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Updated, fixed here: 428201b

…ource convention Replace non-standard __inputs + DS_DEVLAKE template variable pattern with direct "mysql" datasource references, matching all other DevLake dashboards. The DS_DEVLAKE indirection added no value since DevLake provisions the datasource as "mysql" via grafana/provisioning Signed-off-by: Bryan Ramos <bramos@redhat.com>

… multi-project repos Panels in fleet-overview and findings-analysis joined project_mapping in the outer query, duplicating rows when a repo belonged to more than one selected project. This inflated AVG scores, COUNT totals, and produced duplicate table rows. Fleet-overview fix: remove redundant outer pm JOIN, the "latest assessment" subquery already filters by project. Findings-analysis fix: replace JOIN with EXISTS so project filtering does not multiply rows. Signed-off-by: Bryan Ramos <bramos@redhat.com>

CryptoRodeo · 2026-06-05T22:21:47Z

Closing, all work moving to #97

CryptoRodeo mentioned this pull request Jun 2, 2026

chore: update agentready dashboards konflux-ci/konflux-devlake-dashboards#29

Draft

CryptoRodeo mentioned this pull request Jun 3, 2026

feat(agentready): centralized assessment repo onboarding #95

Closed

4 tasks

qodo-app-for-konflux-ci Bot reviewed Jun 4, 2026

View reviewed changes

CryptoRodeo added 2 commits June 4, 2026 11:34

CryptoRodeo mentioned this pull request Jun 5, 2026

feat(agentready): source assessment data from central repo #97

Open

4 tasks

CryptoRodeo closed this Jun 5, 2026

Conversation

CryptoRodeo commented Jun 2, 2026

⚠️ Pre Checklist

Summary

Does this close any open issues?

Screenshots

Other Information

Uh oh!

Dannyb48 commented Jun 2, 2026

Uh oh!

gbenhaim commented Jun 3, 2026

Uh oh!

gbenhaim commented Jun 3, 2026

Uh oh!

qodo-app-for-konflux-ci Bot commented Jun 3, 2026

Code Review by Qodo

Sorry, something went wrong

Uh oh!

Dannyb48 commented Jun 4, 2026

Uh oh!

qodo-app-for-konflux-ci Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

qodo-app-for-konflux-ci Bot Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

CryptoRodeo Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

Dannyb48 Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

CryptoRodeo Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qodo-app-for-konflux-ci Bot Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

CryptoRodeo Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

qodo-app-for-konflux-ci Bot Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

CryptoRodeo Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

CryptoRodeo commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

qodo-app-for-konflux-ci Bot commented Jun 4, 2026 •

edited

Loading

CryptoRodeo Jun 4, 2026 •

edited

Loading