feat: add cortex evalution kpis - #1057
Draft
SoWieMarkus wants to merge 13 commits into
Draft
Conversation
Signed-off-by: Markus Wieland <44964229+SoWieMarkus@users.noreply.github.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the infrastructure KPI plugin to enrich KVM host metrics with additional host metadata (cluster/building block) and introduces an additional metric for “physical” (non-overcommitted) host capacity to support evaluation KPIs.
Changes:
- Add
compute_clusteras an additional label for KVM host metrics. - Derive
building_block(and cluster) from Hypervisor CR labels instead of parsing the host name. - Add a new Prometheus metric for per-host physical capacity and related helper logic.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| internal/knowledge/kpis/plugins/infrastructure/shared.go | Adds compute_cluster label and switches building block / cluster derivation to Hypervisor object labels; adds a physical-capacity helper. |
| internal/knowledge/kpis/plugins/infrastructure/shared_test.go | Updates unit tests to match the new KVM host label order and new label sources. |
| internal/knowledge/kpis/plugins/infrastructure/kvm_host_capacity.go | Introduces cortex_kvm_host_physical_capacity_total and emits physical-capacity samples during collection. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (2)
internal/knowledge/kpis/plugins/infrastructure/kvm_host_capacity.go:184
- Collect() currently skips the entire hypervisor when physical capacity is missing (Status.Capacity nil/zero). That changes existing behavior: hosts that have EffectiveCapacity set (so total/usage metrics are valid) but no physical Capacity will now emit no capacity metrics at all. Prefer emitting the existing capacity metrics and only omitting the new physical-capacity metric when physical capacity is unavailable.
if !hasCPUPhysical || !hasRAMPhysical {
slog.Warn("hypervisor missing physical cpu or ram capacity, skipping", "host", hypervisor.Name)
continue
}
internal/knowledge/kpis/plugins/infrastructure/kvm_host_capacity.go:49
- A new metric descriptor (cortex_kvm_host_physical_capacity_total) was added and Collect() now emits additional metrics, but the existing KPI tests (kvm_host_capacity_test.go) build an exact expected metric set/count. Update those tests to include the new physical-capacity metrics (and any label changes) so the suite continues to validate the full exported surface.
k.totalPhysicalCapacityPerHost = prometheus.NewDesc(
"cortex_kvm_host_physical_capacity_total",
"Total physical resource capacity on the KVM hosts (individually by host, ignoring overcommit factor). CPU in vCPUs, memory in bytes.",
append(kvmHostLabels, "resource"),
nil,
)
Signed-off-by: Markus Wieland <markus.wieland@sap.com>
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The new physical-capacity requirement currently causes the KVM host capacity collector to skip emitting existing capacity/usage metrics for hosts that lack Status.Capacity but still provide EffectiveCapacity.
Review details
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
internal/knowledge/kpis/plugins/infrastructure/shared.go:160
- Now that
getPhysicalCapacityexists,getResourceCapacitycan delegate its physical-capacity fallback to this helper so the nil/map-key/zero checks stay consistent in one place.
internal/knowledge/kpis/plugins/infrastructure/kvm_host_capacity.go:183
- The collector currently skips the entire hypervisor when
Status.Capacity(physical capacity) is missing. That makescortex_kvm_host_capacity_total/cortex_kvm_host_capacity_usagedisappear for hosts that still exposeEffectiveCapacity, even though only the new physical metric is unavailable.
ramPhysical, hasRAMPhysical := hypervisor.getPhysicalCapacity(hv1.ResourceMemory)
if !hasCPUPhysical || !hasRAMPhysical {
slog.Warn("hypervisor missing physical cpu or ram capacity, skipping", "host", hypervisor.Name)
continue
- Files reviewed: 6/6 changed files
- Comments generated: 0 new
- Review effort level: Lite
Signed-off-by: Markus Wieland <markus.wieland@sap.com>
Signed-off-by: Markus Wieland <markus.wieland@sap.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The KVM host capacity collector currently drops existing metrics entirely when physical capacity is missing, which can cause unexpected metric disappearance in environments where only effective capacity is populated.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
internal/knowledge/kpis/plugins/infrastructure/kvm_host_capacity.go:182
- The collector currently skips the entire hypervisor (including existing total/usage metrics) when physical capacity fields are missing. This makes the new physical-capacity metric a hard requirement and can cause established metrics like cortex_kvm_host_capacity_total / _usage to disappear for hosts where Status.Capacity isn't populated (even if EffectiveCapacity is). Prefer emitting physical metrics only when available, without skipping the host entirely.
cpuPhysical, hasCPUPhysical := hypervisor.getPhysicalCapacity(hv1.ResourceCPU)
ramPhysical, hasRAMPhysical := hypervisor.getPhysicalCapacity(hv1.ResourceMemory)
if !hasCPUPhysical || !hasRAMPhysical {
slog.Warn("hypervisor missing physical cpu or ram capacity, skipping", "host", hypervisor.Name)
- Files reviewed: 9/9 changed files
- Comments generated: 1
- Review effort level: Lite
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Markus Wieland <wie.markus@web.de>
Contributor
Test Coverage ReportTest Coverage 📊: 73.1% |
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
KVM host capacity collection can now suppress existing capacity/usage metrics when physical capacity is missing, and the label-source changes risk degrading label fidelity without a compatibility/migration strategy.
Review details
Suppressed comments (3)
Previously missed (2) — in code that hasn't changed since the last review.
internal/knowledge/kpis/plugins/infrastructure/shared.go:77
- Adding
compute_clusterto the commonkvmHostLabelschanges the label set for multiple existing KVM metrics (e.g. capacity, project utilization, hana stacking). Downstream PromQL that aggregatesby (...)may need to includecompute_clusterto avoid accidentally mixing series across clusters ifcompute_hostis not globally unique.
internal/knowledge/kpis/plugins/infrastructure/shared.go:105 - Building block is now sourced only from the Hypervisor label
kubernetes.metal.cloud.sap/bb. If existing clusters still have Hypervisor objects without this label,building_blockwill becomeunknowneven when the name encodes it (previously parsed), which can break dashboards/alerts that filter by building_block.
internal/knowledge/kpis/plugins/infrastructure/kvm_host_capacity.go:184
Collectnow skips the entire hypervisor when physical capacity is missing. This also suppresses the existingcortex_kvm_host_capacity_totalandcortex_kvm_host_capacity_usagemetrics, even whenEffectiveCapacityis present and those totals/usages could still be reported. Consider emitting total/usage metrics regardless, and only conditionally emitting the new physical-capacity metric whenStatus.Capacityis available.
if !hasCPUPhysical || !hasRAMPhysical {
slog.Warn("hypervisor missing physical cpu or ram capacity, skipping", "host", hypervisor.Name)
continue
}
- Files reviewed: 9/9 changed files
- Comments generated: 0 new
- Review effort level: Lite
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First I wanted to create separate metrics, but the infrastructure metrics are literally what we already need. So I added the missing labels / metrics to the infrastructure package (which also fits in my opinion)
compute_clusterlabel to kvm host metricsbblabel of hypervisor crd instead of manually extracting the bb from the nameWIP