feat(graph): support multi-stamp execution graphs#250
Open
geoberle wants to merge 3 commits into
Open
Conversation
ea75893 to
9a6cf99
Compare
There was a problem hiding this comment.
Pull request overview
This PR extends the pipelines graph tooling to represent and execute multi-stamp topologies by duplicating stamped services per stamp while deduplicating unstamped nodes. It also changes Service.Stamped to *bool to preserve explicit user intent (stamped: false) and introduces per-stamp keying in graph identifiers/metadata.
Changes:
- Changed
topology.Service.Stampedfromboolto*bool, updated stamping propagation, and added validation for explicit opt-outs under stamped parents. - Added stamp-aware graph identity (
Identifier.Stamp) plus composite keys for steps/resource groups, and introducedForStampedEntrypoint+mergeStampedto merge per-stamp graphs. - Added
ConfigResolver.GetRegionStampConfigurationto re-resolve configuration templates using a different stamp value.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pipelines/topology/types.go | Switches Stamped to pointer semantics and updates propagation/validation logic. |
| pipelines/topology/types_test.go | Adds/updates tests for pointer-based stamped propagation and validation cases. |
| pipelines/graph/graph.go | Introduces Stamp in identifiers and refactors graph lookups to use composite keys; adds ForStampedEntrypoint. |
| pipelines/graph/merge.go | Adds per-stamp graph merge implementation to combine per-stamp graphs into one execution graph. |
| pipelines/graph/merge_test.go | Adds tests for per-stamp merge behavior (node duplication, rewiring, and RG/step selection). |
| config/config.go | Adds API to resolve config for a region using an alternate stamp and stores resolver inputs for re-resolution. |
| config/config_test.go | Adds test coverage for stamp-specific config resolution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
9a6cf99 to
5ad5152
Compare
5ad5152 to
cb0be5e
Compare
cb0be5e to
fc1c8c7
Compare
12 tasks
fc1c8c7 to
5f52dbb
Compare
3e08b9f to
1d214ad
Compare
1d214ad to
e63e2cb
Compare
ebc2d8e to
1a0d317
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
config/config.go:410
- ValueProvenance() now accepts a
stampargument and GetRegionConfiguration() can return stamp-templated values (e.g. defaults.managementClusterRG uses.ctx.stamp). However, the provenance layers (default/cloud/environment/region) are still read from cr.cfg (the resolver’s initially-templated config), so the returned provenance can be internally inconsistent:Resultreflects the requested stamp but earlier layers may reflect the resolver’s original stamp/region templating.
func (cr *configResolver) ValueProvenance(region, stamp, path string, regionShort ...string) (*Provenance, error) {
cloudCfg, hasCloud := cr.cfg.Overrides[cr.cloud]
if !hasCloud {
return nil, fmt.Errorf("the cloud %s is not found in the config", cr.cloud)
}
envCfg, hasEnv := cloudCfg.Overrides[cr.environment]
if !hasEnv {
return nil, fmt.Errorf("the deployment env %s is not found under cloud %s", cr.environment, cr.cloud)
}
regionCfg, hasRegion := envCfg.Overrides[region]
if !hasRegion {
// a missing region just means we use default values
regionCfg = types.Configuration{}
}
mergedCfg, err := cr.GetRegionConfiguration(region, stamp, regionShort...)
if err != nil {
return nil, err
}
p := &Provenance{}
for name, part := range map[string]struct {
from *types.Configuration
value *any
set *bool
}{
"default": {from: &cr.cfg.Defaults, value: &p.Default, set: &p.DefaultSet},
"cloud": {from: &cloudCfg.Defaults, value: &p.Cloud, set: &p.CloudSet},
"environment": {from: &envCfg.Defaults, value: &p.Environment, set: &p.EnvironmentSet},
"region": {from: ®ionCfg, value: &p.Region, set: &p.RegionSet},
"result": {from: &mergedCfg, value: &p.Result, set: &p.ResultSet},
1a0d317 to
8b5f59a
Compare
8b5f59a to
2933842
Compare
2933842 to
f60a66d
Compare
f60a66d to
58826b5
Compare
58826b5 to
79b6037
Compare
79b6037 to
f8b5e32
Compare
f8b5e32 to
ba9356b
Compare
Stamp-aware graph construction: when a service is marked as stamped in the
topology, accumulate() expands it once per stamp during a single tree walk.
Unstamped services are processed once. No post-hoc merge pass needed.
Key changes:
- ForEntrypoints accepts stamp-keyed pipelines (map[string]map[string]*Pipeline).
Non-stamped callers use ForEntrypoint which wraps pipelines as {"": pipelines}.
ForStampedEntrypoint is removed — callers pass stamp pipelines directly.
- nodesFor bakes stamp into all Identifiers at creation (nodes, edges, steps,
validation steps, resource group keys).
- Leaf-to-root wiring between services is stamp-scoped: stamped parent leaves
connect only to same-stamp child roots. Unstamped parent leaves fan out to
all stamps.
- External dependency resolution inherits stamp from the declaring node when
the target service is stamped; unstamped targets keep empty stamp.
- StepKey removed — Identifier used as map key for Steps.
- GetResourceGroup narrowed to accept ResourceGroupKey instead of Identifier.
- ConfigResolver.GetRegionStampConfiguration resolves config with stamp dimension.
- Service.Stamped changed to *bool — explicit false preserved and validated,
distinguishing "unset" from "intentionally not stamped".
Breaking changes on Graph, ForEntrypoints, and ForStampedEntrypoint (removed).
…nctions Split ForEntrypoints into ForEntrypoints (unstamped, flat pipelines map) and ForStampedEntrypoints (stamp-expanded, stamp→pipelines map). Callers that don't use stamps no longer see the map[string]map[string]*Pipeline type. ForEntrypoint remains as a single-entrypoint convenience wrapper. Extract resolveIterations, accumulateIteration, registerResourceGroups, and wireInterServiceEdges from the monolithic accumulate method. Add stamp and regionShort parameters to GetRegionConfiguration and ValueProvenance so they re-template per-stamp instead of using the resolver's initial stamp. Extract resolveOverrides to share the preprocessing logic.
Signed-off-by: Gerd Oberlechner <goberlec@redhat.com>
7b31b0b to
961e693
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stamp-aware graph construction: when a service is marked as stamped in the topology,
accumulate()expands it once per stamp during a single tree walk. Unstamped services are processed once. No post-hoc merge pass needed.Service.Stampedchanged to*bool— explicitfalseis preserved and validated, distinguishing "unset" from "intentionally not stamped".What changed
map[string]*Pipeline— stamped services appear once with an empty Stamp, suitable for runtimes that handle stamp expansion themselves (e.g. EV2).map[string]map[string]*Pipeline) and expands stamped services once per stamp in the graph, suitable for runtimes where the graph drives per-stamp execution (e.g. templatize).Identifierused as map key forSteps.ResourceGroupKeyinstead ofIdentifier.Breaking changes
Breaking changes on
Graph,ForEntrypoints(signature changed), andForStampedEntrypoint(replaced byForStampedEntrypoints).Companion PRs to adapt: