Skip to content

feat(graph): support multi-stamp execution graphs#250

Open
geoberle wants to merge 3 commits into
mainfrom
stamp-validation
Open

feat(graph): support multi-stamp execution graphs#250
geoberle wants to merge 3 commits into
mainfrom
stamp-validation

Conversation

@geoberle

@geoberle geoberle commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Stamp-aware graph construction: when a service is marked as stamped in the topology, accumulate() expands it once per stamp during a single tree walk. Unstamped services are processed once. No post-hoc merge pass needed.

Service.Stamped changed to *bool — explicit false is preserved and validated, distinguishing "unset" from "intentionally not stamped".

What changed

  • ForEntrypoint / ForEntrypoints accept flat map[string]*Pipeline — stamped services appear once with an empty Stamp, suitable for runtimes that handle stamp expansion themselves (e.g. EV2).
  • ForStampedEntrypoints accepts stamp-keyed pipelines (map[string]map[string]*Pipeline) and expands stamped services once per stamp in the graph, suitable for runtimes where the graph drives per-stamp execution (e.g. templatize).
  • nodesFor bakes stamp into all Identifiers at creation (nodes, edges, steps, validation steps, resource group keys).
  • Leaf-to-root wiring between services is stamp-scoped: stamped parent leaves connect only to same-stamp child roots. Unstamped parent leaves fan out to all stamps.
  • External dependency resolution inherits stamp from the declaring node when the target service is stamped; unstamped targets keep empty stamp.
  • StepKey removedIdentifier used as map key for Steps.
  • GetResourceGroup narrowed to accept ResourceGroupKey instead of Identifier.
  • ConfigResolver.GetRegionConfiguration and ValueProvenance accept stamp and optional regionShort parameters for per-stamp re-templating.

Breaking changes

Breaking changes on Graph, ForEntrypoints (signature changed), and ForStampedEntrypoint (replaced by ForStampedEntrypoints).
Companion PRs to adapt:

Copilot AI review requested due to automatic review settings June 17, 2026 11:58

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the pipelines graph tooling to represent and execute multi-stamp topologies by duplicating stamped services per stamp while deduplicating unstamped nodes. It also changes Service.Stamped to *bool to preserve explicit user intent (stamped: false) and introduces per-stamp keying in graph identifiers/metadata.

Changes:

  • Changed topology.Service.Stamped from bool to *bool, updated stamping propagation, and added validation for explicit opt-outs under stamped parents.
  • Added stamp-aware graph identity (Identifier.Stamp) plus composite keys for steps/resource groups, and introduced ForStampedEntrypoint + mergeStamped to merge per-stamp graphs.
  • Added ConfigResolver.GetRegionStampConfiguration to re-resolve configuration templates using a different stamp value.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pipelines/topology/types.go Switches Stamped to pointer semantics and updates propagation/validation logic.
pipelines/topology/types_test.go Adds/updates tests for pointer-based stamped propagation and validation cases.
pipelines/graph/graph.go Introduces Stamp in identifiers and refactors graph lookups to use composite keys; adds ForStampedEntrypoint.
pipelines/graph/merge.go Adds per-stamp graph merge implementation to combine per-stamp graphs into one execution graph.
pipelines/graph/merge_test.go Adds tests for per-stamp merge behavior (node duplication, rewiring, and RG/step selection).
config/config.go Adds API to resolve config for a region using an alternate stamp and stores resolver inputs for re-resolution.
config/config_test.go Adds test coverage for stamp-specific config resolution.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pipelines/topology/types.go
Comment thread pipelines/graph/merge.go Outdated
Comment thread pipelines/graph/merge.go Outdated
Copilot AI review requested due to automatic review settings June 17, 2026 13:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Comment thread pipelines/graph/graph.go Outdated
Comment thread pipelines/graph/graph.go
Comment thread pipelines/graph/graph.go
Copilot AI review requested due to automatic review settings June 17, 2026 16:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Comment thread pipelines/topology/types.go
Comment thread pipelines/graph/graph.go Outdated
Comment thread pipelines/graph/graph.go
Comment thread pipelines/graph/graph.go Outdated
Comment thread pipelines/graph/graph.go Outdated
Comment thread config/config.go Outdated
Comment thread pipelines/graph/graph.go Outdated
Comment thread pipelines/graph/graph.go Outdated
Comment thread pipelines/graph/merge.go Outdated
Copilot AI review requested due to automatic review settings June 18, 2026 06:08

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings June 18, 2026 09:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comment thread config/config.go Outdated
Copilot AI review requested due to automatic review settings June 18, 2026 21:24
Copilot AI review requested due to automatic review settings June 19, 2026 14:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

config/config.go:410

  • ValueProvenance() now accepts a stamp argument and GetRegionConfiguration() can return stamp-templated values (e.g. defaults.managementClusterRG uses .ctx.stamp). However, the provenance layers (default/cloud/environment/region) are still read from cr.cfg (the resolver’s initially-templated config), so the returned provenance can be internally inconsistent: Result reflects the requested stamp but earlier layers may reflect the resolver’s original stamp/region templating.
func (cr *configResolver) ValueProvenance(region, stamp, path string, regionShort ...string) (*Provenance, error) {
	cloudCfg, hasCloud := cr.cfg.Overrides[cr.cloud]
	if !hasCloud {
		return nil, fmt.Errorf("the cloud %s is not found in the config", cr.cloud)
	}
	envCfg, hasEnv := cloudCfg.Overrides[cr.environment]
	if !hasEnv {
		return nil, fmt.Errorf("the deployment env %s is not found under cloud %s", cr.environment, cr.cloud)
	}
	regionCfg, hasRegion := envCfg.Overrides[region]
	if !hasRegion {
		// a missing region just means we use default values
		regionCfg = types.Configuration{}
	}

	mergedCfg, err := cr.GetRegionConfiguration(region, stamp, regionShort...)
	if err != nil {
		return nil, err
	}

	p := &Provenance{}
	for name, part := range map[string]struct {
		from  *types.Configuration
		value *any
		set   *bool
	}{
		"default":     {from: &cr.cfg.Defaults, value: &p.Default, set: &p.DefaultSet},
		"cloud":       {from: &cloudCfg.Defaults, value: &p.Cloud, set: &p.CloudSet},
		"environment": {from: &envCfg.Defaults, value: &p.Environment, set: &p.EnvironmentSet},
		"region":      {from: &regionCfg, value: &p.Region, set: &p.RegionSet},
		"result":      {from: &mergedCfg, value: &p.Result, set: &p.ResultSet},

Comment thread config/config.go Outdated
Comment thread pipelines/graph/graph.go Outdated
Copilot AI review requested due to automatic review settings June 19, 2026 14:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Comment thread pipelines/graph/graph.go
Comment thread pipelines/graph/graph.go
Comment thread config/config.go Outdated
Copilot AI review requested due to automatic review settings June 19, 2026 19:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

Comment thread pipelines/graph/graph.go
Copilot AI review requested due to automatic review settings June 19, 2026 19:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Comment thread pipelines/graph/graph.go
Comment thread config/config_test.go Outdated
Comment thread config/ev2config/config.go Outdated
Comment thread config/config.go Outdated
Comment thread config/config.go Outdated
Copilot AI review requested due to automatic review settings June 24, 2026 13:34
geoberle added 2 commits June 24, 2026 15:34
Stamp-aware graph construction: when a service is marked as stamped in the
topology, accumulate() expands it once per stamp during a single tree walk.
Unstamped services are processed once. No post-hoc merge pass needed.

Key changes:

- ForEntrypoints accepts stamp-keyed pipelines (map[string]map[string]*Pipeline).
  Non-stamped callers use ForEntrypoint which wraps pipelines as {"": pipelines}.
  ForStampedEntrypoint is removed — callers pass stamp pipelines directly.
- nodesFor bakes stamp into all Identifiers at creation (nodes, edges, steps,
  validation steps, resource group keys).
- Leaf-to-root wiring between services is stamp-scoped: stamped parent leaves
  connect only to same-stamp child roots. Unstamped parent leaves fan out to
  all stamps.
- External dependency resolution inherits stamp from the declaring node when
  the target service is stamped; unstamped targets keep empty stamp.
- StepKey removed — Identifier used as map key for Steps.
- GetResourceGroup narrowed to accept ResourceGroupKey instead of Identifier.
- ConfigResolver.GetRegionStampConfiguration resolves config with stamp dimension.
- Service.Stamped changed to *bool — explicit false preserved and validated,
  distinguishing "unset" from "intentionally not stamped".

Breaking changes on Graph, ForEntrypoints, and ForStampedEntrypoint (removed).
…nctions

Split ForEntrypoints into ForEntrypoints (unstamped, flat pipelines map)
and ForStampedEntrypoints (stamp-expanded, stamp→pipelines map). Callers
that don't use stamps no longer see the map[string]map[string]*Pipeline
type. ForEntrypoint remains as a single-entrypoint convenience wrapper.

Extract resolveIterations, accumulateIteration, registerResourceGroups,
and wireInterServiceEdges from the monolithic accumulate method.

Add stamp and regionShort parameters to GetRegionConfiguration and
ValueProvenance so they re-template per-stamp instead of using the
resolver's initial stamp. Extract resolveOverrides to share the
preprocessing logic.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread pipelines/graph/graph.go
Comment thread pipelines/graph/graph.go
Signed-off-by: Gerd Oberlechner <goberlec@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants