Skip to content

Prototype: demand-driven generics without map-time forward instantiation - #238

Closed
SimonHeybrock wants to merge 4 commits into
235-pep695-single-model-prototypefrom
235-deferred-mapped-labels
Closed

Prototype: demand-driven generics without map-time forward instantiation#238
SimonHeybrock wants to merge 4 commits into
235-pep695-single-model-prototypefrom
235-deferred-mapped-labels

Conversation

@SimonHeybrock

Copy link
Copy Markdown
Member

Q1 spike from the design document (docs/developer/architecture-and-design/demand-driven-generics.md): remove the map-time forward-instantiation hook by relying on cyclebane deriving mapped-node labeling at task-graph build time. Depends on scipp/cyclebane#32 — CI will be red until that lands and is released. Based on the #237 branch; draft for discussion.

What this shows

With cyclebane keeping plain node names and deriving which nodes carry which indices at to_networkx() time, map() commutes with provider insertion: generic providers instantiated on demand after mapping receive their indices from the derivation. Consequences here:

  • The map() hook and the entire seed-restriction machinery are gone; forward chaining survives only behind output_keys(), whose fate is Q3 (rule-graph inspection).
  • Mapped roots count as satisfied keys, so backward instantiation never tries to provide them.
  • reduce() with an explicit key treats the key as a demand and instantiates providers for it. Without a key, the reduced sink is resolved among nodes actually present, so reducing a rule-derived sink requires an explicit key — two tests updated accordingly. This is the user-visible rule to review.
  • get_mapped_node_names reads derived indices via cyclebane's new named_indices accessor instead of scanning for MappedNode labels, and loses the multiple-candidates ambiguity handling (plain names are unique).

All 245 tests pass (against the cyclebane branch); notably, before this change, the whole suite already passed against the cyclebane branch without any sciline modification, i.e. the cyclebane change is transparent to current sciline.

Discussion points

  • The reduce-key rule above: acceptable, or should sciline defer sink resolution further?
  • The shadowing seams (reduce results and grafted branches reusing a mapped node's name) keep a scoped MappedNode alias inside cyclebane — see the discussion in Derive mapped-node labeling at compile time cyclebane#32.

🤖 Generated with Claude Code

SimonHeybrock and others added 3 commits August 21, 2026 08:31
With cyclebane deriving mapped-node labeling at task-graph build time
(scipp/cyclebane#32), dependents of mapped nodes no longer need to
exist when map() is called: providers instantiated on demand after
mapping receive their indices from the derivation. Remove the
forward-instantiation hook from map() and the seed-restriction
machinery; forward chaining remains only behind output_keys().

Mapped roots count as satisfied keys, so backward instantiation never
provides them. reduce() with an explicit key treats that key as a
demand and instantiates providers for it; without a key, only nodes
present in the graph determine the reduced sink, so reducing a
rule-derived sink requires an explicit key (tests updated accordingly).

get_mapped_node_names uses the new cyclebane named_indices accessor
instead of scanning for MappedNode labels.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Q3 decision: output_keys() returns concrete graph sinks plus the
return-type patterns of generic providers not consumed by other generic
providers (approximated by pattern origin); no-argument visualize()
shows the concrete part of the graph, since patterns cannot be demanded.
This removes _instantiate_forward and forward_bindings; the engine is
backward-only.

With cyclebane forbidding reduce results and grafted branches from
shadowing mapped nodes, get_mapped_node_names loses the
multiple-candidates disambiguation; index_names= is now pure validation.
Tests using the shadowing idioms are rewritten with distinct names.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@SimonHeybrock

Copy link
Copy Markdown
Member Author

Follow-up per discussion (4cbf9fd): forward chaining is now fully deleted (_instantiate_forward, forward_bindings) — the engine is backward-only. output_keys() returns concrete sinks plus return-type patterns of generic providers not consumed by other generic providers; no-arg visualize() shows the concrete part of the graph. With scipp/cyclebane#32 forbidding the shadowing idioms, get_mapped_node_names loses the multiple-candidates disambiguation (index_names= is pure validation now) and the affected tests use distinct names for a mapped node and its reduction. Decisions recorded in the design document.

@SimonHeybrock

Copy link
Copy Markdown
Member Author

Merged into #237 (fast-forward, commits preserved), which is now the consolidated end-state proposal against main. See the design document there for the decision log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant