Let process steps run once per scenario (#252)

[WestonVoglesonger]

Closes #252.

What changed

A process step can now name an entry in the top-level scenarios: section and runs once per scenario tuple, with that tuple's values substituted into its configuration:

scenarios:
  sweep:
    module: grid
    parameters:
      beta: [0.1, 0.2]

process:
  fetch:
    module: shell
    command: ./fetch_inputs.sh
  analyze:
    module: shell
    depends: [fetch]
    scenario: sweep
    command: Rscript
    args: ["analyze.R", "--beta", "{beta}"]

flepimop2 process -t analyze runs fetch once, then analyze twice. Verified end to end through the CLI, not just at unit level.

This follows the existing precedent rather than inventing one: SimulateSpecificationModel already has a scenario field naming a key in scenarios:, and SimulateCommand already loops scenario_config.scenarios(). Process steps now do the same thing.

Three decisions worth review

1. Fan-out happens inside a plan step, not by multiplying plan nodes. depends names steps, so if a parameterized step became N nodes, a downstream step would need to say which scenario it depends on, which means per-scenario dependency edges and a much larger change to the #323 contract. Keeping it one node means a dependent runs after all of its dependency's scenarios, which is also the intuitive reading. #323's DAG is untouched, and there's a test pinning that.

2. Substitution is a plain replacement of the scenario's own names, not str.format. Process steps are usually shell commands, and awk '{print $1}' is ordinary text there. Going through str.format would raise KeyError on it. Only names the scenario defines are replaced; every other brace survives. Test included.

3. A placeholder always resolves to text. I initially preserved the raw type when a value was exactly one placeholder, which reads nicer (count: "{n}" staying an int). It's wrong: pydantic rejects an int inside a list[str], which is precisely ShellProcess.args, the most common field this will touch. It coerces the other direction happily ("3" into an int field is an int), so text substitution is strictly safer and loses nothing. There's a test that builds the expanded configs into real ShellProcess instances so this cannot regress into a runtime failure.

module, depends, and scenario describe the step rather than its work, so they are never rewritten.

Validation

Scenario references are checked across the whole section before any step runs, collecting every bad name at once, matching depends. Confirmed that a typo aborts before the dependency executes rather than partway through the pipeline.

Verification

• 20 new tests; tests/process + tests/cli at 126 passing.
• ruff format, ruff check, mypy (139 files), and doctests all clean.
• Full suite: 538 passed. The 85 failures there are identical on main (85 failed / 498 passed) and are a local environment issue, not from this change.
• Documented in docs/guides/scenarios.md, which already ended with a process step consuming scenario output, so the per-scenario shape belongs alongside it.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 539186cd70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Substitute placeholders in one pass

When a string-valued scenario parameter contains another valid placeholder token, this loop substitutes that newly introduced token during a later iteration. For example, values {"first": "{second}", "second": "VALUE"} turn "{first}" into "VALUE" rather than preserving the actual first value, and reversing parameter declaration order changes the result. Match placeholders from the original string and replace them simultaneously so scenario values are not recursively reinterpreted.

Useful? React with 👍 / 👎.

[pearsonca]

one big picture note to try to address: the process and scenario link should be specified by a tag on scenario, rather than on process, so that the same process can be used for multiple scenarios.