PRD: GraphNAMLSS concept demonstrator and METR-LA merit probe

[RMKruse]

Problem Statement

The current repo has moved beyond a paper plan: it contains a self-contained tracer-bullet package with config loading, traffic data contracts, likelihoods, metrics, model components, training artifacts, baselines, and report generation. However, it is not yet shaped for the user's immediate goal: showing colleagues the GraphNAMLSS idea clearly enough to get useful feedback, and then deciding whether the approach has enough empirical merit to justify deeper research investment.

The problem is that the repo has the pieces, but not yet the Guided Demo Path. A Methodologically Sympathetic Colleague still has to infer which command to run, which report to inspect, what the synthetic data means, how local additive effects differ from graph contributions, how graph gates should be interpreted, and which metrics are only sanity checks versus real evidence. Without that separation, the repo risks two opposite failures: underselling the idea because the narrative path is missing, or overclaiming empirical merit from synthetic or simplified tracer-bullet artifacts.

The next phase needs a clear split between a Concept Demonstrator and a Merit Probe. The Concept Demonstrator should make the model decomposition understandable and runnable. The METR-LA Merit Probe should later provide the first real-data go/no-go evidence for whether GraphNAMLSS deserves further time investment.

Solution

Build a colleague-facing Concept Demonstrator around a single-command Concept Demo Bundle. The demo should use a dedicated Demo Fixture with a Localized Congestion Spillover Demo Scenario, where local traffic lags explain ordinary behavior and graph-neighbor congestion adds interpretable distributional effects: expected slowdown through the location parameter and increased uncertainty through the scale parameter.

The Concept Demo Bundle should produce deterministic NAMLSS, Tracer-Bullet GraphNAMLSS, Distributional GNN Baseline, and Point GNN Reference artifacts, then generate a report that leads with model decomposition rather than metric tables. The report should explicitly describe the Demo Data-Generating Story, show local shape contributions, graph contribution maps, Graph Gate Diagnostics, distributional and point metrics, and a Concept Demo Evidence Boundary. It should also include a Tracer-Bullet Fit Disclosure explaining that the demo uses a lightweight deterministic tracer-bullet fit to make the model decomposition inspectable, not to make empirical claims.

After the Concept Demonstrator is clear, the next evidence step is the METR-LA Merit Probe. The first real-data go/no-go comparison should be deterministic NAMLSS versus Tracer-Bullet GraphNAMLSS, with Distributional GNN Baseline and Point GNN Reference as secondary context. The Further-Investment Gate is deliberately modest: GraphNAMLSS should show at least one clear distributional signal, preferably improved NLL or CRPS, without serious degradation in coverage, PIT, MAE, or RMSE, while preserving interpretable local shape and graph contribution artifacts. If that gate fails, the project should diagnose before expanding to PEMS-BAY replication, the full Primary Experiment Matrix, Bayesian graph streams, or medical applications.

User Stories

1. As a GraphNAMLSS researcher, I want a Concept Demonstrator, so that colleagues can understand the modeling idea before judging empirical claims.
2. As a project maintainer, I want the Concept Demonstrator separated from the Merit Probe, so that runnable demo artifacts are not mistaken for benchmark evidence.
3. As a Methodologically Sympathetic Colleague, I want one command to run the Concept Demo Bundle, so that I can inspect the idea without manually orchestrating multiple model runs.
4. As a Methodologically Sympathetic Colleague, I want the demo report to lead with the model decomposition, so that I understand local additive effects and graph contributions before seeing metrics.
5. As a Methodologically Sympathetic Colleague, I want an explicit Demo Data-Generating Story, so that I know what the synthetic scenario is designed to show.
6. As a Methodologically Sympathetic Colleague, I want local shape plots by distributional parameter, so that I can inspect the NAMLSS-style local interpretability claim.
7. As a Methodologically Sympathetic Colleague, I want graph contribution maps by distributional parameter, so that I can inspect where graph-neighbor information enters the prediction.
8. As a Methodologically Sympathetic Colleague, I want Graph Gate Diagnostics, so that I can see whether graph information is used differently for location and scale.
9. As a Methodologically Sympathetic Colleague, I want the report to distinguish diagnostics from causal attribution, so that I do not overinterpret graph gate magnitudes.
10. As a Methodologically Sympathetic Colleague, I want the report to disclose the Tracer-Bullet Fit, so that I know the demo is not yet a final research-grade training path.
11. As a Methodologically Sympathetic Colleague, I want metrics included after the interpretability artifacts, so that I can see the experiment contract computes distributional and point metrics without reading them as proof.
12. As a Methodologically Sympathetic Colleague, I want the Concept Demo Evidence Boundary stated clearly, so that synthetic metrics are not confused with empirical support for the Distributional Calibration Claim.
13. As a GraphNAMLSS researcher, I want a Demo Fixture separate from the Contract Fixture, so that concept storytelling does not destabilize low-level pipeline tests.
14. As a model developer, I want the Demo Fixture to express Localized Congestion Spillover, so that graph-neighbor features have a clear synthetic role.
15. As a model developer, I want Heteroskedastic Spillover Uncertainty in the Demo Fixture, so that the scale parameter has a real scenario-level signal.
16. As a model developer, I want local routine behavior and graph spillover behavior separated in the synthetic scenario, so that the model decomposition is inspectable.
17. As a model developer, I want the Concept Demo Bundle to preserve the existing config-driven workflow, so that the demo uses the same experiment contract as later benchmark runs.
18. As a model developer, I want demo outputs saved in the existing artifact structure, so that the report can consume saved artifacts rather than retraining.
19. As a future contributor, I want glossary terms for Concept Demonstrator, Merit Probe, Demo Fixture, Contract Fixture, and Guided Demo Path, so that implementation work uses the project's domain language consistently.
20. As a future contributor, I want ADR 0003 respected, so that demo narrative work is not replaced by premature architecture expansion.
21. As a project maintainer, I want a Guided Demo Path in the README, so that first-time readers know what to run and what to inspect.
22. As a project maintainer, I want the Guided Demo Path to state the Concept Demo Evidence Boundary, so that readers do not treat synthetic demo metrics as research findings.
23. As a project maintainer, I want the generated report and README to agree on the demo's claim boundary, so that the repo communicates one coherent story.
24. As a GraphNAMLSS researcher, I want the METR-LA Merit Probe defined as the first real-data evidence gate, so that the project does not jump to replication or model breadth too early.
25. As a GraphNAMLSS researcher, I want deterministic NAMLSS versus Tracer-Bullet GraphNAMLSS to be the primary METR-LA Merit Probe comparison, so that the first go/no-go test targets the core graph-augmented NAMLSS claim.
26. As a GraphNAMLSS researcher, I want Distributional GNN Baseline and Point GNN Reference as secondary context, so that I can compare interpretability-preserving graph modeling against less interpretable or mean-only graph references.
27. As a GraphNAMLSS researcher, I want the Point GNN Reference clearly labeled as a point-accuracy guardrail, so that it is not treated as a distributional peer.
28. As a GraphNAMLSS researcher, I want the Further-Investment Gate defined before running the Merit Probe, so that the go/no-go decision is not improvised after seeing results.
29. As a GraphNAMLSS researcher, I want the Merit Probe Training Standard documented, so that real-data evidence uses an optimizer-trained path or explicitly justifies a deterministic path.
30. As a GraphNAMLSS researcher, I want the Merit Probe Failure Rule documented, so that unclear first results lead to diagnosis rather than uncontrolled architecture expansion.
31. As a model developer, I want a future optimizer-trained GraphNAMLSS path scoped separately from the Concept Demo Bundle, so that the demo can teach the idea without blocking on research-grade training.
32. As a statistical collaborator, I want GAMLSS and BAMLSS adapters to remain out of the first Concept Demonstrator, so that the demo focuses on the GraphNAMLSS idea rather than statistical baseline integration.
33. As a reviewer-like colleague, I want the repo to avoid claiming structural uncertainty or Bayesian graph-stream evidence in the Concept Demonstrator, so that the scope is honest.
34. As a future contributor, I want implementation issues to be vertical slices, so that each completed ticket produces a demoable or verifiable step toward the Concept Demonstrator or Merit Probe.
35. As a project maintainer, I want PEMS-BAY treated as a Replication Gate after the METR-LA Merit Probe, so that portability work does not precede the first evidence decision.

Implementation Decisions

• Treat the next phase as Concept Demonstrator first and Merit Probe second, following the glossary updates and ADR 0003.
• Build a Concept Demo Bundle as the immediate deliverable, not another model variant.
• Add a single demo command that orchestrates the demo end to end and writes a complete artifact bundle.
• Keep the existing Contract Fixture focused on data, split, lag, missingness, scaling, model, and report contract tests.
• Add a separate Demo Fixture for the Localized Congestion Spillover Demo Scenario.
• The Demo Fixture should encode Heteroskedastic Spillover Uncertainty so that location and scale have distinct synthetic signals.
• The Demo Fixture should remain synthetic and explicitly non-benchmark; it exists to teach the model decomposition.
• The report should follow the Concept Demo Report Order: model decomposition first, then local shape contributions, graph contributions, graph gates, metrics, and evidence boundary.
• The report should include a Demo Data-Generating Story before or near the first interpretability artifacts.
• The report should include a Tracer-Bullet Fit Disclosure.
• The report should explain Graph Gate Diagnostics as parameter-specific graph reliance diagnostics, not causal proof or standalone evidence.
• Demo metrics should be included as sanity checks that the experiment contract computes NLL, CRPS, coverage, PIT, MAE, and RMSE where applicable.
• The Guided Demo Path should be documented for colleagues, including the command to run, the primary report to open, and the correct reading order.
• The Guided Demo Path should not require METR-LA artifacts.
• The METR-LA Merit Probe should remain the first real-data evidence gate after the Concept Demonstrator.
• The METR-LA Merit Probe Primary Comparison is deterministic NAMLSS versus Tracer-Bullet GraphNAMLSS.
• The METR-LA Merit Probe Secondary Context is Distributional GNN Baseline and Point GNN Reference.
• The Further-Investment Gate should require at least one clear distributional signal, preferably NLL or CRPS improvement, without serious degradation in other distributional or point metrics and without losing interpretability.
• If the METR-LA Merit Probe fails the gate, the next step is diagnosis, not PEMS-BAY replication, full matrix work, Bayesian graph streams, or medical applications.
• The Merit Probe Training Standard requires an optimizer-trained GraphNAMLSS path for real evidence, or an explicit justification that the deterministic fitting path is the intended model.
• PEMS-BAY remains a Replication Gate after METR-LA, not the first go/no-go evidence step.
• Statistical baseline adapters remain important but are not required for the Concept Demo Bundle.
• The deep modules to build or modify are the demo fixture builder, demo orchestration command, report narrative/rendering module, demo documentation path, and later the Merit Probe training/evidence module.

Testing Decisions

• Good tests should verify external behavior and scientific contracts, not private implementation details.
• Demo Fixture tests should verify that the generated synthetic scenario has localized congestion spillover, distinct neighbor signal, and heteroskedastic spillover uncertainty.
• Demo Fixture tests should avoid brittle assertions about exact internal formulas unless those formulas are the contract being exposed to colleagues.
• Demo command tests should verify that one command produces the complete Concept Demo Bundle from a clean output directory.
• Demo command tests should verify that the bundle includes saved run artifacts for deterministic NAMLSS, Tracer-Bullet GraphNAMLSS, Distributional GNN Baseline, and Point GNN Reference.
• Report tests should verify that the Concept Demo Report Order is reflected in the generated markdown.
• Report tests should verify that the Demo Data-Generating Story, Concept Demo Evidence Boundary, and Tracer-Bullet Fit Disclosure appear in the generated report.
• Report artifact tests should verify that local shape plots, graph contribution maps, graph gates, metrics, and resolved settings are produced.
• Documentation tests should verify that the README includes the Guided Demo Path and does not present synthetic metrics as Merit Probe evidence.
• Merit Probe planning tests should verify that the real-data evidence gate can be represented without requiring PEMS-BAY or the full Primary Experiment Matrix.
• Prior art exists in the current CLI, report, config, synthetic traffic fixture, training, metrics, and development workflow documentation tests.

Out of Scope

• Proving that GraphNAMLSS outperforms deterministic NAMLSS or GNN baselines.
• Treating synthetic Concept Demo Bundle metrics as evidence for the Distributional Calibration Claim.
• Completing the METR-LA Merit Probe inside the Concept Demonstrator slice.
• Completing an optimizer-trained GraphNAMLSS research path before the demo can communicate the idea.
• Running PEMS-BAY replication before the METR-LA Merit Probe passes the Further-Investment Gate.
• Implementing the full Primary Experiment Matrix.
• Implementing Bayesian graph streams.
• Implementing structural uncertainty experiments.
• Implementing the Spatial Health Graph application track.
• Completing GAMLSS or BAMLSS external adapters.
• Claiming causal interpretation from graph contribution gates or graph contribution maps.

Further Notes

This PRD is downstream of the original GraphNAMLSS self-contained experiment package PRD and respects the existing ADRs. It narrows the next phase around communication and evidence discipline: first make the idea legible to Methodologically Sympathetic Colleagues, then run a METR-LA Merit Probe before investing in replication, model breadth, Bayesian extensions, or application tracks.

The main product of the Concept Demonstrator is not a benchmark table. It is a trustworthy guided path through the GraphNAMLSS idea: local additive distributional interpretability plus graph-derived, parameter-specific contributions, shown with enough runnable artifacts that colleagues can decide whether the idea is coherent and worth probing on real data.