tracking: execution module consolidation (PipelineExecutor, notifications, Cache)

[mh0lt]

Summary

Tracking issue for consolidating execution pipeline ownership into execmodule. This work makes the execution module more self-contained and aligns with the broader 2-cache rationalization (#19623).

Work Streams

1. PipelineExecutor (feat/pipeline-executor)

Consolidate all staged sync pipeline invocations into a single PipelineExecutor struct:

• ProcessFrozenBlocks (startup)
• RunLoop (FCU catchup)
• ValidateBlock (fork validation)

Move ForkValidator from engine_helpers to execmodule, remove the validatePayloadFunc closure, decouple stage_finish from ForkValidator.

Status: Branch feat/pipeline-executor — complete, waiting for review of dependencies.

2. Pre-commit notifications (#19957)

Move Hook.AfterRun to fire before DB commit. No consumer reads from DB after receiving a notification — all payloads are self-contained. The pipeline's RwTx/overlay has all data needed for reads.

Status: Branch feat/pre-commit-notifications — complete, waiting for review of dependencies.

3. Eliminate fcuOverlay in updateForkChoice

The fcuOverlay (MemoryMutation) in updateForkChoice is redundant — the SD already has blockOverlay for table-level writes (headers, bodies, canonical hashes, stage progress, forkchoice hashes). Consolidate:

• Remove the separate fcuOverlay; use sd.BlockOverlay() as the pipeline tx
• sd.Flush() handles both domain + table writes in one call
• Simplifies CommitCycle (no separate overlay to flush/recreate)
• Simplifies runForkchoiceCommitOverlay (drops overlay parameter)
• Lazy revert unchanged: sd.Close() discards everything

Status: Design complete, implementation pending.

4. Remove tx parameter from pipeline (final step)

Currently RunLoop, Sync.Run, RunUnwind, RunPrune, and every stage func take (sd, tx) separately. Now that tx == sd.BlockOverlay(), the tx parameter is redundant. Remove it:

• RunLoop(ctx, sd, cfg) instead of RunLoop(ctx, sd, tx, cfg)
• Sync.Run(sd, initialCycle, firstCycle) instead of Sync.Run(sd, tx, ...)
• Each stage calls sd.BlockOverlay() when it needs table access
• Purely mechanical refactor across all stages

Status: Deferred to last — big diff, easier when everything else is stable.

5. Cache cleanup (future)

The execmodule.Cache shim bridges RPC reads to SharedDomains for domain data. It predates SD's current capabilities. Cleanup deferred until #19623 Phase 5 stabilizes the StateReader/CacheView interfaces.

Relationship to #19623

These changes prepare the ground for the 2-cache rationalization:

This work	Enables in #19623
PipelineExecutor centralizes pipeline calls	Cleaner integration point for Phase 3 synchronous domain apply
Pre-commit notifications	Aligns with SD as authoritative read layer (Phase 3+)
Eliminate fcuOverlay	SD owns all in-flight state; one flush path
Remove tx from pipeline	SD is the single interface for stages — domain + table access
Cache documents SD dependency	Cleanup path clear once interfaces stabilize (Phase 5)
controlServer interface narrowing (future)	Removes p2p/sentry dependency from validation pipeline

Order of Operations

1. PipelineExecutor (done)
2. Pre-commit notifications (done)
3. Eliminate fcuOverlay ← next
4. Remove tx from pipeline ← last (mechanical, big diff)
5. Cache cleanup (after Rationalize IntraBlockState to a 2-Cache Model #19623 Phase 5)

Related Issues

• Rationalize IntraBlockState to a 2-Cache Model #19623 — Rationalize IntraBlockState to a 2-Cache Model
• execution: extract apply loop into event stream with fan-out consumers #19798 — Extract apply loop into event stream with fan-out consumers
• Design: TransactionState/BlockState separation, TxTask refactoring, cross-process TX caching #19855 — TransactionState/BlockState separation, TxTask refactoring
• execmodule: send notifications before DB commit #19957 — Send notifications before DB commit

[VBulikov]

yperbasis added Imp2 as Importance