pod network integration (Shadow mode) by AryanGodara · Pull Request #4205 · cowprotocol/services

AryanGodara · 2026-02-24T16:43:15Z

Description

Closes #4126. Supersedes #4052.

Adds an opt-in shadow path in the driver that mirrors each auction onto the pod network. After the driver scores its own best solution, a tokio::spawn-ed task submits it as a bid to a pod auction contract, waits for the deadline, fetches everyone's bids, and runs a local copy of the same arbitration. Pod failures never block, delay, or mutate the response to autopilot.

If [pod] is omitted from the driver TOML, nothing pod-shaped is allocated.

Highlights

Isolated. Pod state lives in crates/driver/src/infra/pod. PodManager (in flow.rs) owns the provider, the reusable AuctionClient, and the local arbitrator; it exposes a single spawn(). Competition::solve invokes it once and is otherwise untouched.
Deterministic tie-breaking. winner_selection::Solution::canonical_hash is shared by autopilot and driver, so independent observers tie-break identically on the same logical solution.
Protocol parameter, not a constant. max_winners (used by the local arbitrator) is configurable from the [pod] block; defaults to 10.
Locked-account recovery. Pod allows one in-flight tx per submitter. On the locked-account error we call pod_getRecoveryTargetTx → recover(tx_hash, nonce) on the 0x50d…0003 precompile and retry once. See infra/pod/recovery.rs for the production-mode caveats (TODO(production-pod)).
Malformed bid resilience. A single bad bid is logged and skipped, never aborts arbitration.

What's new

Area	What
Crate	`winner-selection` — `Solution::canonical_hash`, `Arbitrator::arbitrate_paired`, public `SolutionKey`.
Driver module	`infra::pod` (`config`, `flow`, `recovery`) and `domain::competition::solver_winner_selection`.
Driver TOML	`[pod] endpoint, auction-contract-address, max-winners` (`max-winners` defaults to 10). Pod is opt-in.
E2E tests	`pod_test_basic`, `pod_test_multi_order`, `pod_test_multi_solver` (gated by `--run-ignored ignored-only`; need live pod RPC).

How to test

# Lint, exactly what CI runs
just fmt --check && just clippy && just fmt-toml --check

# Unit tests
cargo nextest run -p winner-selection -p driver -p autopilot

# Pod e2e (needs cow.pod.network:11600 reachable)
just test-pod

github-actions · 2026-02-24T16:43:28Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

github-actions · 2026-03-04T00:13:14Z

This pull request has been marked as stale because it has been inactive a while. Please update this pull request or it will be automatically closed.

github-actions · 2026-03-18T00:15:34Z

This pull request has been marked as stale because it has been inactive a while. Please update this pull request or it will be automatically closed.

+[[package]]
+name = "jsonwebtoken"
+version = "9.3.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5a87cc7a48537badeae96744432de36f4be2b4a34a05a5ef32e9dd8a1c169dde"
+dependencies = [
+ "base64 0.22.1",
+ "js-sys",
+ "pem",
+ "ring",
+ "serde",
+ "serde_json",
+ "simple_asn1",
+]


AryanGodara · 2026-04-07T13:33:21Z

Pod Integration: Design Decisions (more context apart from PR description)

Shadow Mode Architecture

Pod integration runs in shadow mode, so bid submission failures are logged as warnings but never block the protocol flow.

Haircut-Based Score Differentiation (fo rthe e2e Tests)

Since i'm using 2 instances of baseline solvers to simulate competition for the e2e tests; they produce the same flowTwo solvers competing for the same order find identical routes → identical scores.

Solution

Use haircut_bps parameter to artificially differentiate scores:

Solver A: haircut_bps = 0 (full score)
Solver B: haircut_bps = 5000 (50% haircut → lower score)

This reliably produces different scores for testing pod winner selection without complex liquidity setup.

Nonce/Pending TX Handling

Current Issue

Pod network rejects new TXs when previous ones from same sender are pending:

error code -32603: Another transaction 0x... is still pending

Approach

Log nonce on provider init for debugging
Shadow mode ensures failures don't block main flow
Pod team to resolve stuck TXs on their end
Future: pod-sdk should handle nonce conflicts internally (replace-by-fee)

Deterministic Test Addresses

E2E tests use deterministic addresses derived from Anvil's default mnemonic. Pre-funding these on pod network:

0xf985d2cf3f3c5cfc798d8e9e4ebdd6777653c3cb - solver 1
0x9315fc8ffae493123b9b6b1c93d50c9b9eef0344 - solver 2

Both funded with ~10 ETH on pod network.

Config Structure

[pod]
endpoint = "http://cow.pod.network:8545"
auction-contract = "0xeDD0670497E00ded712a398563Ea938A29dD28c7"

Wallet uses existing [account] section - no separate pod wallet config needed.

ogabrielides · 2026-04-07T14:33:08Z

I have read the CLA Document and I hereby sign the CLA

gemini-code-assist

Code Review

This pull request implements integration with the pod network to support decentralized solver competitions. Key changes include upgrading the Alloy dependency suite to version 1.8.3, introducing deterministic solution hashing within the winner-selection crate, and adding a pod_flow to the driver for bid submission and local arbitration. The PR also includes new E2E tests, local configurations, and account recovery logic for the pod network. Feedback was provided regarding the fragility of the bid-fetching process, where a single malformed bid currently causes the entire batch to fail; it is recommended to collect individual errors instead of aborting the process.

gemini-code-assist · 2026-04-10T13:06:30Z

+            let bidder_solve_response: dto::SolveResponse =
+                match serde_json::from_slice(bid.data.as_slice()) {
+                    Ok(resp) => resp,
+                    Err(e) => {
+                        tracing::error!(error = %e, "failed to deserialize SolveResponse");
+                        return Err(anyhow::Error::new(e));
+                    }
+                };


Aborting the entire bid fetching process due to a single malformed bid is fragile. However, simply skipping the error is insufficient. According to our rules, when fetching a batch of items where individual fetches can fail, the API response should explicitly indicate which items failed and provide error details for each failure, rather than silently ignoring them.

let mut errors = Vec::new(); for bid in bids { match serde_json::from_slice::<dto::SolveResponse>(bid.data.as_slice()) { Ok(resp) => { for solution in resp.solutions { participants.push(Bid::new(solution)); } } Err(e) => { tracing::error!(error = %e, "failed to deserialize SolveResponse"); errors.push((bid.id, e.to_string())); } } }

References

When fetching a batch of items where individual fetches can fail, do not silently ignore errors. The API response should explicitly indicate which items failed and provide error details for each failure.

…ebase to latest main branch, get test and services compiling again

…imports

…rbosity - Add test-pod-verbose justfile target for detailed test output - Replace [pod] log prefixes with structured tracing spans (pod_flow, pod_submit_bid, pod_fetch_bids, pod_local_arbitration) - Move auction_id and solver context to span fields instead of repeating in each log - Reduce log noise: change balance fetch failure from error to warn, make signer type selection debug-level - Add debug-level payload hex logging,

…module

…t up

… new pod network version

…ls workaround

…ount of skipped bids

…w signature

…signature

…imation driver

…tion

… driver/autopilot wrappers

…te related structures

jmg-duarte

Can't we split this by doing a merge over the winsel changes and their downstream friends?

IMO this is way too much to be merged and released confidently in one go

jmg-duarte · 2026-05-08T09:30:51Z

+    pub solver: Address,
+    pub solution_id: u64,


These don't need to be public, arguably the key doesn't either

jmg-duarte · 2026-05-08T09:35:10Z

+impl<S> From<&Solution<S>> for SolutionKey {
+    fn from(solution: &Solution<S>) -> Self {
+        Self {
+            solver: solution.solver(),
+            solution_id: solution.id(),
+        }
+    }


This makes other constructions of SolutionKey { .. } inconsistent, either use the same for all of them, or remove this one and use the same approach as the others (2nd is better IMO)

jmg-duarte · 2026-05-08T09:35:41Z

+/// Result of [`Arbitrator::arbitrate_paired_and_rejoin`].
+#[derive(Debug)]
+pub struct Rejoined<T> {


The comment should rather explain what this represents, the current one is more like a footnote

jmg-duarte · 2026-05-08T09:36:33Z

+    /// Sorting before arbitration is what lets independent observers
+    /// (autopilot, driver, third-party verifiers) reach the same
+    /// tie-breaking decision on the same logical solution set.
+    pub fn arbitrate_paired<T>(


What is supposed to be the shape of T?

I see this is only used by arbitrate_paired_and_rejoin, we can drop the pub

jmg-duarte · 2026-05-08T09:39:10Z

+    /// A non-zero `orphans` count means two inputs shared the same
+    /// `SolutionKey`. Callers should log and alert on this, not treat it
+    /// as fatal.
+    pub fn arbitrate_paired_and_rejoin<T>(


I see this only being used with Bid<BigPayload, Unscored>; I think we can lock in the type here and extend to generics later if needed

jmg-duarte · 2026-05-08T09:48:08Z

+    pub fn arbitrate_paired<T>(
+        &self,
+        items: Vec<(T, Solution<Unscored>)>,
+        context: &AuctionContext,
+    ) -> (Ranking, HashMap<SolutionKey, T>) {
+        let mut paired = items;
+        paired.sort_by_cached_key(|(_, solution)| solution.canonical_hash());
+        let mut by_key = HashMap::with_capacity(paired.len());
+        let mut solutions = Vec::with_capacity(paired.len());
+        for (item, solution) in paired {
+            by_key.insert(SolutionKey::from(&solution), item);
+            solutions.push(solution);
+        }
+        (self.arbitrate(solutions, context), by_key)
+    }


The move to paired just to make items mut is kinda weird, you can also avoid the extra collection muts with an iterator

Suggested change

pub fn arbitrate_paired<T>(

&self,

items: Vec<(T, Solution<Unscored>)>,

context: &AuctionContext,

) -> (Ranking, HashMap<SolutionKey, T>) {

let mut paired = items;

paired.sort_by_cached_key(|(_, solution)| solution.canonical_hash());

let mut by_key = HashMap::with_capacity(paired.len());

let mut solutions = Vec::with_capacity(paired.len());

for (item, solution) in paired {

by_key.insert(SolutionKey::from(&solution), item);

solutions.push(solution);

}

(self.arbitrate(solutions, context), by_key)

}

pub fn arbitrate_paired<T>(

&self,

mut items: Vec<(T, Solution<Unscored>)>,

context: &AuctionContext,

) -> (Ranking, HashMap<SolutionKey, T>) {

items.sort_by_cached_key(|(_, solution)| solution.canonical_hash());

let (by_key, solutions) = items

.into_iter()

.map(|(item, solution)| ((SolutionKey::from(&solution), item), solution))

.collect();

(self.arbitrate(solutions, context), by_key)

}

jmg-duarte · 2026-05-08T09:54:39Z

+        let mut orphans = 0;
+        let mut rejoin = |s: Solution<Ranked>| -> Option<(T, Solution<Ranked>)> {
+            let key = SolutionKey::from(&s);
+            match by_key.remove(&key) {
+                Some(t) => Some((t, s)),
+                None => {
+                    orphans += 1;
+                    None
+                }
+            }
+        };
+
+        let filtered_out = ranking
+            .filtered_out
+            .into_iter()
+            .filter_map(&mut rejoin)
+            .collect();
+        let ranked = ranking.ranked.into_iter().filter_map(&mut rejoin).collect();


There must be a simpler way of doing this, there's mutation and pure things leading to collects, looks very weird to me

The FnMut + captured mutable state sounds like an accident waiting to happen as well

Can this made simpler?

jmg-duarte · 2026-05-08T09:56:08Z

 };

 /// Auction arbitrator responsible for selecting winning solutions.
+#[derive(Clone)]


This Clone is not actually needed because the SolverArbitrator also doesn't need one

jmg-duarte · 2026-05-08T09:57:25Z

+    /// Clearing prices keyed by token address. Not used by arbitration;
+    /// included so `canonical_hash` fingerprints them.
+    prices: HashMap<Address, U256>,


So, can't we make cannonical_hash's signature be fn(&self, &HashMap<Address, U256>) instead?

Or store the hash of the prices at creation time instead of carrying them around?

jmg-duarte · 2026-05-08T10:00:37Z

+impl<P: Clone, State: Clone> Clone for Bid<P, State> {
+    fn clone(&self) -> Self {
+        Self {
+            payload: self.payload.clone(),
+            state: self.state.clone(),
+        }
+    }
+}


Even though it's not 1:1, we can use derive(Clone) here instead

AryanGodara self-assigned this Feb 24, 2026

AryanGodara force-pushed the aryan/pod-network-integration branch from aff7b2c to dad9d3d Compare February 24, 2026 18:59

MartinquaXD mentioned this pull request Mar 3, 2026

feat: pod integration #4052

Closed

2 tasks

github-actions Bot added the stale label Mar 4, 2026

AryanGodara removed the stale label Mar 9, 2026

jmg-duarte reviewed Mar 10, 2026

View reviewed changes

Comment thread crates/e2e/src/setup/colocation.rs Outdated

github-actions Bot added the stale label Mar 18, 2026

github-actions Bot closed this Mar 26, 2026

AryanGodara reopened this Mar 31, 2026

AryanGodara removed the stale label Mar 31, 2026

AryanGodara force-pushed the aryan/pod-network-integration branch from 99c8b62 to 78a3e71 Compare March 31, 2026 14:14

github-advanced-security AI found potential problems Mar 31, 2026

View reviewed changes

AryanGodara force-pushed the aryan/pod-network-integration branch from e32884c to 7f0f75c Compare April 3, 2026 13:01

AryanGodara force-pushed the aryan/pod-network-integration branch from eec3eab to 62e7fc4 Compare April 7, 2026 13:51

github-actions Bot added a commit that referenced this pull request Apr 7, 2026

@ogabrielides has signed the CLA in #4205

4cab38f

AryanGodara force-pushed the aryan/pod-network-integration branch 7 times, most recently from cad881f to b61e69d Compare April 10, 2026 12:40

AryanGodara marked this pull request as ready for review April 10, 2026 13:00

AryanGodara requested a review from a team as a code owner April 10, 2026 13:00

gemini-code-assist Bot reviewed Apr 10, 2026

View reviewed changes

AryanGodara added 27 commits May 6, 2026 15:59

add configs for local testing, and add instructions for LLM agents

6924fe2

remove driver2 and baseline2 services from playground compose files.R…

8cbd61a

…ebase to latest main branch, get test and services compiling again

fix: consolidate pod types, add pod test infrastructure, and cleanup …

2d5dce2

…imports

pod e2e test with full order flow, and temp logging while testing

08484df

feat: add pod network verification to e2e tests

123d33a

remove redundant script to spin up services

b63713b

add temp verbose test command

6d32e78

refactor: reorganize local config files and remove unused eth domain …

09e6cca

…module

remove local configs, not reqd

512a7e7

reference config files to run driver and autopilot locally

361bb27

chore: lint and clippy fixes

b704cd3

fix: bring back the removed load_account function, missed while rebasing

b653afe

lint: use tombi, missed in last commits

5fbacce

fix: attempt to fix rustls error in forked test CI

bba9df0

chore: tombi lint fix

d66450d

fix: add wait func for driver since the pod config takes longer to se…

c1ea345

…t up

add pod account recovery mechanism for locked accounts, and switch to…

0d6aa9b

… new pod network version

chore: upgrade alloy dependencies from 1.7.3 to 1.8.3 and remove rust…

31a127f

…ls workaround

fix: skip malformed bids instead of failing entire competition, log c…

f6301c8

…ount of skipped bids

fix: update solver_winner_selection to match new winsel::Solution::ne…

364c65e

…w signature

fix: wrap score_value in vec for SolveResponse::new to match updated …

aa562d8

…signature

refactoring, add order-quoting config section with baseline price est…

e2e96a9

…imation driver

refactor: optimize sorting of orders and prices in hash_solution func…

05b8ed6

…tion

update acc to comments

7007634

refactor(winner-selection): expose arbitrate_paired helper and dedupe…

80d6b8f

… driver/autopilot wrappers

refactor(competition): introduce BidPayload for auction bids and upda…

61e1846

…te related structures

AryanGodara force-pushed the aryan/pod-network-integration branch from 53b940c to 61e1846 Compare May 6, 2026 10:36

AryanGodara requested a review from squadgazzz May 6, 2026 11:17

jmg-duarte reviewed May 8, 2026

View reviewed changes

Conversation

AryanGodara commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Highlights

What's new

How to test

Uh oh!

github-actions Bot commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 4, 2026

Uh oh!

Uh oh!

github-actions Bot commented Mar 18, 2026

Uh oh!

Check warning

AryanGodara commented Apr 7, 2026

Pod Integration: Design Decisions (more context apart from PR description)

Shadow Mode Architecture

Haircut-Based Score Differentiation (fo rthe e2e Tests)

Solution

Nonce/Pending TX Handling

Current Issue

Approach

Deterministic Test Addresses

Config Structure

Uh oh!

ogabrielides commented Apr 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

jmg-duarte left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

AryanGodara commented Feb 24, 2026 •

edited

Loading

github-actions Bot commented Feb 24, 2026 •

edited

Loading