bugfix: improve shutdown and stop PoA sooner#3289
bugfix: improve shutdown and stop PoA sooner#3289MitchTurner wants to merge 17 commits intomasterfrom
Conversation
PR SummaryMedium Risk Overview Hardens GraphQL shutdown by running the server in a spawned task and applying a Improves test stability by waiting for node interfaces to become reachable before proceeding and retrying txpool inserts on transient Includes minor dependency/maintenance updates: adds a Reviewed by Cursor Bugbot for commit 9e14736. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 11328e1. Configure here.
| let total_services = self.services.len(); | ||
| futures::future::join_all((1_usize..).zip(self.services.iter()).map( | ||
| |(service_num, service)| { | ||
| stop_sub_service(service_num, total_services, service.as_ref()) |
There was a problem hiding this comment.
I'd prefer to use the actual service name here, but I don't want to touch to many pieces of code in a bugfix like this. This doesn't have access to RunnableService::NAME

Linked Issues/PRs
Description
Because the services were being being shutdown in the same order they were started, the PoA service can be held up by earlier services. This means that our producers could be nominally "shut down" but the PoA service still runs a while. The old leader would still hold on to its lock, but it wouldn't be broadcasting blocks anymore.
The solution here is to shutdown all of the services in parallel, this way none of them is blocking the others.
The other part of the PR is taking care of the GraphQL which was the culprit for PoA never being shut down. We've added a timeout to force it to shutdown if it doesn't shut down gracefully.
Checklist
Before requesting review
After merging, notify other teams
[Add or remove entries as needed]