perf(layered): dynamic service does not allocate anymore

[martintmk]

I got quite surprised how much allocations are happening in fetch with standard pipeline. This change leverages BlindPool that allows memory and future pooling, thus eliminating the memory allocations.

Before:

Allocation statistics:

| Operation                | Mean bytes | Mean count |
|--------------------------|------------|------------|
| custom_minimal_pipeline  |       3409 |         15 |
| custom_standard_pipeline |      11600 |         29 |
| minimal_pipeline         |       2721 |         14 |
| standard_pipeline        |      13667 |         31 |

After:

Allocation statistics:

| Operation                | Mean bytes | Mean count |
|--------------------------|------------|------------|
| custom_minimal_pipeline  |       2713 |         13 |
| custom_standard_pipeline |       3744 |         27 |
| minimal_pipeline         |       2713 |         13 |
| standard_pipeline        |       3771 |         29 |

no changes in public API

[Copilot]

Pull request overview

This PR reduces per-request allocations when using layered’s DynamicService (used by fetch pipelines) by replacing dynosaur-based type erasure with an infinity_pool::BlindPool-backed future pooling approach.

Changes:

• Reworks DynamicService to execute via a pooled future handle (BlindPooledMut), avoiding repeated heap allocations during dynamic dispatch.
• Removes the dynosaur-based DynService generation path from layered and switches the dynamic-service feature to depend on infinity_pool.
• Updates layered crate dependencies accordingly (including lockfile updates).

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File	Description
crates/layered/src/service.rs	Removes dynosaur attribute previously used to generate an internal dynamic service wrapper.
crates/layered/src/dynamic.rs	Implements pooled dynamic dispatch using BlindPool and a custom Future wrapper.
crates/layered/Cargo.toml	Switches dynamic-service feature from dynosaur to infinity_pool and updates deps.
Cargo.lock	Updates the lockfile to reflect the dependency change.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (80bbf15) to head (b7381a0).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #480   +/-   ##
=======================================
  Coverage   100.0%   100.0%           
=======================================
  Files         335      335           
  Lines       25586    25600   +14     
=======================================
+ Hits        25586    25600   +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Vaiz]

considering that infinity_pool never releases memory, we should provide an ability to use user provided instance of a pool

@sandersaares, do you have any recommendations here?

[martintmk]

Also worried about potential unsafety this might bring. How risky is this @sandersaares?

[sandersaares]

considering that infinity_pool never releases memory, we should provide an ability to use user provided instance of a pool

There is a shrink_to_fit() if you want to release memory. It is possible some more advanced capacity-release policies could be implemented but what they are is not obvious to me.

When do we even need to release capacity? There are two reasons that immediately come to mind:

1. When load patterns change (e.g. less load during the night). This may benefit from some "shrink once every 30 minutes" logic (maybe). On the other hand, if load patterns change, will they also change for other services on the same PC or will we just complicate logic by releasing memory that other apps also do not need? In general, we should aim for a steady state, not ups-and-downs, even if load patterns change.
2. After a sudden load spike. However, load spikes should be mitigated by rate limiting and policy systems in the request path, so the actual complexity/value tradeoff here is not obvious to me.

It is an area that could benefit from exploration and data from real customer scenarios. It is not obvious to me that shrinking resource pools is at all necessary/appropriate.

Also worried about potential unsafety this might bring. How risky is this @sandersaares?

There are no open bugs that I am aware of. More risky than safe Rust, less risky than skydiving. infinity_pool is heavily used in oxidizer_executor and oxidizer_io, for some reference point.

[Vaiz]

considering that infinity_pool never releases memory, we should provide an ability to use user provided instance of a pool

There is a shrink_to_fit() if you want to release memory. It is possible some more advanced capacity-release policies could be implemented but what they are is not obvious to me.

is it better to have one shared pool for all instances of DynamicService, or one pool per instance is fine?

[sandersaares]

How is the change in terms of processor time spent?

[sandersaares]

The generics are blinding my eyes today, so I cannot identify it myself but is it possible here to get a type for the thing being inserted, or at least a Layout? If so, switching to OpaquePool would eliminate some overhead. OpaquePool vs BlindPool is basically "one type/layout" versus "multiple types/layouts", with the BlindPool acting as a router over a pool of OpaquePool instances.

[sandersaares]

is it better to have one shared pool for all instances of DynamicService, or one pool per instance is fine?

Unclear - I do not have a good understanding of the actual usage patterns. Sharing a single pool across many callers improves resource utilization, of course.

What I have sometimes done is to have a thread_local! { static POOL: BlindPool = BlindPool::new() } so resources are shared but lock contention is kept minimal since accesses are always from the same thread (unless pooled objects are sent to other threads, of course, which is valid).