Consume pulseengine/wit-bindgen (cabi-realloc-extern): gale-owned arena instead of a per-component one

[avrabe]

Goal

Stop carrying a per-component allocator. cabi_realloc should delegate to gale's __cabi_arena_realloc via pulseengine/wit-bindgen's cabi-realloc-extern feature — one embedder-owned arena across the fused image, which is that fork's design (pulseengine/wit-bindgen#4) and gale's (gale#89).

Where we are

• ✅ feat(build): build no_std components for wasm32-unknown-unknown, not wasip2 #329 — no_std components now build wasm32-unknown-unknown, not wasip2. That's the prerequisite: target_env != "p2", so the fork's feature applies at all.
• ✅ fix(rate): bounded arena instead of a growing allocator — remove memory.grow #327 — a bounded per-component bump arena replaced the growing lol_alloc, killing memory.grow (1 → 0). Explicitly interim.
• ❌ this issue — replace that arena with gale's.

Why it's a migration, not a dependency swap

	version	rt crate
the fork (integration/embedded-rt-no-grow)	wit-bindgen 0.58.0	no separate wit-bindgen-rt — rt is a module inside wit-bindgen
relay's components	wit-bindgen-rt 0.41.0	separate crate

So it's 0.41 → 0.58 plus a crate restructure, which pulls in cargo-component compatibility and regenerated bindings — on the flight path.

Open design question to settle first

With cabi-realloc-extern, __cabi_arena_realloc is embedder-provided. Does the published component then carry an unresolved import only gale satisfies? If so, the published artifact would no longer run standalone in wasmtime — which our tests/rate-loop-proof and any host-side composition test depend on.

Likely answer is two build modes (host build keeps the default allocator; gale build uses the extern arena), which is how the fork frames it — "it's a runtime-crate build mode". Needs verifying before we migrate, not after, because it decides what we publish to ghcr for jess to pull.

Acceptance

• Determine whether cabi-realloc-extern yields an unresolved import; decide what the published artifact is
• Migrate the component crates to the fork (0.58) and confirm bindings still generate under cargo-component
• falcon-rate builds with no relay-side allocator, still a component, 0 WASI, 0 memory.grow
• tests/rate-loop-proof still closes the loop (0.193 s, |err| 0.0059 rad/s baseline)
• Remove the interim arena from wasm/cm/rate/src/lib.rs
• Confirm with gale/meld that fuse --memory shared --address-rebase accepts the result — the actual kill-criterion in gale#89

Refs: pulseengine/wit-bindgen#4, gale#89, meld#299, jess#167.

🤖 Generated with Claude Code

[avrabe]

Reframing: the constraint is NO DYNAMIC MEMORY on embedded — so the goal is no cabi_realloc at all, not a better allocator.

That invalidates the arena direction as an end state. A bounded arena is still dynamic memory; the embedded SW has none. So the target is eliding cabi_realloc entirely (wit-bindgen#6 disable_cabi_realloc), not backing it with something smaller.

I measured what that takes. Two things are required, and they compose.

1. The seam must not spill its parameters — proven, and it's the reshape jess already agreed

Built one component exporting BOTH signatures so the ABI difference is visible side by side in the same module:

tick      (18 flat params) : (param i32)                                    (result i32)   <- SPILLED via memory
tick-flat (10 flat params) : (param f32 f32 f32 f32 f32 f32 f32 f32 f32 f32) (result i32)  <- passed DIRECTLY

Today rate.tick takes vehicle-state (14 f32) + rate-setpoint (4 f32) = 18 flat params, over the canonical MAX_FLAT_PARAMS (16), so the host writes the arguments into our linear memory via cabi_realloc. That is the dynamic memory we cannot have.

The fix is the seam reshape jess already asked for in jess#167: rate takes the raw gyro and bias-corrects it (measured = imu.{gx,gy,gz} − state.{bgx,bgy,bgz}) instead of the whole vehicle-state. That is gyro 3 + bias 3 + setpoint 3 + thrust 1 = 10 params ≤ 16 → no spill.

So the reshape isn't only an architectural preference — it is what makes a no-dynamic-memory component possible at all.

2. Even then, the bindings still FORCE the export — which is exactly what wit-bindgen#6 fixes

With only the flat signature present, cabi_realloc is still exported and referenced. That is precisely what pulseengine/wit-bindgen#6 describes:

A world whose interfaces are entirely flat/scalar never requires canonical-ABI reallocation, yet these bindings still force the cabi_realloc export … That export is allocator-backed and on wasm32 pulls in memory.grow, which blocks single-address-space / no-grow embedded lowering.

#6 (disable_cabi_realloc, on feat/elide-cabi-realloc-flat-worlds) is the opt-out. And note its own caveat — it must not be set for a world whose params spill, or componentization fails loudly. So (1) is a precondition for (2) being safe.

Revised plan for this issue

• Reshape the rate seam with jess (18 → 10 params). Already agreed in jess#167 on estimator grounds; now also a hard embedded requirement.
• Confirm no relay component crosses the boundary with a list/string (checked: cascade.wit has zero — params are the only trigger).
• Consume the fork for disable_cabi_realloc (Add Kani harnesses for stream-transformer state machines (LC, SCH, SC, HS, CFDP) #6) — note this is the elision branch, not the arena one (End-to-end verification chain: Verus → Rocq → Lean → Kani for all five engines #4). Same 0.41 → 0.58 migration cost.
• Then: no allocator, no cabi_realloc, no memory.grow — and the interim arena in wasm/cm/rate is deleted rather than replaced.
• Verify the kill-criterion in gale#89: meld fuse --memory shared --address-rebase accepts it.

Correction to my earlier framing in this issue: I had this pointed at #4 (embedder arena). With no dynamic memory available the right target is #6 (elide), gated on the param reshape. The arena would only be the answer for a component that genuinely must allocate — which, once the seam is flat, ours does not.

[avrabe]

Correction to my previous comment: the bounded arena is FINE — it is work memory, not dynamic memory.

I over-corrected. The constraint is "no dynamic memory" in the sense of no heap, no growth — not "no statically-reserved bytes". The arena in #327 is a fixed-size static buffer that never grows and traps on exhaustion; that is work memory, which the embedded SW does define and allow.

So the position is a preference, not a blocker:

	status
bounded static arena (#327, shipped)	acceptable end state — work memory, no heap, no memory.grow
elided cabi_realloc (wit-bindgen#6 + flat seam)	better — zero bytes reserved, nothing to size or justify

That changes two things I wrote above:

1. The arena is not deleted "rather than replaced". It stands on its own. If the elision path lands, the arena goes away as a simplification; if it doesn't, we are not blocked.
2. This is no longer on the critical path. The seam reshape (18 → 10 params) is still worth doing — for jess's estimator-latency reason, and because it removes the need to size and defend a work-memory budget at all — but it is an optimization, not a prerequisite for flying.

What stays true and worth keeping from the measurement:

• the flat-param budget is real (MAX_FLAT_PARAMS = 16; VehicleState.nav alone is 16 f32), so the seam shape decides whether any work memory is needed — that constraint is now with jess (jess#167);
• cascade.wit crosses zero list/string, so params are the only realloc trigger;
• a growing allocator is still unacceptable — that part of fix(rate): bounded arena instead of a growing allocator — remove memory.grow #327 (killing memory.grow, which blocks meld fuse --memory shared) was the real fix and is unaffected by this correction.

[avrabe]

A stronger reason to migrate than the arena: fixed-length lists — and they are what make allocation-free MESSAGING expressible.

The control path (falcon:cascade) is allocation-free because it crosses only flat f32 records. The messaging path is not, and the WIT says so plainly:

// wit/relay-transport/secure-channel.wit — the inter-core carrier
type commandmessage = list<u8>;
type secureframe    = list<u8>;

// wit/interfaces/relay-common-types/types.wit
record envelope         { …  data: list<u8> }
record scheduled-action { …  payload: list<u8> }
record table-update     { table-name: string, … }

A variable-length list<u8> across a component boundary means the receiver allocates — the host calls the guest's cabi_realloc to place the bytes. That is exactly what the embedded SW cannot do, and it is independent of what gale's ring buffer does underneath: the interface shape decides whether allocation is required.

Fixed-length lists are the WIT-level answer — and our toolchain can't express them

list<T, N> is a first-class WIT feature. Measured:

	list<u8, 8>
WIT spec / wasm-tools	parses ✅
wit-bindgen 0.41 (what relay uses)	expected '>', found ',' ❌
the fork, 0.58	supported, with tests/runtime/fixed-length-lists/ and tests/codegen/named-fixed-length-list.wit ✅

So the migration in this issue is not just about cabi-realloc-extern (#4) or disable_cabi_realloc (#6). Without 0.58 we cannot even declare a bounded payload, which means the messaging seams stay allocation-requiring by construction. That is a much stronger motivation than swapping an allocator.

Open question I have NOT answered — do not assume

A fixed-length list only avoids allocation if its flattened size stays within MAX_FLAT_PARAMS (16). list<u8, 8> fits; list<u8, 64> almost certainly spills to linear memory and would need cabi_realloc again — same mechanism as the 18-param rate.tick spill. So fixed-length lists are necessary but perhaps not sufficient for a realistically-sized frame.

That is measurable the same way the param spill was — declare list<u8, 8> and list<u8, 64> in one component under 0.58, build, and compare whether cabi_realloc is reachable. Worth doing before committing to a payload size, not after.

If large frames do spill, the remaining options are (a) pass a handle/index into gale's ring rather than the bytes, or (b) stream<u8> with gale's ring as the transport — relay's actual thesis, and the one whose MCU lowering behaviour I genuinely don't know yet.

Revised value of this migration

• unlocks fixed-length lists → bounded payloads are expressible at all (new, and the big one)
• unlocks disable_cabi_realloc (Add Kani harnesses for stream-transformer state machines (LC, SCH, SC, HS, CFDP) #6) → elide the forced export once the seam is flat
• unlocks cabi-realloc-extern (End-to-end verification chain: Verus → Rocq → Lean → Kani for all five engines #4) → gale-owned arena where allocation genuinely is needed

Cost is unchanged: wit-bindgen 0.41 → 0.58 plus a crate restructure, on the flight path.

[avrabe]

Correction — for messaging the arena IS the answer, so the open question above is not a blocker.

Settled rule, stated plainly so it stops being re-litigated:

Bounded static work memory (an arena) is acceptable. A heap / anything that grows is not.

Applied to the two paths:

path	shape	allocation
control (falcon:cascade)	flat f32 records; params ≤ 16 after the seam reshape	none needed → elide cabi_realloc (#6)
messaging (relay-transport, envelopes, frames)	bounded payloads that must land somewhere	arena is correct → cabi_realloc → gale's __cabi_arena_realloc (#4)

So my previous comment's worry — "a list<u8, 64> would spill and need cabi_realloc again" — is not a problem. Spilling into a bounded, gale-owned arena is exactly what that arena is for. A messaging seam that lands a frame in work memory is behaving correctly; what must never happen is the allocator growing linear memory (memory.grow), which is the thing that actually blocks meld fuse --memory shared --address-rebase.

That also means the fallbacks I floated (pass a ring handle instead of bytes; stream<u8> transport) are optimizations, not requirements. Worth considering for zero-copy, not needed for correctness.

Both wit-bindgen features are therefore wanted, for different paths — not alternatives:

• Add Kani harnesses for stream-transformer state machines (LC, SCH, SC, HS, CFDP) #6 disable_cabi_realloc — control path, where nothing needs to land anywhere
• End-to-end verification chain: Verus → Rocq → Lean → Kani for all five engines #4 cabi-realloc-extern — messaging path, so the bytes land in ONE embedder-owned arena rather than a per-component one

Fixed-length lists (0.58) remain valuable regardless: a bounded payload lets the arena be sized and defended, instead of an unbounded list<u8> whose worst case is unknown. That is the real argument for them — bounding the work memory, not avoiding it.

[avrabe]

Measured today: one of the two motivations for consuming the fork is now available upstream. This issue’s motivation (the gale-owned arena) is unaffected — but OCI-P03 currently bundles both, and it should not.

What I measured

SWREQ-FALCON-OCI-P03 justifies adopting the fork partly on fixed-length lists: "our wit-bindgen 0.41 rejects list<u8, 8> (expected >, found ,)". That was true for 0.41. It is no longer the state of upstream:

tool	type payload = list<u8, 8>
wasm-tools 1.243.0	parses ✅
wit-bindgen 0.41	parse error (per the artifact)
wit-bindgen 0.43	parses, then panics in codegen — not yet implemented, crates/core/src/types.rs:206
wit-bindgen 0.60.0	generates cleanly ✅

And it generates the right thing — a bounded array, not a heap allocation:

pub type Payload = [u8; 8];

Vec< occurrences in the generated guest code: 0.

Upstream’s remaining gap is bytecodealliance/wit-bindgen#1501, "Fixed Size List code can’t handle non-POD elements". Our bounded payloads are list<u8, N> — u8 is POD — so that limitation does not bind us. It would bind a list<some-record, N>.

What this does and does not change

Does: the fixed-length-list half of OCI-P03 no longer requires the fork. type commandmessage = list<u8, N> / secureframe, envelope.data, scheduled-action.payload can be bounded on upstream — a version bump (plus the wit-bindgen-rt → wit-bindgen crate restructure, which we owe regardless: the components still pin wit-bindgen-rt 0.41 while the workspace is already on wit-bindgen 0.59).

Does not: this issue’s actual subject. The gale-owned __cabi_arena_realloc delegation (fork #4, open) and the zero-heap scalar buffer (fork #5, open) are allocator concerns that upstream does not address. Fork #6 (grow-able allocator behind cabi_realloc on flat-only worlds) is closed.

Caveats — what I did NOT test

• Only the standalone CLI generator. The components build through cargo-component; the generated-bindings path may behave differently and is worth confirming before committing.
• No end-to-end build of an actual component against 0.60.
• Whether 0.60 changes anything about the no-grow property we rely on.

Suggested consequence: split OCI-P03 — bound the messaging payloads on upstream (cheap, unblocks the arena sizing this platform’s "bounded work memory, no heap" rule requires), and keep fork consumption scoped to the allocator work tracked here. That removes fork maintenance from the critical path for bounded payloads.

🤖 Generated with Claude Code