Clear immer transience stamps during GC migration#1220
Conversation
…ing GC migration Immer's gc_transience_policy stamps nodes created by a transient with an ownership token so later operations by the same transient may mutate them in place. Tokens are addresses of small kore-heap allocations, and the young-generation bump allocator restarts at its semispace base on every collection, so token addresses are recycled once their owner is gone. Migration copied nodes verbatim, keeping stale stamps alive indefinitely; when a later hook's fresh token landed on a recycled address matching a stale stamp, can_mutate passed on a shared node and the hook mutated it in place, corrupting every collection value aliasing that node. Clear the stamp of every rbts node, relaxed struct, champ node, and champ values buffer visited during collection. No transient is live during GC (hooks complete within a rewrite step; GC runs between steps), so this only restores copy-on-write the next time a surviving node is touched. All immer stamp writes are creation-time or can_mutate-guarded, so a node cleared at its first migration can never be re-stamped, and within one GC epoch tokens are distinct bump allocations - stamps can no longer collide. Root-cause investigation: evm-semantics docs/2026-06-12-llvm-backend-immer-transience-bug.md (corrupted <txAuthList> tuple in an eip7702 conformance test). Related prior fix in the same family: #1219. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…amp recycling Links the real allocator, collector, and list hooks and arms the token-address collision by construction: build a 6-element list (the final concat's transient stamps the tail leaf), migrate it through two semispace flips so the young allocator returns to the stamp's semispace with the tail promoted to the old generation, pad the young space so the next token allocation lands exactly on the recycled stamp address, then run hook_LIST_range_long(l, 1, 0) - the compiled form of a 'ListItem(X) REST' match. An assertion on the result's tail stamp proves the collision armed. Without the stamp-clearing fix the kept list reads [2,3,4,5,6,6] (head dropped, last element duplicated - byte-for-byte the corruption observed in KEVM); with it the list is intact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Performance check: KEVM conformance suite, with vs without this fix Setup: two interpreters kompiled from the same KEVM semantics, differing only in the llvm-backend:
Functional side-check with the same two binaries, outside the benchmark suite: on the standalone repro extracted from the failing execution-spec-tests fixture ( |
gtrepta
left a comment
gtrepta
left a comment
There was a problem hiding this comment.
LGTM
While looking into it, I found this "gc_transience_policy" in the immer lib: https://github.com/arximboldi/immer/blob/master/immer/transience/gc_transience_policy.hpp
Looks like it may be intended for our GC use case, but I didn't investigate it any further. The fix we have now is probably good enough.
2 participants
This directly affects the evm-semantics, causing conformance tests to fail in runtimeverification/evm-semantics#2833.
The backend instantiates immer collections with
no_refcount_policy, which selectsgc_transience_policy.Under that policy, every transient operation (used by the List/Map/Set hooks) creates an ownership token and stamps the nodes it creates with it;
node::can_mutate(edit)is a raw pointer comparison against that stamp (refs.unique()is alwaysfalseunderno_refcount_policy).Three facts combine into memory corruption:
gc_transience_policy.hpp:make_token_()).migrate_collection_node→memcpy), so stale stamps survived collections, including promotion into the old generation.When a later hook's freshly allocated token lands on a recycled address equal to a surviving node's stale stamp,
can_mutatereturnstrueon a shared node, and the hook mutates it in place, corrupting every collection value that aliases that node.Observed in KEVM : a
ListItem(X) RESTmatch compiles tohook_LIST_range_long(L, 1, 0), whosedrop(1)took the in-place branch ofslice_left_mut_visitor::visit_leafand shifted a shared 6-element leaf from[e1..e6]to[e2..e6, e6].The collision requires an exact allocation-address coincidence, so the symptom is layout-sensitive and can affect any K semantics using collection hooks.
Related prior fix in the same family: #1219.
Fix
runtime/collect/migrate_collection.cpp: clear the transience stamp of every node visited during collection — rbts inner/leaf nodes, rbts relaxed structs, champ nodes, and champvaluesbuffers — using the node types' ownownee/edit_tAPI.can_mutate, so a node cleared at its first migration can never be re-stamped; old-generation nodes were cleared when migrated there. Within one GC epoch tokens are distinct bump allocations, so no collision is possible before first migration.can_mutatehits only occur within a single hook invocation, on nodes allocated after the last collection; those stamps are untouched. GC overhead is one 8-byte store per visited node.Regression test
unittests/runtime-gc(new targetruntime-gc-tests) links the real allocator, collector, and list hooks and arms the collision deterministically:build a 6-element list (final concat stamps the tail leaf), migrate it through two semispace flips (tail promoted to old generation, young allocator back at the stamp's semispace base), pad the young space so the next token lands exactly on the recycled stamp address, then run
hook_LIST_range_long(l, 1, 0).An assertion on the result's tail stamp proves the collision armed.
Without the fix, the kept list reads
[2,3,4,5,6,6]; with the fix, it is intact.Testing
runtime-gc-testsfails on the unfixed runtime, passes with the fix.ctest: all 7 unit-test targets pass.lit test/defn: 88/88 pass.failing-3.korerepro to completion with no stuckauthorizationListcheck.🤖 Generated with Claude Code