Skip to content

Commit fc55c41

Browse files
fix(dict): dict_remove no longer inflates the hash table exponentially
env_hash_rebuild blindly doubled the old capacity — right for the grow-on-insert callers (all rebuild at >70% load), catastrophic for dict_remove's re-index rebuild: N removes on one dict grew its table by 2^N, so ~25 insert/remove cycles of a single key allocated gigabytes and aborted OOM. Size the new table from the live entry count instead — identical doubling behavior at the grow sites, no inflation on remove. Surfaced by liferaft's per-message pending-registry churn during the #523 task-layer migration; regression-pinned with a 200-cycle churn test (impossible pre-fix, instant post-fix) plus a survivors-resolve check in tests/test_dict.eigs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 2ca5f5e commit fc55c41

3 files changed

Lines changed: 42 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,14 @@ All notable changes to EigenScript are documented here.
191191
(the one legitimate empty-expression position) still yields `null` (#494).
192192

193193
### Fixed
194+
- **`dict_remove` no longer inflates the dict's hash table exponentially.**
195+
The re-index rebuild after a removal reused the grow path's blind
196+
capacity-doubling, so N removes on one dict grew its table by 2^N —
197+
~25 insert/remove cycles of a single key allocated gigabytes and OOMed
198+
the process. The rebuild now sizes the table from the live entry count
199+
(identical doubling behavior on the >70%-load grow path). Surfaced by
200+
liferaft's per-message pending-registry churn during the #523 task-layer
201+
migration; regression-pinned in `tests/test_dict.eigs` (200-cycle churn).
194202
- **`args` now rides the trace tape (#471).** The `args` builtin returned
195203
`argv` directly, unwrapped — the last un-taped nondeterminism source
196204
reachable by a pure script, and a hole in the closed-world invariant behind

src/eigenscript.c

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1291,8 +1291,15 @@ void env_hash_insert(EnvHash *ht, uint32_t h, int idx) {
12911291
}
12921292

12931293
static void env_hash_rebuild(EnvHash *ht, char **names, int count) {
1294-
int new_cap = (ht->mask + 1) * 2;
1295-
if (new_cap < ENV_HASH_INIT_CAP) new_cap = ENV_HASH_INIT_CAP;
1294+
/* Size from the live entry count, never by blindly doubling the old
1295+
* capacity. Every grow-path caller rebuilds at >70% load, so 2x-count
1296+
* lands on the same doubling as before — but dict_remove rebuilds to
1297+
* RE-INDEX after a removal, and blind doubling there inflated a
1298+
* shrinking dict's table by 2^N over N removes: ~25 removes of a single
1299+
* key OOMed the process (surfaced by liferaft's per-message registry
1300+
* churn during the #523 task-layer migration). */
1301+
int new_cap = ENV_HASH_INIT_CAP;
1302+
while (new_cap < count * 2) new_cap *= 2;
12961303
free(ht->hashes);
12971304
free(ht->indices);
12981305
free(ht->generations);

tests/test_dict.eigs

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,4 +59,29 @@ assert_eq of [app.db.port, 5432, "nested dot number"]
5959
msg is f"name={d.name}"
6060
assert_eq of [msg, "name=eigen", "dict in f-string"]
6161

62+
# Repeated insert/remove on one dict must not inflate its hash table: the
63+
# re-index rebuild after a remove used to blindly DOUBLE capacity, going
64+
# exponential (2^N after N removes -- OOM at ~25; caught by liferaft's
65+
# per-message registry churn, #523 migration). 200 cycles is impossible
66+
# pre-fix and instant post-fix.
67+
churn is {}
68+
ci is 0
69+
loop while ci < 200:
70+
churn["k"] is ci
71+
dict_remove of [churn, "k"]
72+
ci is ci + 1
73+
assert_eq of [ci, 200, "200 insert/remove cycles complete (no exponential rebuild)"]
74+
assert_eq of [len of churn, 0, "churned dict ends empty"]
75+
76+
# Same churn against a dict that keeps OTHER live keys: removes re-index the
77+
# survivors, lookups must stay correct throughout.
78+
churn2 is {"keep1": 1, "keep2": 2}
79+
ci is 0
80+
loop while ci < 50:
81+
churn2["tmp"] is ci
82+
dict_remove of [churn2, "tmp"]
83+
ci is ci + 1
84+
assert_eq of [churn2.keep1 + churn2.keep2, 3, "surviving keys resolve after churn"]
85+
assert_eq of [len of churn2, 2, "only survivors remain"]
86+
6287
test_summary of null

0 commit comments

Comments
 (0)