From 7c0cff9a0032caf6f9f8b6f61119370e50543879 Mon Sep 17 00:00:00 2001 From: Ilja Heitlager Date: Thu, 27 Aug 2026 22:49:55 +0200 Subject: [PATCH 1/2] fix: use the correct index max_local threshold, not table's MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit local_payload_size computed max_local as usable_size - 35 unconditionally, but SQLite defines a smaller max_local for index cells (leaf AND interior) than for table leaf cells: (usable_size - 12) * 64 / 255 - 23 vs usable_size - 35. Every index cell whose payload landed between the two thresholds was read with a local_size far larger than what SQLite actually reserved on the page. Found while closing 006-btree Req 7's documented coverage gap: adding overflow_index_key.db (an ~8000-byte indexed TEXT key) immediately hit PayloadTooShort, since the payload cleared the correct (smaller) index threshold while staying under the wrong (larger) table one that had been masking the bug. Fixing the threshold surfaced a second latent bug: index delete (src/btree/index/delete.rs) never freed a removed entry's overflow chain at all (table delete already does). Index entries essentially never overflowed under the old, too-generous threshold, so the gap was never exercised. Added free_overflow_chain there, wired into both delete_from_leaf and the interior-match outright-delete path (remove_entry_by_child). spend: ~2x the trivial fixture-generation estimate — the fixture work surfaced two real correctness bugs (wrong max_local, leaked index overflow chains on delete) that needed fixing, not just a missing fixture. --- .openspec/specs/006-btree/spec.md | 18 +++- src/btree.rs | 63 +++++++---- src/btree/index.rs | 5 +- src/btree/index/delete.rs | 101 ++++++++++++++++-- src/btree/index/insert.rs | 3 +- src/btree/table/insert.rs | 3 +- tests/corpus/btree_test.rs | 23 ++++ .../fixtures/btrees/overflow_index_key.db | Bin 0 -> 24576 bytes tools/gen_fixtures.sh | 8 ++ 9 files changed, 188 insertions(+), 36 deletions(-) create mode 100644 tests/corpus/fixtures/btrees/overflow_index_key.db diff --git a/.openspec/specs/006-btree/spec.md b/.openspec/specs/006-btree/spec.md index f5b5301a..1fc454c8 100644 --- a/.openspec/specs/006-btree/spec.md +++ b/.openspec/specs/006-btree/spec.md @@ -175,19 +175,19 @@ The system SHOULD provide enough key ordering to walk an index b-tree correctly **Tests:** `src/btree/index.rs::secondary_index_seek_matches_oracle` -### Requirement 7: No Fixture Yet Covers an Overflowing Index Key [SHOULD] +### Requirement 7: Overflowing Index Keys Use the Index (Not Table) Local-Size Threshold [MUST] -The originating issue's corpus section expects an index fixture with an overflowing key; `tests/corpus/fixtures/btrees/index.db`'s actual generated content (`b TEXT`, max length 15 bytes) does not exercise this — overflow reassembly on index cells reuses the same `reassemble_payload` function already proven byte-identical against table-cell overflow (Requirement 2), so the residual risk is low, but this is a real coverage gap, not a silent oversight. +The originating issue's corpus section expected an index fixture with an overflowing key; none existed (`tests/corpus/fixtures/btrees/index.db`'s content — `b TEXT`, max length 15 bytes — never exercised it), flagged as a real coverage gap rather than a silent oversight. Building `overflow_index_key.db` (an ~8000-byte indexed TEXT key against a 4096-byte page) surfaced that the gap was hiding an actual bug, not just missing coverage: `local_payload_size`'s `max_local` was computed as `usable_size - 35` unconditionally, but SQLite defines a smaller `max_local` for index cells (leaf AND interior) — `(usable_size - 12) * 64 / 255 - 23` — than for table leaf cells. Every index cell whose payload fell between the two thresholds was read with a `local_size` far larger than what SQLite actually reserved on the page, corrupting the read (`PayloadTooShort`) the moment a fixture forced a payload past the *correct* (smaller) index threshold while still under the incorrect (larger) table one. The system MUST select `max_local` by cell kind, not just by whether the payload overflows. -**Implementation:** `tools/gen_fixtures.sh` (planned — no fixture with an overflowing index key exists yet) +**Implementation:** `src/btree.rs::local_payload_size` (takes an `is_index` flag); `tools/gen_fixtures.sh` (`overflow_index_key.db`) #### Scenario: Overflowing index key -- GIVEN an index whose key column is large enough to overflow into one or more overflow pages +- GIVEN an index whose key column is large enough to overflow into one or more overflow pages under the index (not table) local-size threshold - WHEN the cursor reads that entry - THEN the reassembled key payload MUST be byte-identical to the pinned oracle's value -**Tests:** `tools/gen_fixtures.sh` (planned) +**Tests:** `tests/corpus/btree_test.rs::overflowing_index_key_reassembles_byte_identical_to_oracle` ### Requirement 8: Leaf Cell Insert Without Split [MUST] @@ -419,6 +419,14 @@ The system MUST delete the entry with a given key from an index b-tree. Deleting **Tests:** `tests/corpus/btree_index_insert_delete_test.rs::delete_all_entries_leaves_an_empty_index` +#### Scenario: Deleting an entry with an overflowing value frees its overflow chain + +- GIVEN an index entry whose key/value overflows into one or more overflow pages (per Requirement 7's corrected threshold) +- WHEN that entry is deleted, whether directly from a leaf or removed outright by the interior-match path (Requirement 17) +- THEN every page in its overflow chain MUST be returned to the freelist, not leaked — found via Requirement 7's fixture work, since index cells overflow far more readily under the corrected (smaller) threshold than the previous bug allowed + +**Tests:** `src/btree/index/delete.rs::tests::deleting_an_entry_with_overflow_frees_its_overflow_chain`, `src/btree/index/delete.rs::tests::deleting_all_entries_orphans_no_page` + ### Requirement 17: Interior-Match Deletion via Predecessor Swap [MUST] Because index interior cells carry a full entry (Requirement 5), deleting a key that was promoted to interior level by an earlier split MUST NOT simply remove that routing entry — its child pointer is load-bearing, and removing the entry would also discard whichever value it carries. The system MUST instead find that entry's in-order predecessor (the maximum entry within its own left-child subtree, found by recursively descending — preferring the rightmost subtree, falling back to an interior page's own last entry once its rightmost subtree is confirmed drained) and swap the predecessor's value into the matched entry's position, physically removing the predecessor from wherever it actually lived. If the matched entry's subtree is entirely drained (no predecessor available), the entry is removed outright instead. diff --git a/src/btree.rs b/src/btree.rs index c6c8d7ff..8ba2aad1 100644 --- a/src/btree.rs +++ b/src/btree.rs @@ -279,6 +279,7 @@ impl TableCursor

{ &cur.page, cur.tail_start, cur.payload_len, + false, ) } @@ -606,14 +607,24 @@ impl TableCursor

{ } /// SQLite's overflow local-size formula (fileformat2.html "Cell Payload -/// Overflow"), shared by table leaf cells, index leaf cells, and index -/// interior cells (table interior cells have no payload at all). All -/// arithmetic saturates rather than panics — a pathological `usable_size` -/// degrades to a safe (wrong but non-panicking) answer, caught by the -/// length checks around the call site instead of an arithmetic panic -/// here. -fn local_payload_size(usable_size: u32, payload_len: u64) -> u64 { - let max_local = usable_size.saturating_sub(35) as u64; +/// Overflow"). `min_local` is shared by every cell kind, but `max_local` +/// is NOT: table leaf cells use `usable_size - 35`, while index cells +/// (leaf AND interior — table interior cells have no payload at all) use +/// `(usable_size - 12) * 64 / 255 - 23`, a smaller threshold. Passing the +/// wrong one for an index cell computes a `local_size` that doesn't fit +/// the space SQLite actually reserved on the page — caught in practice +/// once a fixture forces an index cell's payload past the *correct* +/// (smaller) index threshold while still under the table one (#Req 7). +/// All arithmetic saturates rather than panics — a pathological +/// `usable_size` degrades to a safe (wrong but non-panicking) answer, +/// caught by the length checks around the call site instead of an +/// arithmetic panic here. +fn local_payload_size(usable_size: u32, payload_len: u64, is_index: bool) -> u64 { + let max_local = if is_index { + ((usable_size.saturating_sub(12) as u64).saturating_mul(64) / 255).saturating_sub(23) + } else { + usable_size.saturating_sub(35) as u64 + }; if payload_len <= max_local { return payload_len; } @@ -643,8 +654,9 @@ fn first_overflow_page( payload_len: u64, usable_size: u32, page_num: u32, + is_index: bool, ) -> Result, BtreeError> { - let local_size = local_payload_size(usable_size, payload_len) as usize; + let local_size = local_payload_size(usable_size, payload_len, is_index) as usize; if (local_size as u64) < payload_len { Ok(Some(read_u32( buf, @@ -702,6 +714,7 @@ fn reassemble_payload( page: &Rc<[u8]>, tail_start: usize, payload_len: u64, + is_index: bool, ) -> Result { if payload_len > MAX_PAYLOAD_LEN { return Err(BtreeError::PayloadTooLarge { @@ -712,7 +725,7 @@ fn reassemble_payload( let cell_tail = page .get(tail_start..) .ok_or(BtreeError::PayloadTooShort { page_num })?; - let local_size = local_payload_size(usable_size, payload_len) as usize; + let local_size = local_payload_size(usable_size, payload_len, is_index) as usize; let local_bytes = cell_tail .get(..local_size) .ok_or(BtreeError::PayloadTooShort { page_num })?; @@ -877,9 +890,14 @@ fn free_btree_pages_inner( let ptr_off = cell_ptr_offset(ptr_base, i); let cell_start = read_cell_pointer(&buf, ptr_off, page_num, i)?; let (_, payload_len, tail_start) = decode_cell_head(&buf, cell_start, page_num)?; - if let Some(overflow_page) = - first_overflow_page(&buf, tail_start, payload_len, usable_size, page_num)? - { + if let Some(overflow_page) = first_overflow_page( + &buf, + tail_start, + payload_len, + usable_size, + page_num, + false, + )? { free_overflow_chain(pager, page_num, overflow_page, visited)?; } } @@ -900,7 +918,7 @@ fn free_btree_pages_inner( let (payload_len, tail_start) = index::decode_payload_len(&buf, cell_start, page_num)?; if let Some(overflow_page) = - first_overflow_page(&buf, tail_start, payload_len, usable_size, page_num)? + first_overflow_page(&buf, tail_start, payload_len, usable_size, page_num, true)? { free_overflow_chain(pager, page_num, overflow_page, visited)?; } @@ -916,7 +934,7 @@ fn free_btree_pages_inner( let (payload_len, tail_start) = index::decode_payload_len(&buf, value_start, page_num)?; if let Some(overflow_page) = - first_overflow_page(&buf, tail_start, payload_len, usable_size, page_num)? + first_overflow_page(&buf, tail_start, payload_len, usable_size, page_num, true)? { free_overflow_chain(pager, page_num, overflow_page, visited)?; } @@ -1022,7 +1040,7 @@ pub(super) fn collect_leaf_cells( let ptr_off = cell_ptr_offset(ptr_base, i); let cell_start = read_cell_pointer(buf, ptr_off, page_num, i)?; let (rowid, payload_len, tail_start) = decode_cell_head(buf, cell_start, page_num)?; - let local_size = local_payload_size(usable_size, payload_len) as usize; + let local_size = local_payload_size(usable_size, payload_len, false) as usize; let has_overflow = (local_size as u64) < payload_len; let cell_end = tail_start .saturating_add(local_size) @@ -1063,7 +1081,7 @@ pub(super) fn scan_leaf_cells( if cell_rowid > rowid && insert_pos == num_cells { insert_pos = i; } - let local_size = local_payload_size(usable_size, payload_len) as usize; + let local_size = local_payload_size(usable_size, payload_len, false) as usize; let has_overflow = (local_size as u64) < payload_len; let cell_end = tail_start .saturating_add(local_size) @@ -1097,7 +1115,7 @@ pub(super) fn find_leaf_cell( if cell_rowid != rowid { continue; } - let local_size = local_payload_size(usable_size, payload_len) as usize; + let local_size = local_payload_size(usable_size, payload_len, false) as usize; let overflow_page = if (local_size as u64) < payload_len { read_u32(buf, tail_start.saturating_add(local_size), page_num)? } else { @@ -1526,7 +1544,7 @@ pub(super) fn splice_delete_cell( } else { index::decode_payload_len(buf, cell_start, page_num)? }; - let local_size = local_payload_size(usable_size, payload_len) as usize; + let local_size = local_payload_size(usable_size, payload_len, !has_rowid) as usize; let has_overflow = (local_size as u64) < payload_len; let cell_end = tail_start .saturating_add(local_size) @@ -2326,7 +2344,7 @@ mod tests { // denom` remainder on opposite sides of a denom (508) multiple, // making the two paths diverge to entirely different results (70 // vs 167) instead of coincidentally agreeing. - assert_eq!(local_payload_size(512, 5150), 70); + assert_eq!(local_payload_size(512, 5150, false), 70); } #[test] @@ -2335,7 +2353,8 @@ mod tests { pages: HashMap::new(), }; let page: Rc<[u8]> = Rc::from(Vec::new().as_slice()); - let err = reassemble_payload(&source, 512, 2, &page, 0, MAX_PAYLOAD_LEN).unwrap_err(); + let err = + reassemble_payload(&source, 512, 2, &page, 0, MAX_PAYLOAD_LEN, false).unwrap_err(); assert!(!matches!(err, BtreeError::PayloadTooLarge { .. })); } @@ -2361,7 +2380,7 @@ mod tests { let mut cell = Vec::new(); cell.extend_from_slice(&encode_varint_for_test(5000)); cell.extend_from_slice(&encode_varint_for_test(1)); - let local_size = local_payload_size(512, 5000) as usize; + let local_size = local_payload_size(512, 5000, false) as usize; cell.extend(std::iter::repeat_n(0u8, local_size)); cell.extend_from_slice(&99u32.to_be_bytes()); page[cell_start..cell_start.saturating_add(cell.len())].copy_from_slice(&cell); diff --git a/src/btree/index.rs b/src/btree/index.rs index 994ac216..013627c7 100644 --- a/src/btree/index.rs +++ b/src/btree/index.rs @@ -360,6 +360,7 @@ impl IndexCursor

{ &frame.page, tail_start, payload_len, + true, )?; Ok(IndexRow { payload }) } @@ -388,6 +389,7 @@ impl IndexCursor

{ &frame.page, tail_start, payload_len, + true, )?; Ok(IndexRow { payload }) } @@ -525,7 +527,7 @@ fn decode_value_cell( encoding: TextEncoding, ) -> Result<(Vec, Vec), BtreeError> { let (payload_len, tail_start) = decode_payload_len(buf, value_start, page_num)?; - let local_size = local_payload_size(usable_size, payload_len) as usize; + let local_size = local_payload_size(usable_size, payload_len, true) as usize; let has_overflow = (local_size as u64) < payload_len; let cell_end = tail_start .saturating_add(local_size) @@ -542,6 +544,7 @@ fn decode_value_cell( &page, tail_start, payload_len, + true, )?; let key = decode_record(&payload, encoding)?; Ok((key, cell_bytes)) diff --git a/src/btree/index/delete.rs b/src/btree/index/delete.rs index ebd0de01..79147e37 100644 --- a/src/btree/index/delete.rs +++ b/src/btree/index/delete.rs @@ -41,16 +41,62 @@ use std::cmp::Ordering; use crate::btree::index::{ build_index_interior_cell, collect_index_interior_entries, collect_index_leaf_cells, - compare_keys, descend_index_tree, write_index_interior_page, IndexDescent, INTERIOR_INDEX, - LEAF_INDEX, + compare_keys, decode_payload_len, descend_index_tree, write_index_interior_page, IndexDescent, + INTERIOR_INDEX, LEAF_INDEX, }; use crate::btree::{ - page1_header_start, read_page_type, splice_delete_cell, BtreeError, MAX_PAGES_VISITED, + local_payload_size, page1_header_start, read_page_type, read_u32, splice_delete_cell, + BtreeError, MAX_PAGES_VISITED, }; use crate::header::DatabaseHeader; use crate::pager::Pager; use crate::record::{TextEncoding, Value}; +/// Returns the first overflow page of a value cell's raw bytes (`0` if +/// its payload is entirely local). `value_bytes` is the verbatim +/// `payload-length varint + local bytes [+ 4-byte overflow pointer]` +/// shape [`collect_index_leaf_cells`]/[`collect_index_interior_entries`] +/// already extract — decoding it directly (rather than re-reading from a +/// live page) works because that shape is self-contained, with the +/// payload-length varint at offset 0. +fn overflow_page_of(value_bytes: &[u8], usable_size: u32) -> Result { + let (payload_len, tail_start) = decode_payload_len(value_bytes, 0, 0)?; + let local_size = local_payload_size(usable_size, payload_len, true) as usize; + if (local_size as u64) < payload_len { + Ok(read_u32( + value_bytes, + tail_start.saturating_add(local_size), + 0, + )?) + } else { + Ok(0) + } +} + +/// Walks and frees an overflow-page chain starting at `first_page` (a +/// no-op if it's `0` — the cell had no overflow). Mirrors +/// `table::delete::free_overflow_chain` (duplicated rather than shared — +/// see `insert.rs`'s `write_overflow_chain` doc comment for why). +fn free_overflow_chain(pager: &mut Pager, first_page: u32) -> Result<(), BtreeError> { + let mut page_num = first_page; + let mut visited = std::collections::HashSet::new(); + while page_num != 0 { + if !visited.insert(page_num) { + return Err(BtreeError::OverflowChainCycle { + page_num: first_page, + revisited_page: page_num, + }); + } + let next = { + let buf = pager.get_page_mut(page_num)?; + read_u32(buf, 0, page_num)? + }; + pager.deallocate_page(page_num)?; + page_num = next; + } + Ok(()) +} + /// Deletes the entry with exactly `key` (via `compare_keys`) from the /// index b-tree rooted at `root_page`. Returns `Err(BtreeError::KeyNotFound)` /// if no such entry exists, leaving the tree unchanged. See the module @@ -95,9 +141,17 @@ fn delete_from_leaf( .iter() .position(|(existing_key, _)| compare_keys(existing_key, key) == Ordering::Equal) .ok_or(BtreeError::KeyNotFound)?; + let overflow_page = overflow_page_of( + &cells + .get(pos) + .ok_or(BtreeError::Internal("delete_from_leaf: pos out of bounds"))? + .1, + usable_size, + )?; let buf = pager.get_page_mut(leaf_page)?; - splice_delete_cell(buf, header_start, leaf_page, usable_size, pos, false) + splice_delete_cell(buf, header_start, leaf_page, usable_size, pos, false)?; + free_overflow_chain(pager, overflow_page) } /// Handles a delete target found at interior level — see the module doc @@ -258,13 +312,15 @@ fn remove_entry_by_child( page_num, child: child_to_remove, })?; - entries.remove(idx); + let (_, _, removed_value_bytes) = entries.remove(idx); + let overflow_page = overflow_page_of(&removed_value_bytes, usable_size)?; let cell_bytes: Vec> = entries .iter() .map(|(c, _, value_bytes)| build_index_interior_cell(*c, value_bytes)) .collect(); let buf = pager.get_page_mut(page_num)?; - write_index_interior_page(buf, header_start, page_num, &cell_bytes, rightmost) + write_index_interior_page(buf, header_start, page_num, &cell_bytes, rightmost)?; + free_overflow_chain(pager, overflow_page) } #[cfg(test)] @@ -336,6 +392,39 @@ mod tests { assert!(cells.is_empty()); } + /// Regression test: index cells use a smaller `max_local` than table + /// leaf cells (`(usable_size-12)*64/255-23`, not `usable_size-35` — + /// 006-btree Requirement 7 flagged this via a real fixture), so a key + /// well under the table threshold can still overflow here. Deleting + /// such an entry must free its overflow chain, not leak it — mirrors + /// `table::delete::tests::deleting_a_row_with_overflow_frees_its_overflow_chain`. + #[test] + fn deleting_an_entry_with_overflow_frees_its_overflow_chain() { + let page_size = 512u32; + let (vfs, header) = minimal_index_db(page_size); + let mut pager = Pager::open(&vfs, Path::new("/test.db"), page_size).unwrap(); + + // `max_local` for a 512-byte page is (512-12)*64/255-23 = 102, so + // 300 bytes of key text forces an overflow chain. + let big_key = key(&"x".repeat(300), 1); + insert_entry(&mut pager, &header, 1, &big_key, TextEncoding::Utf8).unwrap(); + + let freelist_before = freelist_page_count(&mut pager); + assert_eq!(freelist_before, 0); + delete_entry(&mut pager, &header, 1, &big_key, TextEncoding::Utf8).unwrap(); + let freelist_after = freelist_page_count(&mut pager); + + assert!( + freelist_after > freelist_before, + "the entry's overflow chain must be returned to the freelist on delete" + ); + } + + fn freelist_page_count(pager: &mut Pager) -> u32 { + let page1 = pager.get_page_mut(1).unwrap().clone(); + u32::from_be_bytes(page1[36..40].try_into().unwrap()) + } + #[test] fn deleting_one_of_two_entries_keeps_the_other() { let page_size = 512u32; diff --git a/src/btree/index/insert.rs b/src/btree/index/insert.rs index 45f8803b..99cc119a 100644 --- a/src/btree/index/insert.rs +++ b/src/btree/index/insert.rs @@ -86,7 +86,8 @@ fn encode_index_cell( payload: &[u8], ) -> Result, BtreeError> { let payload_len = payload.len() as u64; - let local_size = (local_payload_size(usable_size, payload_len) as usize).min(payload.len()); + let local_size = + (local_payload_size(usable_size, payload_len, true) as usize).min(payload.len()); let (local_bytes, overflow_bytes) = payload.split_at(local_size); let mut cell = encode_varint(payload_len); cell.extend_from_slice(local_bytes); diff --git a/src/btree/table/insert.rs b/src/btree/table/insert.rs index d59edaa1..a9604faf 100644 --- a/src/btree/table/insert.rs +++ b/src/btree/table/insert.rs @@ -71,7 +71,8 @@ fn encode_leaf_cell( payload: &[u8], ) -> Result, BtreeError> { let payload_len = payload.len() as u64; - let local_size = (local_payload_size(usable_size, payload_len) as usize).min(payload.len()); + let local_size = + (local_payload_size(usable_size, payload_len, false) as usize).min(payload.len()); let (local_bytes, overflow_bytes) = payload.split_at(local_size); let mut cell = encode_varint(payload_len); cell.extend(encode_varint(rowid as u64)); diff --git a/tests/corpus/btree_test.rs b/tests/corpus/btree_test.rs index 89e664e6..b953d407 100644 --- a/tests/corpus/btree_test.rs +++ b/tests/corpus/btree_test.rs @@ -88,3 +88,26 @@ fn without_rowid_row_count_matches_oracle() { } assert_eq!(n, 500); } + +/// 006-btree Requirement 7: `overflow_index_key.db`'s single index entry +/// has an ~8000-byte TEXT key against a 4096-byte page — the index cell +/// itself (not just the table row sharing the same column) overflows. +/// Index-cell overflow reuses `reassemble_payload` (Requirement 2), but +/// no fixture exercised that path on an index leaf until now. +#[test] +fn overflowing_index_key_reassembles_byte_identical_to_oracle() { + use sqlite_rs::record::{decode_record, TextEncoding, Value}; + + let mut cursor = open_index_cursor("overflow_index_key.db", 3); + let row = cursor.first().unwrap().unwrap(); + let values = decode_record(&row.payload, TextEncoding::Utf8).unwrap(); + // Index entries append the rowid as the record's trailing value. + let key = match &values[0] { + Value::Text(s) => s.as_ref(), + other => panic!("expected a TEXT index key, got {other:?}"), + }; + assert_eq!(key.len(), 8002, "prefix 'a-' + 8000 hex chars"); + assert!(key.starts_with("a-")); + assert!(key[2..].bytes().all(|b| b.is_ascii_hexdigit())); + assert!(cursor.next().unwrap().is_none()); +} diff --git a/tests/corpus/fixtures/btrees/overflow_index_key.db b/tests/corpus/fixtures/btrees/overflow_index_key.db new file mode 100644 index 0000000000000000000000000000000000000000..9358733e1de66143ae509746df765de2768e3e37 GIT binary patch literal 24576 zcmeI&O-jQ+6u|MBBnB#I%qrViwopI73mA1;gwkToqB}`wgn%C)v*<-UfG6+88m?l}5I_I{1Q0*~0R#|0 M009ILK%k`ppRN$%q5uE@ literal 0 HcmV?d00001 diff --git a/tools/gen_fixtures.sh b/tools/gen_fixtures.sh index 2b8867c1..b7cf5e36 100755 --- a/tools/gen_fixtures.sh +++ b/tools/gen_fixtures.sh @@ -256,6 +256,12 @@ CREATE TABLE t(a INTEGER, blb BLOB); INSERT INTO t VALUES(1, zeroblob(60000)); SQL +"$ORACLE" btrees/overflow_index_key.db <<'SQL' +CREATE TABLE t(a INTEGER, b TEXT); +CREATE INDEX idx_b ON t(b); +INSERT INTO t VALUES(1, 'a-' || hex(zeroblob(4000))); +SQL + "$ORACLE" btrees/select_parity.db <<'SQL' CREATE TABLE t(id INTEGER PRIMARY KEY, i INTEGER, s TEXT, r REAL, b BLOB); INSERT INTO t VALUES(1, NULL, NULL, NULL, NULL); @@ -273,6 +279,8 @@ index.db — an indexed column over 3000 rows, multi-page index b-tree. without_rowid.db — WITHOUT ROWID table, 500 rows. overflow_single_page.db — a 6000-byte blob, forces one overflow page. overflow_multi_page.db — a 60000-byte blob, forces a 14-page overflow chain. +overflow_index_key.db — a single row whose indexed TEXT column is ~8000 +bytes, forcing the index key itself (not just the table row) to overflow. select_parity.db — a plain INTEGER PRIMARY KEY table aimed at SELECT parity: NULL in every nullable column, empty string vs NULL, zero vs NULL, duplicate rows (for DISTINCT), mixed-case text (for NOCASE), and From 8221349be80f0055f37c87b798fcb97f9d53c4df Mon Sep 17 00:00:00 2001 From: Ilja Heitlager Date: Thu, 27 Aug 2026 23:04:02 +0200 Subject: [PATCH 2/2] docs: changelog entry for index max_local fix --- CHANGELOG.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 262d750f..1d6b2d76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,17 @@ All notable changes to sqlite-rs. Format follows [Keep a Changelog](https://keep pages were freed. Table leaf, index leaf, and index interior cells now have their first overflow page located and their whole chain walked and deallocated before the tree structure itself is freed. +- Index cells (leaf and interior) were reading their local-payload size + with the table leaf cell's `max_local` formula (`usable_size - 35`) + instead of the smaller one SQLite defines for index cells + (`(usable_size - 12) * 64 / 255 - 23`), corrupting reads on any index + cell whose payload landed between the two thresholds. Found while + closing 006-btree Req 7's documented "no overflowing-index-key + fixture" coverage gap — adding one immediately hit `PayloadTooShort`. + Fixing the threshold also surfaced that index entry delete never freed + a removed entry's overflow chain at all (table delete already did); + index entries essentially never overflowed under the old, too-generous + threshold, so the gap was never exercised. ### Docs