local-store: add contains, len and count metadata queries - #527
Closed
lgahdl wants to merge 1 commit into
Closed
Conversation
get/list-keys force a guest to copy a full payload across the wasm boundary to answer questions that don't need the payload: existence checks marshal the whole value, size queries fetch it, prefix counts materialise the whole key list. Add three metadata queries that answer without transferring bytes: - contains(key) -> bool: existence without fetching the value. - len(key) -> option<u64>: value byte length, none if absent. - count(prefix) -> u64: key cardinality without materialising the list. The redb backend answers contains/len from the entry in place and count from a bounded range scan (no key strings materialised). The StateHandle trait and the SDK's LocalStoreHost trait ship default bodies over get/list_keys, so existing backends and mocks keep compiling; the redb backend and both mock crates additionally override with the cheap path. Doc comments flag that len/count may still be a scan on backends that can't do better (per the issue's needs-design note on naming/complexity), rather than promising O(1). Closes #291
Collaborator
Author
|
Duplicated: #457 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nexum:host/local-storeonly exposesget/set/delete/list-keys, so a guest answering "does this key exist," "how big is this value," or "how many keys match this prefix" has to transfer the full payload (or full key list) across the wasm boundary just to discard it.Adds three metadata queries, all answering without transferring the bytes:
contains(key) -> boollen(key) -> option<u64>(byte length,noneif absent)count(prefix) -> u64(key cardinality)Design
wit/nexum-host/local-store.wit: the three new funcs.StateHandle(crates/nexum-runtime/src/host/component/state.rs) and the SDK'sLocalStoreHosttrait (crates/nexum-sdk/src/host.rs) both gain default bodies overget/list_keys, so existing implementors keep compiling unchanged.local_store_redb.rs) overrides all three with cheap paths:contains/lenread the entry in place without copying the value out;countis a bounded B-tree range scan that never materialises key strings.needs-designnote: doc comments saylen/count"may be a scan" rather than promising O(1), since not every backend can beat the default.crates/nexum-runtime/src/host/impls/local_store.rs(host trait impl),crates/nexum-sdk/src/wit_bindgen_macro.rs(guest-side macro), and both mock crates (nexum-sdk-test,shepherd-sdk-test) with matching overrides + tests.Testing
contains/len/count, including namespace isolation) and fornexum-sdk-test'sMockLocalStore(including fault-injection interaction).nexum-sdk/src/host.rsverifyingcontains/len/countderive correctly from only the four required trait methods.cargo check --workspace,cargo clippy --all-targets -- -D warnings(on the touched crates), andcargo fmt --all -- --checkall pass.Closes #291