Skip to content

feat(server,cli): auto-renew statement-store allowances - #308

Draft
pgherveou wants to merge 186 commits into
mainfrom
pg/statement-allowance-renewal
Draft

feat(server,cli): auto-renew statement-store allowances#308
pgherveou wants to merge 186 commits into
mainfrom
pg/statement-allowance-renewal

Conversation

@pgherveou

Copy link
Copy Markdown
Collaborator

Why

Statement-store allowances are registered per UTC-day period (period = unix_seconds / 86400) and die at the boundary, ~12h average lifetime plus a ~1h grace, after which product statement submissions fail with "no allowance set for account". Per RFC-0010, keeping an allowance alive is the Account Holder's job; in the headless world the signing host plays that role, but it only registered allowances on demand for the current period and never renewed. This ports the mobile app's auto-renewal model into the shared core.

Design

Rule: core owns what to renew and how; the host owns only when.

 tick  (min(1h, next UTC-day boundary + 2min))
  ├─ no active session? ──────► skip
  ├─ read ledger, resolve targets against current root entropy
  ├─ fetch metadata + chain state + ring   (once per tick)
  └─ per target, under the registration lock:
       scan slots ── AlreadyAllocated ► fresh no-op
                 └── Free(seq) ► ring-VRF proof ► submit ► inBlock
                 └── exhausted ► stop pass, report slots_exhausted
  • statement_allowance/renewal.rs — chain-pure pass reusing the existing register_statement_account orchestrator. No persisted (period, seq): chain state is the truth via the AlreadyAllocated fast path.
  • signing_host/allowance_renewal.rs — target ledger persisted through CoreStorage (new StatementRenewalTargets slot). Entries are derivation recipes (WalletSso, ProductStatementAllowance{product_id}) so they survive the CLI's root-entropy rotation on slot exhaustion; only pairing peer device keys are raw account ids. Driver (renew_now) + idempotent loop spawned via Spawner (weak refs only, dies with the runtime; futures/futures-timer only, no tokio in core).
  • A registration lock serializes the pass (held per target, so on-demand allocation waits at most one registration) with allocate_statement_store_allowance, which now also records its product target in the ledger.
  • Public API on SigningHostRuntime (native-only): renew_statement_allowances() (primary one-pass entry, what a mobile WorkManager/BGTaskScheduler shim would call over FFI), start_statement_allowance_renewal(), track_statement_renewal_targets(), next_statement_renewal_delay().

CLI

  • build_signing_runtime starts the renewal loop (covers startup and /session switches, all modes).
  • Pairing registration returns the device key and tracks [WalletSso, device] in the ledger.
  • New /renew command (interactive + signing-host exec '/renew') as the manual escape hatch: prints per-target outcomes and, on slot exhaustion, marks auto-managed accounts exhausted so the next pairing rotates.

Tests

  • Tick-delay boundary math, outcome folding (mid-list failure continues; exhaustion skips the rest).
  • Ledger round-trip / dedupe / order, strict decode, recipe resolution matches the sso_responder derivations.
  • /renew parse accept/reject; codegen golden updated for the new CoreStorageKey variant.
  • Gates: cargo +nightly fmt --check, clippy --workspace --all-targets --all-features -D warnings, cargo test --workspace, cargo check --target wasm32-unknown-unknown -p truapi-server (renewal is native-only; on web the paired mobile app is the Account Holder).

Follow-ups (out of scope)

  • Bulletin long-term-storage renewal (additive ProductBulletinAllowance ledger variant).
  • Core→host exhaustion callback so the background loop (not just /renew) can trigger account rotation.
  • Move register_pairing_allowances into core to close the residual CLI-side registration race (benign: duplicate submits rescan).

pgherveou added 30 commits June 30, 2026 17:30
Adds the canonical testing module (api/testing.rs) and its v01/v02/versioned
wiring used by the Rust host runtime and generated clients.
New crate defining the host syscall traits (storage, navigation, consent,
permissions, ...) that host runtimes implement. Types are re-exported from
truapi::versioned/v01 rather than redefined.
WASM host runtime that hosts implement: dispatcher, SCALE frames, subscription
streams, chain runtime, host logic (sessions, SSO pairing, permissions,
statement store, dotns) and the wasm bindings. Includes the committed generated
dispatcher/wire-table under src/generated/.
…backs

Extends the rustdoc-JSON code generator to emit the Rust dispatcher and wire
table consumed by truapi-server, plus the TS host-callbacks adapter. Golden
tests pin the emitted shapes.
New WASM-backed host runtime package embedding the Rust core, with web iframe
and Web Worker entry points. Updates the @parity/truapi client (SCALE, sandbox,
transport) and drops the obsolete explorer 0.3.2 codegen snapshot.
Updates CLAUDE.md/README, CI workflows, Makefile, deny.toml, changesets, and
linguist attributes for generated code, and bumps the dotli submodule to the
host integration that consumes the WASM runtime.
Adds the canonical testing module (api/testing.rs) and its v01/v02/versioned
wiring used by the Rust host runtime and generated clients.
New crate defining the host syscall traits (storage, navigation, consent,
permissions, ...) that host runtimes implement. Types are re-exported from
truapi::versioned/v01 rather than redefined.
…backs

Extends the rustdoc-JSON code generator to emit the Rust dispatcher and wire
table consumed by truapi-server, plus the TS host-callbacks adapter. Golden
tests pin the emitted shapes.
pgherveou added 14 commits July 21, 2026 18:50
BREAKING CHANGE: use bare /script for the editable getUserId template, or run whoami.ts explicitly by path.
Statement-store allowances are registered per UTC-day period and die at
the boundary (~12h average lifetime plus a 1h grace), after which product
statement submissions fail with "no allowance set for account". RFC-0010
assigns keeping them alive to the Account Holder; for the headless host
that is the signing host, which until now only registered allowances on
demand for the current period.

Core (truapi-server):
- statement_allowance/renewal: chain-pure pass that re-registers every
  resolved target for the current period (AlreadyAllocated = fresh no-op,
  chain state is the truth), continues past per-target failures, stops on
  slot exhaustion, and computes the next tick delay
  (min(1h, period boundary + 2min)).
- signing_host/allowance_renewal: durable target ledger persisted via
  CoreStorage (new StatementRenewalTargets slot) storing derivation
  recipes (//wallet//sso, //allowance//statement-store//{product}) so
  entries survive root-entropy rotation, plus raw device keys; renew_now
  driver and an idempotent Spawner-based loop holding only weak refs.
- A registration lock serializes the renewal pass (per target) with
  on-demand allocation; allocate_statement_store_allowance now records
  its product target in the ledger.
- SigningHostRuntime gains track_statement_renewal_targets,
  renew_statement_allowances (primary one-pass API, FFI-friendly for
  mobile OS schedulers), start_statement_allowance_renewal, and
  next_statement_renewal_delay.

CLI (truapi-host):
- The signing host starts the renewal loop at runtime construction and
  tracks the wallet-sso + peer device targets after pairing registration.
- New /renew command (interactive and `signing-host exec '/renew'`)
  runs one pass, prints per-target outcomes, and marks auto-managed
  accounts exhausted on slot exhaustion so the next pairing rotates.
@valentunn

valentunn commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Note that mobile does that from a worker: doing so when app is running is not 100% reliable since the slot might expire when the app is not running
We should account that in the rust core: we want to be able to iniate & run stuff from the worker. One idea: rust core to send a command to native that it wants to schedule a specific / general purpose worker worker? We can (potentially, not sure) make "initiate worker" generic if rust core would pass opaque command data that worker will pass back to the core in onWorkerStarted or something like that

…e-renewal

# Conflicts:
#	Cargo.lock
#	Makefile
#	README.md
#	deny.toml
#	docs/issue-drafts/bulletin-preimage-in-core.md
#	explorer/diagnosis-reports/pairing-host-cli.md
#	explorer/diagnosis-reports/signing-host-cli.md
#	playground/tests/e2e/dotli/helpers/signer-bot.ts
#	rust/crates/truapi-codegen/tests/golden/host-callbacks.ts
#	rust/crates/truapi-host-cli/Cargo.toml
#	rust/crates/truapi-host-cli/README.md
#	rust/crates/truapi-host-cli/SPEC.md
#	rust/crates/truapi-host-cli/js/battery-reporter.ts
#	rust/crates/truapi-host-cli/js/diagnosis.test.ts
#	rust/crates/truapi-host-cli/js/diagnosis.ts
#	rust/crates/truapi-host-cli/js/runner.ts
#	rust/crates/truapi-host-cli/js/scripts/battery.ts
#	rust/crates/truapi-host-cli/js/ws-provider.ts
#	rust/crates/truapi-host-cli/src/accounts.rs
#	rust/crates/truapi-host-cli/src/attestation.rs
#	rust/crates/truapi-host-cli/src/chain.rs
#	rust/crates/truapi-host-cli/src/frame_server.rs
#	rust/crates/truapi-host-cli/src/main.rs
#	rust/crates/truapi-host-cli/src/network.rs
#	rust/crates/truapi-host-cli/src/platform.rs
#	rust/crates/truapi-host-cli/src/script_runner.rs
#	rust/crates/truapi-host-cli/src/sessions.rs
#	rust/crates/truapi-host-cli/src/signing_shell.rs
#	rust/crates/truapi-host-cli/src/terminal_ui.rs
#	rust/crates/truapi-host-cli/tests/signing_host_cli.rs
#	rust/crates/truapi-platform/src/lib.rs
#	rust/crates/truapi-server/README.md
#	rust/crates/truapi-server/src/chain_runtime.rs
#	rust/crates/truapi-server/src/host_core.rs
#	rust/crates/truapi-server/src/host_logic/attestation.rs
#	rust/crates/truapi-server/src/host_logic/product_account.rs
#	rust/crates/truapi-server/src/host_logic/sso/messages.rs
#	rust/crates/truapi-server/src/host_logic/sso/messages/v1.rs
#	rust/crates/truapi-server/src/host_logic/sso/pairing.rs
#	rust/crates/truapi-server/src/host_logic/statement_store.rs
#	rust/crates/truapi-server/src/host_logic/statement_store/statement.rs
#	rust/crates/truapi-server/src/host_logic/transaction.rs
#	rust/crates/truapi-server/src/lib.rs
#	rust/crates/truapi-server/src/runtime.rs
#	rust/crates/truapi-server/src/runtime/bulletin_rpc.rs
#	rust/crates/truapi-server/src/runtime/signing_host.rs
#	rust/crates/truapi-server/src/runtime/signing_host/local_activation.rs
#	rust/crates/truapi-server/src/runtime/signing_host/ring_vrf.rs
#	rust/crates/truapi-server/src/runtime/signing_host/sso_responder.rs
#	rust/crates/truapi-server/src/runtime/statement_allowance.rs
#	rust/crates/truapi-server/src/runtime/statement_allowance/extension.rs
#	rust/crates/truapi-server/src/runtime/statement_allowance/extrinsic.rs
#	rust/crates/truapi-server/src/runtime/statement_allowance/proof.rs
#	rust/crates/truapi-server/src/runtime/statement_allowance/ring.rs
#	rust/crates/truapi-server/src/runtime/statement_allowance/rpc.rs
#	rust/crates/truapi-server/src/runtime/statement_allowance/slot.rs
#	rust/crates/truapi-server/src/runtime/statement_store_rpc.rs
#	rust/crates/truapi/src/api/chain.rs
The checked-in golden had 208 lines re-indented with tabs, which also moved
rustfmt's line wrapping in three statements because a tab counts as one
column for width. `golden_host_callbacks_ts` compares byte-for-byte, so the
Rust workspace job failed with exit 101.

Regenerated from the emitter, so the file is whatever codegen produces.

The tabs entered in the `origin/main` merge, not in a feature commit, so
nothing about the renewal work is involved.
Two places turned a single unusable entry into a total failure.

`renew_now` collected `resolve_target` over the whole ledger with `?`, so one
entry whose derivation failed meant no target was renewed that tick, and the
next tick failed the same way. Unresolvable entries are now skipped with a
warning and the rest of the pass proceeds, matching how the pass already
tolerates a per-target registration failure.

`read_targets` propagated a SCALE decode error, so a ledger this build cannot
read stopped renewal permanently — the one state that cannot recover on its
own. An undecodable ledger now reads as empty and is rebuilt by the next
allocation or pairing, which is how `allowances.rs` already handles a corrupt
allowance-key blob.
…them

`CoreStorageKey::StatementRenewalTargets` is one global slot, unlike
`AllowanceKeys { session_id }`, and nothing clears it. Derivation recipes are
fine that way — that is the point of storing recipes, and they resolve under
whatever root entropy is active, so they survive the CLI's rotation on slot
exhaustion.

A raw `Account` target does not re-derive. Left unscoped it outlives the
identity that promised it, and the next identity renews it: proving its own
ring membership, claiming its own free slot, and spending that slot to keep a
previous account's pairing peer allowed. With ten Lite slots per period on
paseo, a handful of stale device entries can exhaust a fresh identity's table
before any of its own products are served.

Ledger entries now carry the root public key that promised them, set only for
raw account ids. A pass renews the entries owned by the active identity and
drops the rest from the ledger as it goes, so the cost is paid once per
identity change rather than every tick. Two identities may each promise the
same account; they are distinct entries and each renews it for itself.

`StatementRenewalTarget` is unchanged, so `track_statement_renewal_targets`
and the CLI keep their signatures: the owner comes from the active root
entropy inside the core, which is the only place that knows it.

The encoding change is why the preceding commit made an undecodable ledger
read as empty — an existing ledger is discarded and rebuilt rather than
stopping renewal.
Whether a renewal pass stops early and reports `slots_exhausted` was decided
by `reason.contains("no free StatementStore slot")`, matching the rendered
text of `SlotError::NoFreeStatementStoreSlot`. Rewording that error silently
turns exhaustion into an ordinary per-target failure: the pass keeps walking
the ledger, every remaining target fails the same way, and the report claims
the host still has slots.

The per-target result now carries a typed `RenewalFailure` whose
`slots_exhausted` is read off the error variant, so the two decisions —
stop the pass, flag the report — come from the same place the error was
constructed.

`main.rs:1135` matches the same string across the `anyhow` boundary on the
pairing-registration path. That one is left alone: `register_pairing_allowances`
discards the typed error deliberately, so fixing it is a CLI signature change
rather than part of this behaviour.
`track_targets` read the ledger, appended, and wrote it back with nothing
holding the three steps together. Two products allocating an allowance at the
same time both read the same ledger, each append their own target, and the
second write drops the first — leaving a product allocated but unrenewed until
its next allocation happens to re-add it.

A `ledger_lock` alongside the existing `registration_lock` covers the cycle,
and the identity prune in `renew_now` takes it too, since that is the other
read-modify-write on the same slot.

`allowance_renewal::track` is now the single way in: it derives the owner from
the active root entropy and takes the lock, so `SigningHost` and the
`sso_responder` allocation path share one implementation instead of each
assembling the arguments. `owner_key` and `track_targets` go back to private.
`PERIOD_BOUNDARY_MARGIN` exists so a pass runs after the chain has rotated
into the new period, but the hourly cap was applied last and could undo it.
From exactly one hour before a boundary, `min(3600 + 120, 3600)` scheduled the
tick on the boundary itself — the local clock reports the new period while the
chain may still be in the old one, so the pass scans slots for a period the
chain does not agree on and every target fails.

The cap now only applies while the boundary is more than an hour away. Inside
that hour the delay is the full wait plus the margin, so a tick either lands
before a boundary — harmless, it renews the current period — or at least
`PERIOD_BOUNDARY_MARGIN` past it.

Covered by a test that walks every second of the last two hours before a
boundary and asserts no tick lands in `[boundary, boundary + margin)`.
Brings the branch up to `aa878d1f`, so the PR is mergeable again.

One conflict, in `host_core.rs`, between two additions to the same region:
this branch's native-only `impl SigningHostRuntime` carrying the renewal entry
points, and `ConnectionAdapters` from the Chat integration (#326). Both are
kept; neither references the other.

The rest of main merged cleanly, including the FFI type rename (#345),
Extrinsic V5 signing (#333), and RFC-0026 chain discovery (#354, #358).

Verified on the merge result: `cargo +nightly fmt --check`, `clippy
--workspace --all-targets --all-features -D warnings`, 683 workspace tests,
and `cargo check --target wasm32-unknown-unknown -p truapi-server`.
Auditing the earlier fixes against "does a test fail without it", two had
nothing holding them: the ledger lock and the skip-on-unresolvable path.

`concurrent_tracks_do_not_drop_an_entry` drives two `track_targets` calls
through `futures::join!` over a storage that yields after serving a read, so
both observe the same ledger before either writes. Without the lock the second
write drops the first entry — the assertion reports `[b.dot]` where
`[a.dot, b.dot]` is expected. A first attempt at this test yielded *before*
the read, which let the cycles serialize by luck and passed with the lock
removed; the yield has to sit between read and write to expose the race.

`an_unresolvable_target_is_skipped_not_fatal` uses a 25-digit product id,
which fails junction derivation with `NumericJunctionOutOfRange` — the only
way `resolve_target` errors in practice. It asserts both that a strict
`collect::<Result<_, _>>` loses the healthy target alongside the broken one,
and that `resolve_targets` keeps it.

`resolve_targets` is split out of `renew_now` for that test; the pass would
otherwise need a live chain to exercise the filter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants