Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,16 +90,13 @@ Pin material is **plan / batch only** (`batch_pin`, `BatchParents`). No process
create pin FIFO. IBD confirm intake
is **body queue wire only** → lookup → load.

**RecentCreates** is a write-published Arc layer chain (same splice as
live_union): identity **and** `CreatePin` outs after Class A+idx. Each layer
`until = lookup_started_hi.max(hi)` at publish; drop when `class_a_hi >= until`.
Not BQ depth, not EWMA. Stamp attaches the CreatePin; pin does not re-walk the
ring. Not a coins cache or process pin FIFO.
In-flight CreatePin layers drop at drain+fence **and**, for pin layers, write
`until = lookup_started_hi.max(hi)` then `class_a_hi >= until`. Stamp attaches
the CreatePin from `InFlightView`. Not a coins cache or process pin FIFO.

Leftover union, stage IO, S0–S4: **[`docs/invariants.md`](docs/invariants.md)**
(the only Allowed/Forbidden IO table). In-flight prune after pin + scripts
handoff; no leftover pending / pin FIFO; RecentCreates drops when Class A
covers lookup-started. Union miss is permanent.
handoff; no leftover pending / pin FIFO. Union miss is permanent.

### Confirm pipeline timers

Expand Down
13 changes: 12 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,12 @@ before 1.0).

### Changed

- **In-flight keep-until:** pin layers stay until drain+fence **and**
`class_a_hi >= until` (`until = lookup_started_hi.max(hi)` frozen at
write). Stamp walks `InFlightView` only (then live_union, then TipOnly).
[`docs/invariants.md`](docs/invariants.md),
[`docs/concurrency.md`](docs/concurrency.md).

- **`header.head` open-grow replaces the file:** an undersized single-gen
OA is deleted and recreated at the create target (`.mlt` kept). Crash
after unlink is a missing `header.head`, not a zeroed live table.
Expand Down Expand Up @@ -169,6 +175,11 @@ before 1.0).

### Removed

- **RecentCreates ring and `PipelineParentStore`:** write no longer clones
a second identity+outs layer list; IBD never used the Weak pin registry.
`ibd: sizes` `recent=` / `pstore=` stay 0. CreatePin outs live on
in-flight keep-until and batch-local `BatchParents`.

- **Dead production APIs:** SH catalog materialize is always k-way (no
fan-in reduce / CHECKPOINT / READY, no `RBITCOIN_SH_MERGE_FANIN` /
`TARGET_RUN_BYTES` / `MAX_DIRECT_MERGE`). One catalog write policy (no
Expand All @@ -180,7 +191,7 @@ before 1.0).
Unused `NodeClock::mock_value`, `ChainParams::min_difficulty_target`,
`outbound_for_ibd`, `InvalidHashSet::{mark_path,is_invalid_fn}`,
`MempoolHub::mining_frontier_snapshot`, `ConfirmParentCache::get_header_plan_arc`,
`NodeConfig::with_datadir_cold`. RecentCreates identity **and** outs stay.
`NodeConfig::with_datadir_cold`.

### Fixed

Expand Down
2 changes: 1 addition & 1 deletion OPERATOR.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ onto loadq). Tip-batch getdata races up to 4 peers
half-median outlier only after ~60s warm-up and only when the peer pack is not
tight (max/min bps > 2×); good-but-slightly-slower peers are kept.

**Create pins:** pipeline-local only (`batch_pin` / `BatchParents`). No process pin FIFO on IBD (no `PipelineParentStore`). Header plans via ConfirmParentCache. Just-confirmed **identity + full create outs** live in RecentCreates Arc layers (drop when Class A covers lookup-started). Not a coins cache.
**Create pins:** pipeline-local only (`batch_pin` / `BatchParents`). No process pin FIFO. Header plans via ConfirmParentCache. Just-confirmed **identity + full create outs** stay on in-flight until write keep-until (`class_a_hi >= lookup_started_hi.max(hi)`). Not a coins cache.

**Archive `tx.head` split (perf_dbg):** `plan_batch … head_rd=` is parent
**read** resolve (`get_fk_by_txid_batch`, with `probe` / `idx` / `body` subtimers).
Expand Down
3 changes: 0 additions & 3 deletions crates/rbitcoin-consensus/src/confirm_run/bq_resolve.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1057,7 +1057,6 @@ mod tests {
&[g_cb.to_byte_array()],
&rbitcoin_query::InFlightView::empty(),
published.as_ref(),
q.recent_creates().as_ref(),
)
.expect("stamp helper after wave");
assert_eq!(
Expand Down Expand Up @@ -1198,7 +1197,6 @@ mod tests {
parent_hash: None,
next_tx_start: q.tx_body_count().saturating_add(1).max(1),
in_flight: view,
parent_store: None,
published: std::sync::Arc::new(rbitcoin_query::PublishedIds::new()),
};
let items = [(Height(1), std::sync::Arc::new(b1), None)];
Expand Down Expand Up @@ -1292,7 +1290,6 @@ mod tests {
parent_hash: None,
next_tx_start: q.tx_body_count().saturating_add(1).max(1),
in_flight: log.snapshot(),
parent_store: None,
published: std::sync::Arc::new(rbitcoin_query::PublishedIds::new()),
};
let items = [(Height(1), std::sync::Arc::new(b1), None)];
Expand Down
6 changes: 1 addition & 5 deletions crates/rbitcoin-consensus/src/confirm_run/lookup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use super::*;
pub struct DenserelsWarmStats {
/// Unique external parent creates considered (stamped create_fk, not same-batch).
pub parents: u32,
/// Already covered via in-flight / same-batch / RecentCreates outs / pstore adopt.
/// Already covered via in-flight / same-batch / stamp-carried outs.
pub already: u32,
/// Cold denserels body loads (`txout` by stamped range). Always 0 on the
/// shipped pin path — range-fill is `PIN_NEW`, not this field.
Expand Down Expand Up @@ -172,7 +172,6 @@ pub(super) fn stamp_parent_pin_archived(
&need_vec,
ifo,
query.published_ids(),
query.recent_creates(),
)
.map_err(ConsensusError::from)?;
let mut stamp = ParentPinStamp {
Expand Down Expand Up @@ -235,15 +234,13 @@ pub fn confirm_wire_load_from_plan(
} = stamped;

let ifo = pipeline.map(|p| &p.in_flight);
let parent_store = pipeline.and_then(|p| p.parent_store.as_ref());
let (batch_parents, spend_edges, _warm) = pin_for_wire_batch(
query,
plan.as_ref(),
&mut parent_pin,
&metas,
&wire_blocks,
ifo,
parent_store,
)?;
if let Some(ref mut p) = plan {
p.freeze_after_pin();
Expand Down Expand Up @@ -850,7 +847,6 @@ mod tests {
&[parent_txid],
&rbitcoin_query::InFlightView::empty(),
q.published_ids(),
q.recent_creates(),
)
.expect("shared helper");

Expand Down
12 changes: 1 addition & 11 deletions crates/rbitcoin-consensus/src/confirm_run/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,7 @@ pub use scripts::{
};
pub use write::confirm_write_phase;
#[cfg(test)]
use write::{
recent_create_height_slices, recent_create_rows_for_slices, write_batch_vs_tip,
write_height_needed, WriteBatchVsTip,
};
use write::{write_batch_vs_tip, write_height_needed, WriteBatchVsTip};

/// Pure-write annotate backend from global `RBITCOIN_IO`.
#[inline]
Expand Down Expand Up @@ -156,11 +153,6 @@ pub struct WireLoadPipeline {
/// Load looks up create fk / full CreatePin for parents still only in the
/// pipeline (body-ahead-of-head). Built via [`rbitcoin_query::InFlightLog::snapshot`].
pub in_flight: rbitcoin_query::InFlightView,
/// Pipeline-wide sparse parent pin store (Weak map; load get-or-insert only).
/// `None` on IBD (RecentCreates outs + in-flight). Tip-follow may still set
/// `Some`. Batches hold `Arc` handles so concurrent stages share one payload
/// per create when a store is present.
pub parent_store: Option<std::sync::Arc<rbitcoin_query::PipelineParentStore>>,
/// Lookup-published parent identity union (wave hits still live in the BQ window).
pub published: std::sync::Arc<rbitcoin_query::PublishedIds>,
}
Expand Down Expand Up @@ -439,7 +431,6 @@ pub fn confirm_wire_load_phase_pipelined(
let ns_filter_plan = t_fp.elapsed().as_nanos() as u64;

let inflight = pipeline.map(|p| &p.in_flight);
let parent_store = pipeline.and_then(|p| p.parent_store.as_ref());
let mut parent_pin = match plan.as_mut() {
Some(p) => ParentPinStamp::take_from_plan(p),
None => stamp_parent_pin_archived(query, params, &metas, &wire_blocks, inflight)?,
Expand All @@ -451,7 +442,6 @@ pub fn confirm_wire_load_phase_pipelined(
&metas,
&wire_blocks,
inflight,
parent_store,
)?;
if let Some(ref mut p) = plan {
p.freeze_after_pin();
Expand Down
32 changes: 5 additions & 27 deletions crates/rbitcoin-consensus/src/confirm_run/pin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use super::*;

/// Pin parents for wire load: **only spent parents** (sparse outs).
///
/// Sources: plan/in-flight offline denserels → RecentCreates create_pin
/// Sources: plan/in-flight offline denserels → stamp-carried CreatePin
/// **txout body by range** from [`ParentPinStamp`] (lookup-stamped). Load never
/// reads head / `tx.idx` / `txid.body`. Load **copies** lookup-stamped
/// `spent_range` onto pins. Write [`ensure_spend_abs_layouts`] is holes-only.
Expand All @@ -15,7 +15,6 @@ pub(super) fn pin_for_wire_batch(
metas: &[BodyMeta],
wire_blocks: &[Arc<Block>],
in_flight: Option<&rbitcoin_query::InFlightView>,
pipeline_parent_store: Option<&std::sync::Arc<rbitcoin_query::PipelineParentStore>>,
) -> Result<
(
rbitcoin_query::BatchParents,
Expand Down Expand Up @@ -168,18 +167,7 @@ pub(super) fn pin_for_wire_batch(
confirm_load_stats::PIN_RECENT_OUTS_NS.fetch_add(recent_outs_ns, Ordering::Relaxed);
}

let mut batch_parents = match pipeline_parent_store {
Some(store) => rbitcoin_query::BatchParents::with_store(
std::sync::Arc::clone(store),
parent_vouts.len(),
),
None => rbitcoin_query::BatchParents::with_capacity(parent_vouts.len()),
};
let t_adopt = Instant::now();
if pipeline_parent_store.is_some() {
batch_parents.adopt_from_store(parent_vouts.keys().copied());
}
let adopt_ns = t_adopt.elapsed().as_nanos() as u64;
let mut batch_parents = rbitcoin_query::BatchParents::with_capacity(parent_vouts.len());
let thin_ns = t_thin.elapsed().as_nanos() as u64;
if thin_ns > 0 {
confirm_load_stats::THIN_NS.fetch_add(thin_ns, Ordering::Relaxed);
Expand All @@ -190,7 +178,7 @@ pub(super) fn pin_for_wire_batch(
let t_plan = Instant::now();
for (id, need) in &parent_vouts {
let fk = rbitcoin_primitives::Fk(*id);
// Pure adopt hit: refresh meta only when plan/layout material is present
// Same-batch / in-flight pin: refresh meta only when plan/layout material is present
// (skip empty refresh_pin_meta — it would reload outs).
if !need.is_empty() && batch_parents.pin_covered(fk, need) {
if let Some(pin) = plan_by_id.get(id) {
Expand Down Expand Up @@ -351,10 +339,6 @@ pub(super) fn pin_for_wire_batch(
}
let contract_ns = t_contract.elapsed().as_nanos() as u64;

let t_publish = Instant::now();
batch_parents.publish_to_store();
let publish_ns = t_publish.elapsed().as_nanos() as u64;

let n_unique = parent_vouts.len() as u64;
if n_unique > 0 {
confirm_load_stats::PARENT_UNIQUE.fetch_add(n_unique, Ordering::Relaxed);
Expand All @@ -370,25 +354,19 @@ pub(super) fn pin_for_wire_batch(
if plan_pin_ns > 0 {
confirm_load_stats::PLAN_PIN_NS.fetch_add(plan_pin_ns, Ordering::Relaxed);
}
if adopt_ns > 0 {
confirm_load_stats::PIN_ADOPT_NS.fetch_add(adopt_ns, Ordering::Relaxed);
}
if contract_ns > 0 {
confirm_load_stats::PIN_CONTRACT_NS.fetch_add(contract_ns, Ordering::Relaxed);
}
if publish_ns > 0 {
confirm_load_stats::PIN_PUBLISH_NS.fetch_add(publish_ns, Ordering::Relaxed);
}
// Last-batch pin residual for slow-load logs (overwrite; not window-summed).
let cold_batch_ns = cold_range_batch_ns
.saturating_add(cold_io_ns)
.saturating_add(cold_decode_ns);
confirm_load_stats::note_last_pin(
adopt_ns,
0,
plan_pin_ns,
cold_batch_ns,
contract_ns,
publish_ns,
0,
n_plan_pin,
n_cold.saturating_add(n_range_new),
);
Expand Down
Loading
Loading