rpc: eth_getLogs and trace_filter resolve block tags on the overlay view but read data from the committed view

[yperbasis]

#23193 fixes debug_getModifiedAccountsByNumber, which resolved latest/pending on the overlay (fork-choice) view while scanning history on the committed view. The review of that PR found the same class of bug in two more endpoints.

Common setup: during an FCU background-commit window — after PublishOverlay makes block N+1 visible to overlay-aware readers, but before the batch is committed to the DB — overlay-aware tag resolution returns N+1 while the data reads only reach block N. The window is short-lived but recurs on every FCU.

eth_getLogs: spurious errors on the latest tag

resolveLogsRange resolves its baseline latest on the committed view (nil filters), but resolves user-supplied tags on the overlay view (api.filters):

erigon/rpc/jsonrpc/eth_receipts.go

Lines 158 to 196 in 70cb10d

	latest, _, _, err := rpchelper.GetBlockNumber(ctx, rpc.BlockNumberOrHashWithNumber(rpc.LatestExecutedBlockNumber), tx, api._blockReader, nil)
	if err != nil {
	return 0, 0, err
	}
	begin = latest
	if crit.FromBlock != nil {
	fromBlock := crit.FromBlock.Int64()
	if fromBlock > 0 {
	begin = uint64(fromBlock)
	} else {
	blockNum := rpc.BlockNumber(fromBlock)
	begin, _, _, err = rpchelper.GetBlockNumber(ctx, rpc.BlockNumberOrHashWithNumber(blockNum), tx, api._blockReader, api.filters)
	if err != nil {
	return 0, 0, err
	}
	}
	if checkFuture && begin > latest {
	return 0, 0, &rpc.CustomError{Message: ErrBlockRangeIntoFuture, Code: rpc.ErrCodeInvalidParams}
	}
	}
	end = latest
	if crit.ToBlock != nil {
	toBlock := crit.ToBlock.Int64()
	if toBlock > 0 {
	end = uint64(toBlock)
	} else {
	blockNum := rpc.BlockNumber(toBlock)
	end, _, _, err = rpchelper.GetBlockNumber(ctx, rpc.BlockNumberOrHashWithNumber(blockNum), tx, api._blockReader, api.filters)
	if err != nil {
	return 0, 0, err
	}
	}
	if checkFuture && end > latest {
	return 0, 0, &rpc.CustomError{Message: ErrBlockRangeIntoFuture, Code: rpc.ErrCodeInvalidParams}
	}
	}

With overlay head N+1 and committed execution progress N:

• eth_getLogs({"fromBlock":"latest"}) resolves begin = N+1 (api.filters, line 170) but latest = N (nil filters, line 158), so the checkFuture guard returns ErrBlockRangeIntoFuture.
• Even without that guard, GetLogs rejects begin > latestExecuted with "node is still syncing" (eth_receipts.go line 247).

So a plain latest logs poll — the most common eth_getLogs shape — fails transiently during every commit window.

Suggested fix, same as #23193: pass nil filters at lines 170 and 187 so user tags resolve on the same committed view that the guards and the log scan use. Trade-off (same as accepted in #23193): pending then resolves to the latest executed block instead of the txpool's pending block.

trace_filter: silently omits the overlay-head block

TraceAPIImpl.Filter resolves fromBlock/toBlock tags on the overlay view, then scans txnum indexes on the committed tx without any executed-progress guard:

erigon/rpc/jsonrpc/trace_filtering.go

Lines 325 to 356 in 70cb10d

	if req.FromBlock == nil {
	fromBlock = 0
	} else {
	fromBlock, _, _, err = rpchelper.GetBlockNumber(ctx, *req.FromBlock, dbtx, api._blockReader, api.filters)
	if err != nil {
	if errors.As(err, &rpc.BlockNotFoundErr{}) {
	stream.WriteEmptyArray()
	return nil // waiting for spec: not error for historical reasons
	}
	return err
	}
	}
	if req.ToBlock == nil {
	headNumber, err := api._blockReader.HeaderNumber(ctx, dbtx, rawdb.ReadHeadHeaderHash(dbtx))
	if err != nil {
	return err
	}
	toBlock = *headNumber
	} else {
	toBlock, _, _, err = rpchelper.GetBlockNumber(ctx, *req.ToBlock, dbtx, api._blockReader, api.filters)
	if err != nil {
	if errors.As(err, &rpc.BlockNotFoundErr{}) {
	stream.WriteEmptyArray()
	return nil // waiting for spec: not error for historical reasons
	}
	return err
	}
	}
	if fromBlock > toBlock {
	return errors.New("invalid parameters: fromBlock cannot be greater than toBlock")
	}

TxNums.Min/Max silently clamp a missing block to the last available txnum (see db/kv/rawdbv3/txnum.go). With overlay head N+1:

• trace_filter({"toBlock":"latest"}) resolves toBlock = N+1, clamps to block N's last txnum, and returns traces only up to block N. The response is a bare array that does not echo the resolved range, so an omitted head block looks identical to an empty one. An incremental indexer that advances its cursor to the requested head permanently skips block N+1's traces.
• {"fromBlock":"latest"} combined with an older numeric toBlock errors with "fromBlock cannot be greater than toBlock" instead.

trace_block/trace_transaction guard against this via rpchelper.CheckBlockExecuted in callBlock/callTransaction; Filter alone lacks the guard.

Suggested fix: resolve the tags with nil filters like #23193, and optionally add an executed-progress guard so an explicit out-of-range block number errors instead of silently clamping.

Related

debug_getModifiedAccountsByHash is missing the startNum > latestBlock guard its ByNumber twin has (only the two-param path checks endNum), so it returns a silent empty result for a not-yet-executed block where ByNumber errors. Worth aligning while touching this area.

[VBulikov]

yperbasis added Imp2 as Importance

[AskAlexSharov]

About Correctness problems of sharing Cache between exec and RPC:

tx - is an object which must guarantee consistent data view (it does now).
If other object (overlay) can break it - it's likely miss-use of overlay.

All consistency guaranties must be solved at compile time. For example:

• db.BeginTemporalRo() method can accept overlay as a param (then easy to find all cases where we pass nil)
• Or overlay must wrap db object - so all .BeginTemporalRo() calls auto-wrap underlying tx objet
• In any case: it must not be lazy-optional-binding - if dev forgot wrap tx we will catch it only in production in very rare cases. There must not be possible to forget.

Example of existing anti-pattern: ReceiptGenerator manually applying overlay to external tx object. Means 1 RPC method (eth_getReceipts) - See data in one places and doesn't see in another (because not all methods used in eth_getReceipts are )

Example of StateCache assuming that it used only in Execution (not by rpc) now: overlay's method IteratePrefix does by-pass all in-mem updates (breaking consistency).

My actionable advise:

• don't pass sd/overlay objects to any low-level methods. on highest-possible level - wrap it once (or not wrap - by any biz-logic-reason) - and pass down 1 wrapped tx object. Low-level methods must not care: do they work with mdbx's tx, temporal tx, overaly wrapped temporal tx: low-level methods already expecting quite-small interface - like kv.Getter
• we already have this rule in Erigon: low-level methods never open their own TX object. we even don't pass DB anywhere - means even if low-level method wanna open new tx - it can't - compiler protects us. This rule making all low-level methods composable means any high-level method - can use any low-level funcs, re-org them, and all will just work - high-level method will see consistent data view.

---

About Perf problems of sharing Cache between exec and RPC:

Exec and RPC doing very different queries:

• Execution need Get/Put on State (it can write inverted_indices to ETL because it never reading them).
• RPC need: Seek/Range on all data types (including inverted_indices).
  If Cache will serve RPC - it will slow-down execution (because you will need switch from map to btree, from etl to map+roaring_bitmaps)

here is no magic - maybe just separate concerns

---

[VBulikov]

yperbasis changed Importance from Imp2 to Imp1

[AskAlexSharov]

one more example: tx object used before got wrapped

func (api *APIImpl) GasPrice(ctx context.Context) (*hexutil.Big, error) {
	tx, err := api.db.BeginTemporalRo(ctx)
	if err != nil {
		return nil, err
	}
	defer tx.Rollback()
	oracle := api.newGasOracle(tx)
	tipcap, err := oracle.SuggestTipCap(ctx)
	if err != nil {
		return nil, err
	}
	gasResult := uint256.NewInt(0)
	gasResult.Set(tipcap)
	overlayTx := api.filters.WithTemporalOverlay(tx)
	if head := rawdb.ReadCurrentHeader(overlayTx); head != nil && head.BaseFee != nil {
		gasResult.Add(tipcap, head.BaseFee)
	}