fix: make the quantity-zero tolerance scale-relative

docs/concepts/quantity-zero-tolerance.md

@@ -0,0 +1,164 @@
+# Quantity-zero tolerance
+
+When a fill closes a position the engine must decide whether the resulting
+quantity is *zero*. Floating-point arithmetic rarely lands on exactly `0.0`, so
+that decision needs a tolerance. This page states the contract that tolerance
+obeys.
+
+The implementation is `quantity_zero_tolerance()` in
+`ml4t/backtest/core/shared.py`.
+
+## The arithmetic that creates a residual
+
+A fill updates an existing position with one addition:
+
+```python
+old_qty = pos.quantity
+new_qty = old_qty + ctx.signed_qty        # execution/fill_executor.py
+```
+
+When the close is economically exact, `old_qty` and `ctx.signed_qty` are equal
+and opposite. Their magnitudes are then within a factor of two of each other, so
+by Sterbenz's lemma **the addition itself is exact**. It contributes no error.
+
+A non-zero `new_qty` therefore means the two operands were not exact negatives as
+float64 values. They were produced by different routes — the book quantity by
+accumulating prior fills, the closing quantity by sizing an order — and those
+routes round differently in the last bits. The residual is the gap between them.
+
+That gap is bounded by the spacing of float64 **at the magnitude of the
+quantities being cancelled**. It is not bounded by any fixed absolute quantity.
+
+## Controlling scale
+
+The scale is `max(|old_qty|, |signed_qty|)`: the magnitudes actually
+participating in the cancellation.
+
+It is emphatically **not** `|new_qty|`. `new_qty` is approximately zero by
+construction, so a tolerance derived from it would be approximately zero as well:
+`abs(new_qty) < k * ulp(abs(new_qty))` is false for every `k < 1` and every
+normal float. A relative tolerance taken against zero is mathematically
+ineffective, which is why the tolerance takes the *operands* rather than the
+result.
+
+`max()` rather than either operand alone, because the two are interchangeable in
+the closure case and `max()` is the only choice that is symmetric in them. It is
+also the correct upper bound on the last-place spacing of both.
+
+## Why the previous absolute epsilon failed
+
+The engine previously compared against an absolute `1e-12`. Float64 spacing grows
+with magnitude, so a single absolute threshold can only be correct over one
+binade:
+
+| position size | one ULP | caught by `1e-12`? |
+|---|---|---|
+| 4096 | 9.094947e-13 | yes |
+| 8191 | 9.094947e-13 | yes |
+| **8192** | **1.818989e-12** | **no** |
+| 10927 | 1.818989e-12 | no |
+| 874442 | 1.164153e-10 | no |
+
+At exactly 2<sup>13</sup> = 8192 units the spacing crosses `1e-12` and the rule
+stops working. From there upward a one-ULP residual survives, the position key is
+retained, and the engine reports a position that holds a quantity of the order of
+1e-12 shares.
+
+A **larger** fixed epsilon is not the repair. It moves the failure point to a
+larger magnitude without removing it — every absolute threshold is scale-blind
+somewhere — and it erases genuine small positions below the new threshold. The
+defect is the absoluteness, not the value.
+
+For the same reason the tolerance carries **no upper cap**. A ceiling is itself a
+fixed absolute value, so capping would reintroduce exactly the scale-blindness
+being removed.
+
+## The rule
+
+```
+scale = max(|operand|) over the finite operands
+tol   = QTY_ZERO_FLOOR                       if scale == 0
+        max(QTY_ZERO_FLOOR, QTY_ZERO_ULPS * ulp(scale))   otherwise
+
+QTY_ZERO_FLOOR = 1e-12
+QTY_ZERO_ULPS  = 16
+```
+
+A quantity counts as zero when `|q| <= tol`, and is open when `|q| > tol`. That
+single predicate holds at every site. The engine previously mixed `<` and `<=`
+against its absolute epsilon, so its sites disagreed with each other exactly at
+the threshold; they are unified here on the `<=` form the majority already used.
+
+### Why 16 ULP
+
+* **Mechanism.** The residual is the last-bits gap between the book quantity and
+  the order quantity. Order sizing (a target weight through equity and price, or
+  a re-read of the book) costs a small number of ULP; accumulating a position
+  over *n* fills adds at most `n/2` ULP of drift between the book value and the
+  exact sum.
+* **Margin over observation.** Across every retained run available for this
+  repair, every unsnapped residual is **exactly one ULP** of the operation scale.
+  16 ULP is a sixteen-fold margin over the worst case actually seen, and covers
+  roughly 32 accumulation steps at the worst-case half-ULP each.
+* **Relative size.** 16 ULP is `16 * 2**-52 ≈ 3.6e-15` of the operation scale —
+  the fifteenth significant digit. float64 carries about 15.95 significant
+  decimal digits, so a distinction that small cannot be *reliably* produced by
+  any float64 computation working at that scale.
+* **Headroom below a submittable order.** `OrderBook._MIN_ORDER_SIZE` is `1e-8`:
+  the engine refuses to submit anything smaller. At the largest operation scale
+  observed in practice (874,443 units) the tolerance is 1.86e-9 — still well
+  under that floor.
+
+### Why the absolute floor
+
+`ulp(scale)` collapses toward zero as the scale does, so a pure ULP rule would
+have no useful width for small positions and none at all at `scale == 0`. The
+floor keeps the historical absolute behaviour everywhere float64 spacing is finer
+than `1e-12`, which is every magnitude below 8192.
+
+This gives the migration a provable property: **a call with a single operand is
+exactly equivalent to the old `abs(q) < 1e-12` rule**. For any normal `x`,
+`16 * ulp(x) < x`, so the ULP term can never make such a predicate true on its
+own; only the floor can. Call sites that merely ask "is this book quantity zero?"
+therefore keep their previous behaviour bit for bit, while call sites that pass
+both closure operands gain the scale-aware rule.
+
+## Residue versus a genuine small position
+
+A **residue** is what remains after an economically exact close: its magnitude is
+a handful of ULP of the quantities that cancelled.
+
+A **genuine** small position is one a strategy meant to hold. It is preserved
+because the tolerance is relative to *the operation that produced it*, not to a
+global constant:
+
+* Opening 1e-9 units on an empty book is an operation at scale 1e-9, where the
+  tolerance is the floor, 1e-12. The position survives.
+* Holding 10,000 units and selling 9,999.999999999 leaves a genuine 1e-9. The
+  operation scale is 10,000, where the tolerance is 1.46e-11. 1e-9 is seventy
+  times larger, so the position survives.
+
+The only quantity the rule erases is one arrived at by cancelling two much larger
+quantities — and at that point the result is below what float64 arithmetic at
+that scale can warrant.
+
+## Edge cases
+
+| input | behaviour |
+|---|---|
+| `scale == 0` (no position, zero fill) | tolerance is the floor, `1e-12` |
+| subnormal operands | `ulp` is the minimum subnormal; the floor dominates |
+| `inf` / `NaN` operand | ignored when computing the scale; the tolerance stays finite and positive, and falls back to the floor if no operand is finite |
+| `NaN` quantity under test | `abs(nan) < tol` is false, so it is not treated as zero — unchanged from the previous rule |
+| long versus short | the tolerance is built from magnitudes, so it is identical for `+q` and `-q` |
+
+## Sites
+
+One shared primitive serves every site that asks whether a *quantity* is zero:
+position closure in the fill executor, the submission-precheck shadow book, the
+shadow-queue validation and commit paths, and the gatekeeper's position
+normalisation.
+
+`OrderBook._MIN_ORDER_SIZE` is deliberately **not** migrated. It is an economic
+policy — the smallest order the engine will accept — not a statement about
+floating-point residue.

mkdocs.yml

@@ -119,6 +119,7 @@ nav:
     - Quickstart: getting-started/quickstart.md
   - Concepts:
     - How It Works: concepts/how-it-works.md
+    - Quantity-Zero Tolerance: concepts/quantity-zero-tolerance.md
   - User Guide:
     - Strategies: user-guide/strategies.md
     - Order Types: user-guide/orders.md

src/ml4t/backtest/accounting/gatekeeper.py

@@ -4,6 +4,7 @@
 ensuring they meet account policy constraints and preventing invalid trades.
 """
 
+from ..core.shared import quantity_zero_tolerance
 from ..models import CommissionModel
 from ..types import Order, OrderSide
 from .account import AccountState
@@ -123,7 +124,7 @@ def validate_order(self, order: Order, price: float) -> tuple[bool, str]:
         """
         # Get current position quantity (0 if no position)
         current_qty = self.account.get_position_quantity(order.asset)
-        if abs(current_qty) < 1e-12:
+        if abs(current_qty) <= quantity_zero_tolerance(current_qty):
             current_qty = 0.0
 
         # Determine order direction (positive=buy, negative=sell)

src/ml4t/backtest/core/execution_engine.py

@@ -5,7 +5,7 @@
 import copy
 
 from ..types import ExecutionMode, OrderSide, OrderStatus, OrderType, Position
-from .shared import is_exit_order
+from .shared import is_exit_order, quantity_zero_tolerance
 
 
 class ExecutionEngine:
@@ -238,15 +238,15 @@ def _validate_shadow_queue_order(
         )
         new_qty = current_qty + qty_delta
         is_reversal = (
-            abs(current_qty) > 1e-12
-            and abs(new_qty) > 1e-12
+            abs(current_qty) > quantity_zero_tolerance(current_qty)
+            and abs(new_qty) > quantity_zero_tolerance(current_qty, qty_delta)
             and ((current_qty > 0 and new_qty < 0) or (current_qty < 0 and new_qty > 0))
         )
         commission = broker.commission_model.calculate(
             order.asset, order.quantity, validation_price
         )
 
-        if abs(current_qty) <= 1e-12:
+        if abs(current_qty) <= quantity_zero_tolerance(current_qty):
             return policy.validate_new_position(
                 asset=order.asset,
                 quantity=qty_delta,
@@ -290,7 +290,7 @@ def _commit_shadow_queue_fill(
         commission = broker.commission_model.calculate(order.asset, order.quantity, fill_price)
         shadow_cash += -qty_delta * fill_price * broker.get_multiplier(order.asset) - commission
 
-        if abs(new_qty) <= 1e-12:
+        if abs(new_qty) <= quantity_zero_tolerance(current_qty, qty_delta):
             shadow_positions.pop(order.asset, None)
             return shadow_cash
 
@@ -307,13 +307,13 @@ def _commit_shadow_queue_fill(
             return shadow_cash
 
         is_reversal = (
-            abs(current_qty) > 1e-12
-            and abs(new_qty) > 1e-12
+            abs(current_qty) > quantity_zero_tolerance(current_qty)
+            and abs(new_qty) > quantity_zero_tolerance(current_qty, qty_delta)
             and ((current_qty > 0 and new_qty < 0) or (current_qty < 0 and new_qty > 0))
         )
         position.quantity = new_qty
         position.current_price = broker._current_prices.get(order.asset, fill_price)
-        if abs(current_qty) <= 1e-12 or is_reversal:
+        if abs(current_qty) <= quantity_zero_tolerance(current_qty) or is_reversal:
             position.entry_price = fill_price
 
         return shadow_cash

src/ml4t/backtest/core/order_book.py

@@ -5,7 +5,7 @@
 from datetime import datetime
 
 from ..types import ExecutionMode, Order, OrderSide, OrderStatus, OrderType, Position
-from .shared import SubmitOrderOptions, is_exit_order
+from .shared import SubmitOrderOptions, is_exit_order, quantity_zero_tolerance
 
 
 class OrderBook:
@@ -15,7 +15,6 @@ class OrderBook:
         {"quantity", "limit_price", "stop_price", "trail_amount"}
     )
     _MIN_ORDER_SIZE: float = 1e-8
-    _QTY_EPS: float = 1e-12
 
     def __init__(self, broker):
         self.broker = broker
@@ -254,15 +253,15 @@ def _reset_submission_shadow_if_needed(self) -> None:
                 or pos.entry_price,
             )
             for asset, pos in broker.positions.items()
-            if abs(pos.quantity) > self._QTY_EPS
+            if abs(pos.quantity) > quantity_zero_tolerance(pos.quantity)
         }
 
     def _build_shadow_policy_positions(self) -> dict[str, Position]:
         broker = self.broker
         ts = broker._current_time or datetime(1970, 1, 1)
         positions: dict[str, Position] = {}
         for asset, (qty, basis_price) in self._submission_shadow_positions.items():
-            if abs(qty) <= self._QTY_EPS:
+            if abs(qty) <= quantity_zero_tolerance(qty):
                 continue
             mark_price = broker.get_mark_price(asset, quantity=qty) or basis_price
             positions[asset] = Position(
@@ -281,10 +280,10 @@ def _simulate_position_update(
     ) -> tuple[float, float, float, float]:
         """Mirror Backtrader Position.update for pseudo-exec prechecks."""
         new_qty = old_qty + size
-        if abs(new_qty) < OrderBook._QTY_EPS:
+        if abs(new_qty) <= quantity_zero_tolerance(old_qty, size):
             return 0.0, 0.0, 0.0, size
 
-        if abs(old_qty) < OrderBook._QTY_EPS:
+        if abs(old_qty) <= quantity_zero_tolerance(old_qty):
             return new_qty, price, size, 0.0
 
         if old_qty > 0:
@@ -352,7 +351,7 @@ def _passes_submission_precheck(self, order: Order) -> bool:
         # Keep shadow effects even for rejected orders to mirror Backtrader's
         # sequential submitted-queue pseudo-execution behavior.
         self._submission_shadow_cash = shadow_cash
-        if abs(new_qty) <= self._QTY_EPS:
+        if abs(new_qty) <= quantity_zero_tolerance(old_qty, size):
             self._submission_shadow_positions.pop(order.asset, None)
         else:
             self._submission_shadow_positions[order.asset] = (new_qty, new_price)
@@ -438,7 +437,7 @@ def _passes_buying_power_check(self, order: Order) -> bool:
 
         # Accepted — commit shadow changes
         self._submission_shadow_cash = shadow_cash
-        if abs(new_qty) <= self._QTY_EPS:
+        if abs(new_qty) <= quantity_zero_tolerance(old_qty, size):
             self._submission_shadow_positions.pop(order.asset, None)
         else:
             self._submission_shadow_positions[order.asset] = (new_qty, new_price)
@@ -465,12 +464,12 @@ def _passes_margin_submission_precheck(self, order: Order, signal_price: float)
 
         shadow_positions = self._build_shadow_policy_positions()
         is_reversal = (
-            abs(old_qty) > self._QTY_EPS
-            and abs(new_qty) > self._QTY_EPS
+            abs(old_qty) > quantity_zero_tolerance(old_qty)
+            and abs(new_qty) > quantity_zero_tolerance(old_qty, size)
             and ((old_qty > 0 and new_qty < 0) or (old_qty < 0 and new_qty > 0))
         )
 
-        if abs(old_qty) <= self._QTY_EPS:
+        if abs(old_qty) <= quantity_zero_tolerance(old_qty):
             valid, reason = broker.account.policy.validate_new_position(
                 asset=order.asset,
                 quantity=size,
@@ -510,7 +509,7 @@ def _passes_margin_submission_precheck(self, order: Order, signal_price: float)
 
         multiplier = broker.get_multiplier(order.asset)
         self._submission_shadow_cash += -size * signal_price * multiplier - commission
-        if abs(new_qty) <= self._QTY_EPS:
+        if abs(new_qty) <= quantity_zero_tolerance(old_qty, size):
             self._submission_shadow_positions.pop(order.asset, None)
         else:
             self._submission_shadow_positions[order.asset] = (new_qty, new_price)

src/ml4t/backtest/core/shared.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 from dataclasses import dataclass
+from math import fabs, isfinite, ulp
 from typing import TYPE_CHECKING
 
 from ..types import ExitReason, OrderSide
@@ -14,6 +15,56 @@
 # Prevents order rejections due to rounding in equity/price arithmetic.
 CASH_TOLERANCE: float = 0.01
 
+# Quantity-zero tolerance.
+#
+# A fill updates a position with ``new_qty = old_qty + signed_qty``. When the
+# close is economically exact the two operands cancel as real numbers, but they
+# are floats produced by different routes, so the sum can land a few units in
+# the last place away from zero instead of on it. The size of that residue is
+# set by the spacing of float64 at the scale of the quantities being cancelled,
+# and float64 spacing grows with magnitude.
+#
+# A single absolute epsilon cannot express that. 1e-12 is wider than one ULP
+# while |q| < 8192 and narrower than one ULP from |q| = 8192 upward, so an
+# absolute rule silently stops closing positions at exactly 2**13 units.
+#
+# The floor preserves the historical absolute behaviour wherever float64 spacing
+# is finer than it, which is every magnitude below 8192.
+QTY_ZERO_FLOOR: float = 1e-12
+QTY_ZERO_ULPS: int = 16
+
+
+def quantity_zero_tolerance(*operands: float) -> float:
+    """Return the tolerance under which a residual quantity counts as zero.
+
+    The tolerance is the larger of :data:`QTY_ZERO_FLOOR` and
+    :data:`QTY_ZERO_ULPS` units in the last place of the largest finite operand.
+
+    Pass the quantities that *produced* the residual — for a fill, the pre-fill
+    position and the signed fill size. Do not pass the residual itself: it is
+    approximately zero by construction, so it carries no scale and a tolerance
+    derived from it could never fire.
+
+    Non-finite operands are ignored, so the tolerance is always a finite
+    positive number; if no operand is finite the floor applies. The tolerance is
+    built from magnitudes, so it is symmetric for long and short quantities.
+
+    Args:
+        *operands: Quantities defining the scale of the operation.
+
+    Returns:
+        The tolerance, always >= ``QTY_ZERO_FLOOR``.
+    """
+    scale = 0.0
+    for operand in operands:
+        if isfinite(operand):
+            magnitude = fabs(operand)
+            if magnitude > scale:
+                scale = magnitude
+    if scale == 0.0:
+        return QTY_ZERO_FLOOR
+    return max(QTY_ZERO_FLOOR, QTY_ZERO_ULPS * ulp(scale))
+
 
 @dataclass
 class SubmitOrderOptions: