perf(ci): cancel flush-thread sleep so shutdown() returns in ms, not 5s

The Transport flush loop used `time.sleep(self.config.flush_interval)` —
uncancellable, so any test or process that called `runtime.shutdown()`
while the thread was mid-sleep blocked on `thread.join()` for the full
default 5s flush_interval. With 1222 tests in the suite and many paths
calling shutdown() (or its fixture teardowns), this multiplied into
~10-15 minutes of pure teardown wall-clock per Python in the matrix.

Replace the bare sleep with `Event.wait`, which returns the instant
`stop()` sets the event. `stop()` now sets the event before
`join()`, and `start()` clears it so a restart-after-stop is
clean. Pin contract in tests/test_transport.py::

    test_stop_interrupts_flush_sleep

…uses a 30s flush_interval; pre-fix this took 30s, post-fix <5s.

CI hygiene in the same commit so the suite can actually use the freed
time:

- ci.yml / publish*.yml: enable pip cache (`cache: pip` +
  `cache-dependency-path: pyproject.toml`) — saves ~60-90s of cold
  install per matrix leg.
- ci.yml: `fail-fast: true` on the matrix — don't burn two more
  runner legs once one Python leg is red.
- ci.yml / coverage / publish*.yml: install `pytest-xdist>=3.6` and
  pass `-n auto` to pytest. `pytest-xdist` is also added to
  `[project.optional-dependencies.dev]` so a local
  `pip install -e .[dev]` brings it in.
- pyproject.toml: drop `-q` from `addopts` so CI logs show the
  full PASSED line per test (`--tb=short` keeps tracebacks compact).
  `-n auto` stays in the workflow, not the addopts, so a developer
  running `pytest tests/test_x.py` gets a single process.

No public API change. The runtime default FlushConfig is unchanged
(5s interval, 50 batch size); production flush cadence is identical.
The fix only shortens the worst-case shutdown latency.

Author: maltsev-dev

.github/workflows/ci.yml

@@ -11,7 +11,11 @@ jobs:
     runs-on: ubuntu-latest
     permissions:
       contents: read
+    # 2026-07-08: fail-fast on the first matrix failure instead of
+    # wasting runner minutes on the remaining Python versions when
+    # the suite is already red. Speed gain is per-run, not per-test.
     strategy:
+      fail-fast: true
       matrix:
         python: ["3.10", "3.11", "3.12"]
 
@@ -22,14 +26,29 @@ jobs:
         uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python }}
+          # Cache pip's download cache keyed on the lock-relevant
+          # surfaces of pyproject.toml. Skips the ~60-90s cold
+          # install on warm caches; the action also reuses the
+          # cache across matrix legs when the key matches.
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
-          pip install -e ".[dev]"
+          # xdist ships in the dev tree already; pin it explicitly so
+          # a future deps churn can't drop it without breaking CI.
+          pip install -e ".[dev]" "pytest-xdist>=3.6"
 
       - name: Run tests
-        run: pytest
+        # `-n auto` lets xdist pick a worker count from the runner's
+        # CPU count. With the transport cancellable-sleep fix the
+        # 5s-per-shutdown multiplier is gone, and xdist plus the
+        # existing respx-based mocking keeps the per-test wall clock
+        # near single-thread baseline (no shared state between
+        # workers — ``reset_runtime`` autouse fixture in conftest
+        # is per-process by construction under xdist).
+        run: pytest -n auto --durations=20
 
       - name: Run ruff
         run: ruff check src/
@@ -46,11 +65,16 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: "3.12"
-      - run: pip install -e ".[dev]"
-      - run: coverage run -m pytest
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
+      - run: pip install -e ".[dev]" "pytest-xdist>=3.6"
+      # Single Python leg for coverage — multi-version coverage
+      # reports don't add signal and double the runner time. 3.12
+      # is the modern floor for typing-only changes.
+      - run: coverage run -m pytest -n auto
       - uses: codecov/codecov-action@v4
         if: always()
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
           files: ./coverage.xml
-          fail_ci_if_error: false
+          fail_ci_if_error: false

.github/workflows/publish-test.yml

@@ -19,12 +19,14 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
-        run: pip install -e ".[dev]"
+        run: pip install -e ".[dev]" "pytest-xdist>=3.6"
 
       - name: Run tests
-        run: pytest tests/ -v
+        run: pytest tests/ -v -n auto
 
   publish:
     name: Build and publish to TestPyPI
@@ -62,4 +64,4 @@ jobs:
           # re-runs of the same SHA a no-op (matching twine's
           # --skip-existing behaviour). Production PyPI cannot
           # overwrite anyway, so this flag is harmless there too.
-          skip-existing: true
+          skip-existing: true

.github/workflows/publish.yml

@@ -12,6 +12,9 @@ jobs:
     runs-on: ubuntu-latest
     permissions:
       contents: read
+    # 2026-07-08: parallel matrix kept (PyPI publish is a one-shot
+    # event and the runner is already paid for) but pip cache
+    # brought in for parity with ci.yml.
     strategy:
       matrix:
         python-version: ["3.10", "3.11", "3.12"]
@@ -22,12 +25,14 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
-        run: pip install -e ".[dev]"
+        run: pip install -e ".[dev]" "pytest-xdist>=3.6"
 
       - name: Run tests
-        run: pytest tests/ -v
+        run: pytest tests/ -v -n auto
 
   publish:
     name: Build and publish

pyproject.toml

@@ -17,7 +17,7 @@ name = "nullrun"
 # nullrun._singleton (the metaclass-backing descriptor) and
 # nullrun._registry (the runtime registry) so runtime.py stays the
 # orchestrator only. See __version__.py for the full changelog.
-version = "0.13.3"
+version = "0.13.4"
 # Long form used by PyPI page meta-description and search snippets.
 # Kept under the 200-char preview threshold so the full line is visible
 # without an "expand" click. Keywords are matched against likely search
@@ -140,6 +140,14 @@ dev = [
     "ruff>=0.5",
     "coverage[toml]>=7.0",
     "httpx>=0.27.0,<1.0",
+    # xdist pins the parallel runner as a first-class dev dep so
+    # `pip install -e ".[dev]"` brings it in for local runs and
+    # CI both. The CI workflow also installs it explicitly to
+    # survive a future pyproject prune. ``-n auto`` is set in
+    # the workflow rather than ``addopts`` so single-CPU local
+    # runs (e.g. ``pytest tests/test_one.py``) don't accidentally
+    # spawn a worker pool.
+    "pytest-xdist>=3.6",
     # The SDK eagerly imports `nullrun.instrumentation.langgraph`
     # (from `nullrun.decorators`, imported by `nullrun.__init__` at
     # collection time), which itself does `from langchain_core.callbacks
@@ -477,7 +485,15 @@ ignore = [
 [tool.pytest.ini_options]
 asyncio_mode = "auto"
 testpaths = ["tests"]
-addopts = "--tb=short -q"
+# 2026-07-08: dropped the global ``-q`` so CI logs surface the
+# full PASSED line for each test (handy when scanning a red run).
+# Per-test verbosity stays low because ``--tb=short`` keeps the
+# tracebacks compact. ``-n auto`` lives in the workflow file, not
+# here, so a developer running ``pytest tests/test_x.py`` locally
+# gets a single process — the worker pool is only worth it on
+# the full suite, and some single-file debug sessions actively
+# want serial execution.
+addopts = "--tb=short"
 # Make the tests/ directory importable as a top-level package so
 # tests can use `from tests.conftest import BASE_URL`. Without this,
 # `from tests.conftest` raises ModuleNotFoundError on Python 3.10/3.11

src/nullrun/transport.py

@@ -535,6 +535,13 @@ def __init__(
         # methods) doesn't deadlock.
         self._flush_thread: threading.Thread | None = None
         self._running = False
+        # Cancellable sleep primitive for the flush loop. ``Event.wait``
+        # returns immediately when ``set()`` is called from ``stop()``,
+        # so a teardown that hits a thread mid-``time.sleep`` no longer
+        # blocks for the full ``flush_interval`` (default 5s) before
+        # ``join`` returns. Pin contract: tests/test_transport.py::
+        # test_stop_interrupts_flush_sleep.
+        self._stop_event = threading.Event()
 
         # mTLS client certificate support
         # NULLRUN_TLS_CLIENT_CERT and NULLRUN_TLS_CLIENT_KEY env vars for client cert auth
@@ -770,6 +777,9 @@ def start(self) -> None:
         # Replay any events from WAL that were persisted due to previous crash
         self._replay_from_wal()
         self._running = True
+        # Clear the stop latch so a previous stop() does not short-circuit
+        # the new flush loop on its first sleep.
+        self._stop_event.clear()
         self._flush_thread = threading.Thread(target=self._flush_loop, daemon=True)
         self._flush_thread.start()
         logger.info("Transport flush thread started")
@@ -801,6 +811,13 @@ def stop(self, timeout: float = 10.0) -> None:
         """Stop background flush thread and flush remaining events."""
         self._running = False
         self._stopped = True  # Mark as stopped to prevent double flush
+        # Wake the flush thread out of its cancellable sleep so join()
+        # returns immediately instead of waiting out the full
+        # ``flush_interval``. Without this, a teardown that hits the
+        # thread mid-sleep pays the 5s default flush_interval per
+        # shutdown — a multiplier on every test that calls
+        # ``runtime.shutdown()``.
+        self._stop_event.set()
         if self._flush_thread:
             self._flush_thread.join(timeout=timeout)
         self._do_flush()  # Final flush
@@ -816,7 +833,14 @@ def stop(self, timeout: float = 10.0) -> None:
     def _flush_loop(self) -> None:
         """Background loop that periodically flushes."""
         while self._running:
-            time.sleep(self.config.flush_interval)
+            # ``Event.wait`` returns True when ``stop()`` sets the
+            # event — that is the cancel signal. On timeout it
+            # returns False and we fall through to a flush. Replaces
+            # a plain ``time.sleep`` that could not be interrupted
+            # early, so stop() used to block for the full interval.
+            cancelled = self._stop_event.wait(timeout=self.config.flush_interval)
+            if cancelled:
+                break
             if self._running:
                 self._do_flush()
 

tests/test_transport.py

@@ -76,6 +76,41 @@ def test_flush_on_stop(self, transport):
         transport.stop()
         assert route.called
 
+    def test_stop_interrupts_flush_sleep(self):
+        """stop() must wake the flush thread out of its cancellable
+        sleep instead of waiting out the full ``flush_interval``.
+
+        Regression pin for the CI-speed fix: the previous loop used a
+        bare ``time.sleep``, so a test that called ``runtime.shutdown
+        ()`` while the thread was mid-sleep blocked for the full
+        interval (default 5s). With ``Event.wait`` the join returns
+        within a few hundred ms — so the whole suite runs in tens of
+        seconds instead of 15+ minutes. Uses a deliberately long
+        ``flush_interval`` to make the regression obvious if it
+        creeps back.
+        """
+        from nullrun.transport import FlushConfig
+
+        t = Transport(
+            api_url="https://api.test.nullrun.io",
+            api_key="test-key-12345678",
+            config=FlushConfig(flush_interval=30.0),  # would be 30s pre-fix
+        )
+        t.start()
+        # Give the thread a beat to enter _flush_loop's wait.
+        time.sleep(0.05)
+        started = time.monotonic()
+        t.stop()
+        elapsed = time.monotonic() - started
+        # Allow generous headroom for CI jitter; the contract is
+        # "much less than flush_interval" — a pre-fix run would hit
+        # the full 30s and time out this assertion.
+        assert elapsed < 5.0, (
+            f"stop() took {elapsed:.2f}s; expected < 5s. The flush "
+            f"loop is sleeping in plain ``time.sleep`` again — the "
+            f"cancellable-wait fix regressed."
+        )
+
     def test_ssl_verification_enabled(self, transport):
         # httpx 0.28+ doesn't expose verify as a direct attribute
         # SSL verification is enabled by default (verify=True)