Streaming, builder API, notifications, native kernel bundle, and event loop threading

[morozow]

Highlights

• Streaming requests — real-time chunk delivery via async generators
• Notification subscriptions with overflow control
• Fluent builder API for ergonomic client construction
• Bundled native kernel — prebuilt libstdio_bus.a for 4 platforms, zero external build steps
• Native backend: event loop integration replaces background thread polling
• Binary resolution module for subprocess backend

Streaming API

async for event in bus.stream_request("pool_id", {"prompt": "..."}):
    if event.type == StreamEventType.CHUNK:
        print(event.data, end="", flush=True)
    elif event.type == StreamEventType.RESULT:
        final = event.data

Chunks are delivered incrementally as they arrive — not accumulated and returned at the end.

Notification Subscriptions

sub = bus.subscribe("pool_id", overflow_policy=OverflowPolicy.DROP_OLDEST)
async for notification in sub:
    process(notification)
bus.unsubscribe(sub)

Builder API

bus = (
    StdioBusBuilder()
    .config(BusConfig(pools=[...]))
    .backend("native")
    .native_options(NativeOptions(listen_mode=ListenMode.TCP, tcp_port=9000))
    .build_sync()
)

Introspection

bus.get_worker_count()   # int
bus.get_client_count()   # int
bus.get_listen_mode()    # ListenMode

Native Backend: Event Loop Integration

All C API calls now execute exclusively on the owning asyncio event loop thread, matching the C library's single-thread contract.

• fd-driven mode (preferred): poll fd registered via loop.add_reader() — zero-latency wake via kqueue/epoll
• timer-driven mode (fallback): loop.call_later() timer when poll fd unavailable
• Thread affinity enforced: send() from a foreign thread raises TransportError
• No background threads, no GIL contention, no race conditions

Native Kernel

Prebuilt static libraries in kernel/prebuilds/:

• aarch64-apple-darwin
• x86_64-apple-darwin
• aarch64-unknown-linux-gnu
• x86_64-unknown-linux-gnu

CLI binaries in kernel/dist/ for darwin-{amd64,arm64}, linux-{amd64,arm64,armv7}, linux-musl-{amd64,arm64}.

pip install stdiobus[native]

Binary Resolution

Subprocess backend now resolves the stdio_bus binary with explicit precedence:

1. User-provided path (override)
2. Bundled kernel/dist/stdio_bus launcher
3. System PATH fallback

Internal

• Renamed _native → native (public module)
• Added setup.py with conditional cffi extension build
• stream_echo_worker.py for E2E streaming validation
• test_real_e2e.py — integration tests against live binary
• test_native_threading.py — thread affinity invariant tests
• 317+ unit/integration tests

Full Changelog

v2.1.1...v2.2.1

---

This discussion was created from the release Streaming, builder API, notifications, native kernel bundle, and event loop threading.