Skip to content

Port bounded Zig runtime graph and parallel memory scheduling - #469

Open
ajroetker wants to merge 349 commits into
mainfrom
codex/port-pr450-build-memory
Open

Port bounded Zig runtime graph and parallel memory scheduling#469
ajroetker wants to merge 349 commits into
mainfrom
codex/port-pr450-build-memory

Conversation

@ajroetker

Copy link
Copy Markdown
Contributor

Summary

  • ports the high-value PR Reduce Zig ReleaseFast runtime codegen graph #450 ReleaseFast partition into four static runtime units: API, distributed/application storage, inference, and remote CLI
  • links the single PIC distributed storage archive into the executable and both C APIs, with section GC and production LSM-only reachability
  • preserves narrow API, standalone inference, and restore-staging bridges while removing broad cross-runtime imports
  • shards the monolithic storage unit-test codegen graph and gives storage, metadata, and remaining aggregate compile/run steps explicit RSS claims
  • removes CI reliance on -j1; the Zig 0.16 max-RSS scheduler now uses 80% of the cgroup limit with the wake-accounting backport

Verification

  • clean ARM64 Linux musl ReleaseFast install + lite-capi build, linked runtimes, production LSM-only, normal concurrency, 20 GiB scheduler budget
  • unit-storage-test with HA disabled, normal concurrency, patched runner, 12 GiB scheduler budget
  • linked Debug install build
  • runtime, codegen, and API-kernel graph boundary checks
  • 15 graph analyzer and max-RSS patch unit tests
  • generated OpenAPI, Snowball, and C ABI checks

PR #467 job note

The referenced job reached about 17.4 GB cgroup peak but recorded no OOM event; its terminal failure was the document-scan authorization assertion in api.http_server. This PR addresses the build-memory exposure that job demonstrated without masking or changing that separate functional failure.

Boris Popadiuk and others added 30 commits July 14, 2026 20:00
…time-hardening

# Conflicts:
#	scripts/ci/zig-build-e2e-binaries.sh
#	zig/build.zig
#	zig/e2e/antfly/test_standby.py
#	zig/lib/httpx/src/client/client.zig
#	zig/pkg/antfly/src/api/http_server.zig
#	zig/pkg/antfly/src/api/public_table_http.zig
#	zig/pkg/antfly/src/api/table_writes.zig
#	zig/pkg/antfly/src/common/http/std_http_executor.zig
#	zig/pkg/antfly/src/data/runtime.zig
#	zig/pkg/antfly/src/metadata/table_manager.zig
#	zig/pkg/antfly/src/raft/storage/catalog.zig
#	zig/pkg/antfly/src/standalone/runtime.zig
#	zig/pkg/antfly/src/storage/db/db.zig
#	zig/pkg/antfly/src/storage/ha/http_admin.zig
#	zig/pkg/antfly/src/storage/ha/http_internal.zig
#	zig/pkg/antfly/src/storage/ha/standby.zig
# Conflicts:
#	zig/pkg/inference/build.zig
#	zig/pkg/inference/src/architectures/deberta.zig
#	zig/pkg/inference/src/architectures/session_factory.zig
#	zig/pkg/inference/src/backends/metal_kernels.m
#	zig/pkg/inference/src/backends/metal_runtime.zig
#	zig/pkg/inference/src/bench/gliner2_native.zig
#	zig/pkg/inference/src/graph/backend_contracts.zig
#	zig/pkg/inference/src/inference.zig
#	zig/pkg/inference/src/ops/metal_compute.zig
#	zig/pkg/inference/src/ops/ops.zig
#	zig/pkg/inference/src/pipelines/gliner.zig
#	zig/pkg/inference/src/server/model_manager.zig
#	zig/pkg/inference/src/server/server.zig
…450-build-memory

# Conflicts:
#	.github/workflows/zig-tests.yml
#	scripts/ci/zig-build-e2e-binaries.sh
#	zig/build.zig
#	zig/pkg/antfly/src/api/http_server.zig
#	zig/pkg/antfly/src/api/linear_merge.zig
#	zig/pkg/antfly/src/cmd/ha.zig
#	zig/pkg/antfly/src/main.zig
#	zig/pkg/antfly/src/standalone/runtime.zig
…r450-build-memory

# Conflicts:
#	.github/workflows/zig-tests.yml
#	zig/build.zig
#	zig/lib/httpx/src/client/client.zig
#	zig/pkg/antfly/src/inference/managed_embedder.zig
#	zig/pkg/antfly/src/standalone/runtime.zig
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants