Egg LLM

Real, GPU-accelerated LLM inference inside a hardware-isolated VM, on the host Apple GPU — driven from inside the egg.

Where this runs — egg

This VM runs under egg (the EggRun hypervisor): Apple Hypervisor.framework, in userspace, no daemon. An "egg" is a hardware-isolated VM — EggRun is a Docker competitor that uses VMs instead of a shared kernel: Containers share a kernel. Eggs don't.

• Guest: aarch64 Ubuntu (this VM).
• Host: Apple Silicon (M5 Max), macOS, running egg.
• GPU: the real host Apple GPU, reached from the guest over virtio-gpu.

Proven

Qwen2.5-0.5B-Instruct (Q4_K_M), -ngl 99 → real, coherent tokens at ~295–350 tok/s on the Apple M5 Max GPU, generated from inside the egg — with the GNOME desktop still rendering on Venus alongside it (APIR capset 10 coexists with the graphics path; the GUI never flinches).

Model choice

Prefer standard-attention models (Qwen, Llama, Mistral, Gemma) over linear-attention ones (Mamba/SSM, Gated Delta Net, GLA, RWKV) at the same parameter size. The GPU runs over APIR, which remotes one ggml compute graph per virtio-gpu round-trip; linear-attention architectures are far more op-dense per token, so that per-graph overhead drags their throughput well below a standard-attention model of equal size. For best tok/s here, pick standard attention.

Run it (in the guest)

./build.sh    # install dependencies and build
./test.sh     # test the build
./egg-llm     # launch the TUI and chat with the model

Keep the model in ~ (persistent) — /tmp is wiped on reboot.

---

EggRun — hardware-isolated VMs for autonomous workloads. eggrun.ai · a Camouflage Networks company.