laguna-mlx

MLX (Apple Silicon) port of poolside/Laguna-S-2.1 — Poolside's 118B-total / ~8B-active sparse-MoE model for agentic coding and long-horizon work (1M-token context).

laguna.py is a single, stock-mlx-lm-compatible model definition. Register it, then use mlx_lm convert to build quants and mlx_lm.generate / mlx_lm.server to run them. Published quants live under pipenetwork/ on the Hub.

Architecture

Laguna is close to Qwen3-MoE, with these additions (all handled in laguna.py):

Feature	Detail
MoE	256 routed experts, top-10, moe_intermediate_size=1024; sigmoid router + aux-loss-free e_score_correction_bias; 1 shared expert on every token
Dense layer	layer 0 only (mlp_only_layers=[0], intermediate_size=12288)
Attention	48 heads / 8 KV heads, head_dim=128; per-head Q/K RMSNorm; softplus per-head output gating (g_proj)
Layout	48 layers interleaving full (12) and sliding-window 512 (36) attention
RoPE	full layers: partial-rotary (0.5) YaRN (θ=5e5, factor 128, mscale 1.4852); sliding layers: plain RoPE (θ=1e4, full rotary)
Vocab	100,352 · tie_word_embeddings=False

The laguna.py forward pass is validated against the reference modeling_laguna.py structure by tests/test_laguna.py (exact checkpoint-key mapping over all 36,769 tensors + a tiny forward/decode run) — no weights download required.

Usage

The published quants run on stock mlx-lm. The only extra step is registering laguna.py once, since the laguna architecture isn't in mlx-lm yet. Pick the variant that fits your unified memory (peak ≈ model size + a few GB):

Variant	Size	Peak RAM	Fits a Mac with
2bit	35 GB	~38 GB	48 GB+
3bit	48 GB	~52 GB	64 GB+
4bit	62 GB	~66 GB	96 GB+
6bit	89 GB	~95 GB	128 GB+
8bit	116 GB	~122 GB	192 GB+
bf16	219 GB	~225 GB	256 GB+

pip install mlx-lm

# register the bundled loader once (arch not in stock mlx-lm)
python - <<'PY'
import os, shutil, mlx_lm
from huggingface_hub import hf_hub_download
dst = os.path.join(os.path.dirname(mlx_lm.__file__), "models", "laguna.py")
shutil.copy(hf_hub_download("pipenetwork/Laguna-S-2.1-MLX-4bit", "laguna.py"), dst)
print("registered laguna ->", dst)
PY

Generate (CLI)

mlx_lm.generate --model pipenetwork/Laguna-S-2.1-MLX-4bit \
    --prompt "Write a Rust function that reverses a singly linked list." \
    --max-tokens 512

mlx_lm.generate applies the model's chat template automatically. Laguna enables interleaved reasoning by default (enable_thinking=true), so replies open with a thinking pass before the final answer.

Python API

from mlx_lm import load, generate

model, tokenizer = load("pipenetwork/Laguna-S-2.1-MLX-4bit")
messages = [{"role": "user", "content": "Explain a red-black tree in three sentences."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
text = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)

OpenAI-compatible server

mlx_lm.server --model pipenetwork/Laguna-S-2.1-MLX-4bit --port 8080
# POST chat completions to http://localhost:8080/v1/chat/completions

Develop locally (offline)

Set up the repo and run the no-weights validation:

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# fetch reference config + index (for the offline test) — small files only
mkdir -p reference
for f in config.json model.safetensors.index.json; do
  curl -sL "https://huggingface.co/poolside/Laguna-S-2.1/resolve/main/$f" -o "reference/$f"
done

scripts/install_model.sh          # register laguna.py into .venv's mlx-lm
python tests/test_laguna.py       # offline validation

# with the source weights present, also checks all 36,769 tensor shapes:
LAGUNA_SRC=./Laguna-S-2.1-src python tests/test_laguna.py

Build + publish the quants

scripts/download.sh                       # ~235 GB bf16 source -> ./Laguna-S-2.1-src
scripts/convert_all.sh                    # -> ./out/Laguna-S-2.1-MLX-{8,6,4,3}bit + bf16
huggingface-cli login                     # once, needs write access to pipenetwork/
scripts/run_uploads.sh                    # -> pipenetwork/Laguna-S-2.1-MLX-*

Or one variant at a time:

scripts/convert.sh ./Laguna-S-2.1-src ./out/Laguna-S-2.1-MLX-4bit 4
python scripts/upload.py 4bit ./out/Laguna-S-2.1-MLX-4bit

The 2-bit tier uses a mixed-precision recipe (pure 2-bit is incoherent), driven by the LAGUNA_QUANT_PROFILE=mixed2 predicate in laguna.py:

scripts/build_mixed2.sh                   # convert + smoke-test -> ./out/Laguna-S-2.1-MLX-2bit
python scripts/upload.py 2bit ./out/Laguna-S-2.1-MLX-2bit

After adding any variant, refresh the sibling cards so every table lists it:

python scripts/refresh_cards.py           # pushes README-only updates to the other repos

Published models

Variant	Size	Bits/weight	Notes
8bit	116 GB	8.500	near-lossless
6bit	89 GB	6.000	high quality
4bit	62 GB	4.501	balanced default
3bit	48 GB	3.502	smallest footprint
2bit	35 GB	2.570	mixed precision (experts 2b, rest 4b)
bf16	219 GB	16	full precision

Router gate kept at 8-bit across all 47 MoE layers. The 2-bit build is mixed precision (routed experts 2-bit; attention, embeddings, LM head, and shared expert 4-bit) — pure 2-bit is incoherent, and since the experts are ~96% of the weights this costs almost nothing (35 GB vs a pure-2bit 34 GB). Smoke tests on Apple Silicon all produce coherent code: 4-bit ~64 tok/s / ~66 GB; 3-bit ~67 tok/s / ~52 GB; 2-bit ~69 tok/s / ~38 GB peak.

License

Code: Apache-2.0 (LICENSE). Model weights: OpenMDW-1.1, inherited from the base model (LICENSE.md).