Feat/pack mind

.gitignore

@@ -86,3 +86,5 @@ htmlcov/
 
 # Memory2 autorecord
 recording*.db
+.scratch/
+.gstack/

dimos/experimental/pack_mind/LIVE_RUNBOOK.md

@@ -0,0 +1,166 @@
+# PACK MIND — LIVE RUNBOOK (2 dogs / 2 laptops / AP-per-dog + Tailscale)
+
+The on-site page for the 2-dog live demo. Documents the topology the README's "Live demo"
+section does NOT cover: each Go2 stays in its own **AP mode**, each laptop is **dual-homed**
+(Wi-Fi → its dog's AP, a USB-tether uplink → the internet), and the two laptops reach the
+shared coordinator over a **Tailscale** tunnel across the internet.
+
+> README's live section assumes one shared router + dogs in STA (rejected: no trusted net on
+> site) and its `--robot-ip` flag is stale. `live.py`'s LAN_IP only works on one LAN. Neither
+> fits this topology — use THIS page.
+
+```
+ Go2-A ──AP 192.168.12.1── Laptop-A(alpha) ── USB-tether ─→ INTERNET ─┐
+                                │ runs coordinator :8090               │  Tailscale
+                                │ tailscale 100.x.A                    │  100.64/10
+ Go2-B ──AP 192.168.12.1── Laptop-B(bravo) ── USB-tether ─→ INTERNET ─┘
+                                  PACK_COORDINATOR_URL=http://100.x.A:8090
+```
+Both dogs share IP `192.168.12.1` — two separate Wi-Fi L2 nets, each laptop sees only its
+own dog. No conflict. The dog never touches Tailscale; only laptop↔laptop coordinator HTTP does.
+
+### Honesty red line (from PLAN §4 — Q&A will probe this)
+The live demo proves **embodiment + coordination + memory-outlives-the-robot** (no-overlap,
+inheritance). It does **NOT** prove the coverage-speed A/B claim — that's the sim's job.
+Narrate: "two bodies, one shared memory; lose one, the mission survives."
+
+---
+
+## 0. NIGHT BEFORE (cannot be debugged inside the demo window)
+
+1. **USB-tether uplink on both laptops** (Wi-Fi is taken by the dog AP). A single-Wi-Fi Mac
+   with no second uplink CANNOT do this topology.
+2. **Tailscale on both, same tailnet.** From Laptop-B: `ping <Laptop-A-tailscale-ip>` over
+   the internet (no dogs needed).
+3. **Service order: uplink ABOVE Wi-Fi** (System Settings → Network → ⋯ → Set Service
+   Order). Else macOS routes WAN through the dog AP → Tailscale dies. Verify
+   `route -n get 8.8.8.8` shows the uplink iface.
+4. **Cross-net smoke (no dogs)** — prove the ledger works over Tailscale end to end:
+   ```bash
+   # Laptop-A:
+   uv run python -m dimos.experimental.pack_mind.pack_coordinator_server \
+     --zones north,east,south,west --prefs "alpha:south,north,east,west;bravo:north,east,west,south"
+   # Laptop-B (A's tailscale ip):
+   uv run python -m dimos.experimental.pack_mind.mock_dog --dog bravo \
+     --url http://<Laptop-A-tailscale-ip>:8090 --reset --target "red kit"
+   ```
+   Dashboard at `http://<A-tailscale-ip>:8090` animates from Laptop-B's browser → brain works.
+5. **Per-laptop `~/.packmind.env`** pre-written (§2). Confirm `OPENAI_API_KEY` valid on uplink.
+6. **Prefetch models** (insurance): `uv run python -m dimos.experimental.pack_mind.prefetch_live_models`.
+
+---
+
+## 1. MORNING — preflight each laptop FIRST (fix every FAIL before starting dogs)
+
+```bash
+# Laptop-A:
+uv run python -m dimos.experimental.pack_mind.preflight \
+  --role alpha --peer <Laptop-B-tailscale-ip> --coordinator http://127.0.0.1:8090
+# Laptop-B:
+uv run python -m dimos.experimental.pack_mind.preflight \
+  --role bravo --peer <Laptop-A-tailscale-ip> --coordinator http://<Laptop-A-tailscale-ip>:8090
+```
+
+---
+
+## 2. Per-laptop env (`~/.packmind.env`, `source` before anything)
+
+```bash
+# BOTH:
+export HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1
+export ROBOT_IP=192.168.12.1            # each laptop's OWN dog over its AP
+export LISTEN_HOST=0.0.0.0
+export OPENAI_API_KEY=<key>             # agent brain (hosted) — needs the uplink
+# Laptop-A only:
+export PACK_DOG_NAME=alpha
+export PACK_COORDINATOR_URL=http://127.0.0.1:8090
+# Laptop-B only:
+export PACK_DOG_NAME=bravo
+export PACK_COORDINATOR_URL=http://<Laptop-A-tailscale-ip>:8090
+```
+`dimos run` has **no `--robot-ip` flag** in this build — it reads `ROBOT_IP`; verify with
+`dimos show-config`. `ROBOT_IP=192.168.12.1` = dog's-own-AP WebRTC path (one laptop per dog).
+
+---
+
+## 3. Bring-up (Laptop-A first)
+
+**Laptop-A — coordinator + dashboard + dog alpha:**
+```bash
+source ~/.packmind.env
+uv run python -m dimos.experimental.pack_mind.pack_coordinator_server \
+  --zones north,east,south,west \
+  --prefs "alpha:south,north,east,west;bravo:north,east,west,south" &
+#   → open http://<Laptop-A-tailscale-ip>:8090 on the projector NOW
+uv run dimos run unitree-go2-pack --daemon
+```
+**Laptop-B — dog bravo:**
+```bash
+source ~/.packmind.env
+uv run dimos run unitree-go2-pack --daemon
+```
+Per-dog gate before demo: ping dog → `dimos show-config` → run to "running" → `mcp
+list-tools` + `speak` (proves WebRTC) → a `drive` burst → `look_for_red` → search skills.
+
+---
+
+## 4. THE DEMO — STABLE first, SHOWCASE only if stable lands
+
+### 4a. STABLE (scripted, can't-miss) — `scripted_pack_run`
+Real dogs move via velocity teleop; the **real coordinator** drives no-overlap + inheritance;
+the dashboard animates. No LLM/nav-RPC on the critical path.
+```bash
+# Laptop-A (alpha = leader, holds target zone, you drop it on cue):
+uv run python -m dimos.experimental.pack_mind.scripted_pack_run \
+  --role alpha --leader --target "red kit" --target-zone south \
+  --hold-zone --drop-on-enter --drive
+# Laptop-B (bravo = sweeper, inherits + finds):
+uv run python -m dimos.experimental.pack_mind.scripted_pack_run \
+  --role bravo --target "red kit" --target-zone south \
+  --find --keep-polling --drive
+```
+Beat: alpha claims `south` and holds it → bravo clears `north/east/west` **zero overlap** →
+on the host's cue press **Enter** on Laptop-A ("we lose Alpha") → alpha OFFLINE, `south`
+returns to the pool → bravo **inherits** `south`, finds the kit → pack stops. Mission
+outlived the robot. Drop `--drive` to run the pure-ledger story if the floor is tight.
+
+### 4b. SHOWCASE (full autonomy, higher risk)
+Voice/`mcp` to each dog: *"find the red kit."* Agent runs `start_search → next_zone →
+navigate → look_for_red → report_*`. Fragile: `navigate_with_text`/`relative_move` stall on
+CPU/LCM-RPC. If a dog hangs, fall back to 4a — coordinator state is shared, so scripted and
+autonomous dogs can even mix.
+
+---
+
+## 5. FAILURE → FIX
+
+| Symptom | Cause | Fix |
+|---|---|---|
+| Laptop-B can't reach coordinator | NAT / wrong URL / service order | `tailscale ping <A>`; uplink is default route; URL uses A's **tailscale** ip not 192.168.x |
+| Internet dead on dog AP | Wi-Fi above uplink in service order | reorder uplink above Wi-Fi; `route -n get default` = uplink iface |
+| `tailscale` not found (App Store) | CLI off PATH | `/Applications/Tailscale.app/Contents/MacOS/Tailscale` |
+| Dog unreachable | not on its AP / wrong ROBOT_IP | join dog Wi-Fi; `ping 192.168.12.1`; `dimos show-config` |
+| `dimos run` ignores robot ip | used a flag | unset; set `ROBOT_IP` env; `dimos show-config` |
+| EdgeTAM/CUDA crash on deploy | GPU-only modules | none needed — `unitree-go2-pack` already disables SecurityModule + PersonFollow |
+| Dog reaches goal but driver freezes | `relative_move` waits on flaky `is_goal_reached` RPC | use `scripted_pack_run --drive` (VelocityTeleop lane) |
+| `look_out_for`/moondream slow 10–60s | VLM on CPU | use `look_for_red` (GPU-free) on the critical path |
+| bravo stops before inheriting | next_zone empty while alpha still held south | run bravo with `--keep-polling` |
+| projector state stale | dashboard cache | refresh `http://<A-tailscale-ip>:8090` (it polls `/state`) |
+
+---
+
+## 6. Rehearse with ZERO dogs (do the night before)
+```bash
+uv run python -m dimos.experimental.pack_mind.demo_pack_live --pace 2     # http://localhost:8090
+# OR the real two-process story locally (no --drive):
+uv run python -m dimos.experimental.pack_mind.pack_coordinator_server \
+  --prefs "alpha:south,north,east,west;bravo:north,east,west,south" &
+uv run python -m dimos.experimental.pack_mind.scripted_pack_run --role alpha --leader \
+  --target-zone south --hold-zone --drop-on-enter
+uv run python -m dimos.experimental.pack_mind.scripted_pack_run --role bravo \
+  --target-zone south --find --keep-polling
+```
+
+## 7. Teardown
+`kill` the coordinator (holds the only shared state), Ctrl-C the `dimos run` daemons,
+`tailscale down` to leave the tailnet. Nothing persists server-side.

dimos/experimental/pack_mind/README.md

@@ -0,0 +1,155 @@
+# PACK MIND
+
+**One brain, many bodies, one memory that outlives any single dog.**
+
+A team of robot dogs explores an unknown space. The question the demo answers:
+**does *sharing memory* across the pack actually help?** We A/B it head-to-head — same
+dogs, same start, same map; the *only* difference is whether they share one discovered
+map or each keep a private one.
+
+## Pitch deck & diagrams
+
+**🎤 Pitch deck (browser):** <https://pack-mind.pages.dev/> — the idea, the two magic beats
+(handoff + inheritance), the A/B proof, and the fleet-memory-layer business case.
+
+**Share meaning, not maps** — a static zone partition is just a brittle special case of
+shared memory (it can't adapt to failure, discovery, or rebalancing):
+
+![Static partition vs shared memory](docs/pack_mind_partition_vs_memory.png)
+
+**How it works** — every dog, every tick: sense → remember → choose → plan → move, all
+over one shared memory; the A/B knob is whether dogs share that memory or each keep their own:
+
+![PACK MIND search loop and shared memory](docs/packmind_how_it_works.png)
+
+System design — one coordination layer, two substrates (sim shares *cells*, live shares
+*zones*, never coordinates): [`docs/packmind_system_design.png`](docs/packmind_system_design.png).
+
+## Thesis (and honest limits)
+
+- **Speed:** a pack sharing coverage/discovery memory avoids redundant re-searching, so
+  it searches the whole area faster.
+- **Resilience (the strongest point):** when a dog goes offline, its discoveries persist
+  in shared memory and the survivors keep using them. With private memory, that dog's
+  knowledge dies with it.
+- **Honest caveats:** this is a 2D-grid sim (rendered in 3D); point robots, scripted
+  sensing, no real SLAM/perception. The win is *complete-area search speed + knowledge
+  persistence* — **not** "finds victims faster" (redundant independent robots can stumble
+  on victims sooner). Keep the narration honest.
+
+## Two simulations
+
+### 1. Fog-of-war exploration (the current demo) — `explore_sim.py`
+Unknown maze, revealed by a raycast sensor (walls block line of sight, RTS-style). Dogs
+navigate to **frontiers** (known-free / unknown boundary). The discovered map is **shared**
+(one map all dogs read+write) or **independent** (one per dog; the team only sees the union
+of *online* dogs, so killing a dog erases its territory). Measured (seed 0, 3 dogs):
+shared reaches 95% revealed at **tick 246** vs **335** independent; killing a dog drops
+independent's team knowledge **0.80 → 0.32**, shared **drops 0**.
+
+### 2. Coverage race (earlier model) — `sim.py`
+Known map; dogs sweep with `CoveragePatrolRouter`; A/B = shared vs private
+`VisitationHistory`. Shared hits 75% coverage at tick 1745 vs 3275; final 96.5% vs 88.9%.
+
+Both reuse DimOS navigation primitives (`min_cost_astar`, patrol routers,
+`VisitationHistory`) on a fabricated `OccupancyGrid` — pure numpy/scipy, **no CUDA/ROS/sim**.
+
+## Run it
+
+```bash
+# Web 3D demo (FastAPI + Three.js) — the centerpiece. Side-by-side fog-of-war race,
+# 3-level fog (visible / remembered / unknown), kill-a-dog + reset controls.
+uv run python -m dimos.experimental.pack_mind.server      # → http://localhost:8000
+
+# DimOS Viewer (Rerun) — one maze, the pack exploring on the shared map. Scrub the
+# "tick" timeline; add --kill DOG TICK for the resilience beat, --save run.rrd to skip the GUI.
+uv run python -m dimos.experimental.pack_mind.view_explore_rerun --dogs 3 --seed 0
+
+# Native DimOS blueprint — coverage A/B as two live OccupancyGrid streams in Rerun.
+dimos --viewer rerun run pack-mind-sim
+
+# Standalone A/B video of the coverage race.
+uv run python -m dimos.experimental.pack_mind.render --out pack_mind_ab.mp4
+
+# Tests
+bin/pytest-fast dimos/experimental/pack_mind/test_explore_sim.py -v
+bin/pytest-fast dimos/experimental/pack_mind/test_pack_mind_sim.py -v
+```
+
+## Live demo — 2 dogs / 2 laptops
+
+The intelligence (shared memory) runs on the laptops; the dogs are bodies. Movement
+is teleop-assisted; the agent + coordinator own the memory layer.
+
+**1. One-time per laptop (with internet):** cache the runtime models so the demo runs
+offline (a dog's WiFi AP has no internet, and venue WiFi is flaky):
+```bash
+uv run python -m dimos.experimental.pack_mind.prefetch_live_models   # moondream2 + whisper
+```
+CLIP/YOLO ship in `data/` (git-lfs). The agent brain (GPT-4o) is a hosted API — keep
+the coordinator laptop on a router with WAN.
+
+**2. Network:** one travel router; put each Go2 in STA mode (Unitree app) onto it; both
+laptops + both dogs on that subnet. DimOS picks the WebRTC method purely from
+`ROBOT_IP` — `192.168.12.1` → the dog's own AP (single-laptop only), any other IP →
+LocalSTA (shared router, required for 2 laptops).
+
+**3. Laptop A (Alpha + coordinator + dashboard):**
+```bash
+export HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1
+export ROBOT_IP=<dog-A-ip> LISTEN_HOST=0.0.0.0 OPENAI_API_KEY=<key>
+uv run python -m dimos.experimental.pack_mind.pack_coordinator_server \
+  --zones north,east,south,west --prefs "alpha:north,east;bravo:south,west" &   # dashboard: http://<laptopA-ip>:8090
+PACK_DOG_NAME=alpha PACK_COORDINATOR_URL=http://127.0.0.1:8090 \
+  uv run dimos run unitree-go2-pack --daemon
+```
+
+**4. Laptop B (Bravo):**
+```bash
+export HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1
+export ROBOT_IP=<dog-B-ip> LISTEN_HOST=0.0.0.0 OPENAI_API_KEY=<key>
+PACK_DOG_NAME=bravo PACK_COORDINATOR_URL=http://<laptopA-ip>:8090 \
+  uv run dimos run unitree-go2-pack --daemon
+```
+
+**Gotchas (hard-won on site):**
+- `unitree-go2-pack` already disables EdgeTAM modules (SecurityModule, PersonFollow) —
+  they hard-require CUDA. No `--disable` flag needed.
+- `dimos run` has **no `--robot-ip` flag** in this build — set the `ROBOT_IP` env and
+  verify with `dimos show-config`.
+- `HF_HUB_OFFLINE=1` is required on a no-internet AP; run prefetch first.
+- **Detection: use `look_for_red`** (RedObjectDetector) — a fast, GPU-free colour check
+  that reports the finding to the coordinator instantly. moondream/`look_out_for` also
+  works but is slow on a CPU host (~10–60s first call); keep it off the critical path.
+- Movement: `relative_move` (skill) or keyboard teleop. `navigate_with_text` works but is
+  slow on CPU (Qwen).
+- Bring-up order: ping dog → `dimos show-config` → run to "running" → `mcp list-tools` +
+  `speak` → `relative_move` → `look_for_red` → `start_search`/`next_zone` (dashboard at :8090).
+
+**Hardware-free rehearsal (no dogs):**
+`uv run python -m dimos.experimental.pack_mind.demo_pack_live --pace 2`, open http://localhost:8090.
+
+## File map
+
+| File | Role |
+|---|---|
+| `world.py` | Fabricate the maze `OccupancyGrid` + plant survivors |
+| `explore_sim.py` | **Fog-of-war exploration engine** (raycast reveal, frontier, shared/private discovered map, offline persistence) |
+| `server.py` | FastAPI WebSocket backend streaming both explore sims |
+| `static/explore.html` | Three.js 3D fog-of-war frontend (side-by-side, kill/reset) |
+| `view_explore_rerun.py` | **DimOS Viewer (Rerun)** view of one shared-memory maze search (fog + dogs + trails + coverage scalar) |
+| `live.py` | **Live blueprint** `unitree-go2-pack` (one dog per laptop, EdgeTAM disabled) + PACK system prompt |
+| `pack_coordinator.py` / `pack_coordinator_server.py` | Shared zone ledger (no-overlap, find, **inheritance**) + JSON/HTTP API + dashboard route |
+| `pack_dashboard.html` | Projector dashboard — zones, finding, offline/inheritance, causal chain |
+| `pack_search_skills.py` | Dog agent tools: `start_search` / `next_zone` / `report_*` / `where_is` |
+| `red_detector.py` | **Fast GPU-free red-object detector** (`look_for_red`) — HSV-free colour test → auto-reports the find |
+| `pack_search_runner.py` | `RobotDriver`-protocol search loop + `MockDriver` |
+| `mock_dog.py` / `demo_pack_live.py` / `demo_pack_scene.py` | Hardware-free test + projector rehearsal of both magic beats |
+| `prefetch_live_models.py` | Pre-cache moondream2 + whisper so the live stack runs offline |
+| `sim.py` / `sim_robot.py` | Coverage-race sim (known map) |
+| `blueprint.py` | `pack-mind-sim` native DimOS blueprint (publishes coverage as `OccupancyGrid`) |
+| `render.py` | Standalone matplotlib → mp4 A/B render |
+| `view_rerun.py` | Log the coverage A/B into a Rerun `.rrd` / live viewer |
+| `demo_spike.py` | Feasibility spike (kept as a smoke test of the reused primitives) |
+| `test_explore_sim.py` / `test_pack_mind_sim.py` | Engine tests |
+| `sim_perception.py` | MuJoCo+VLM single-frame perception probe (optional, off the critical path) |

dimos/experimental/pack_mind/__init__.py

@@ -0,0 +1,15 @@
+# Copyright 2025-2026 Dimensional Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""PACK MIND — a shared semantic memory for a team of Unitree Go2s."""