Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c913b62
update
CeciliaZ030 May 26, 2026
6889392
mcp optimization
CeciliaZ030 May 26, 2026
106afa6
webapp scafold
CeciliaZ030 May 26, 2026
1985fb6
fix normal macos
yichulau May 26, 2026
f142e71
Merge pull request #1 from CeciliaZ030/yc/fix-macos
yichulau May 26, 2026
7ed2063
webapp: replace scaffold with Goldie (voice + manual control app)
mkuwdev May 27, 2026
f7a599d
webapp: speak agent replies via OpenAI TTS (iOS-reliable)
mkuwdev May 27, 2026
7144af2
webapp: track package.json so the openai TTS dep is captured
mkuwdev May 27, 2026
dbcecc3
fix
yichulau May 27, 2026
bfd889f
stairs update
yichulau May 28, 2026
337c026
Merge pull request #5 from CeciliaZ030/yc/stairs
yichulau May 28, 2026
3b4d8ed
webapp: filter tool noise, tune TTS, harden STT + network, barge-in, …
mkuwdev May 28, 2026
2b9cb36
Merge pull request #4 from CeciliaZ030/goldie-webapp
mkuwdev May 28, 2026
b78e85b
backend: forward LLM agent replies on agent_responses as typed envelopes
mkuwdev May 28, 2026
a6ac132
backend: forward LLM agent replies on agent_responses as typed envelopes
mkuwdev May 28, 2026
aebb67c
Merge pull request #6 from CeciliaZ030/goldie-agent-envelopes
mkuwdev May 28, 2026
0560913
docs: hackathon submission README + architecture diagram + techflow
mkuwdev May 28, 2026
25b076a
Merge pull request #7 from CeciliaZ030/goldie-webapp
mkuwdev May 28, 2026
fad6445
docs: update README with team name, video link, numbered challenges, …
mkuwdev May 28, 2026
d22d640
Merge pull request #8 from CeciliaZ030/goldie-webapp
mkuwdev May 28, 2026
44e0d40
docs: fix demo video link
mkuwdev May 28, 2026
f8049ea
Merge pull request #9 from CeciliaZ030/goldie-webapp
mkuwdev May 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ __pycache__

# node env (used by devcontainers cli)
node_modules
package.json
package-lock.json
!docs/package.json
!docs/package-lock.json
Expand Down
340 changes: 340 additions & 0 deletions DOG-PHONE-INTERFACE.md

Large diffs are not rendered by default.

355 changes: 92 additions & 263 deletions README.md

Large diffs are not rendered by default.

3 changes: 0 additions & 3 deletions assets/dimensional.command-center-extension-0.0.1.foxe

This file was deleted.

3 changes: 0 additions & 3 deletions assets/dimos_interface.gif

This file was deleted.

3 changes: 0 additions & 3 deletions assets/dimos_terminal.png

This file was deleted.

3 changes: 0 additions & 3 deletions assets/framecount.mp4

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/agentic_control.gif

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/agents.png

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/dimos_demo.gif

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/lidar.gif

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/lidar.png

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/navigation.gif

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/navigation.png

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/perception.png

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/spacer.png

This file was deleted.

3 changes: 0 additions & 3 deletions assets/readme/spatial_memory.gif

This file was deleted.

3 changes: 0 additions & 3 deletions assets/simple_demo.mp4

This file was deleted.

3 changes: 0 additions & 3 deletions assets/simple_demo_small.gif

This file was deleted.

3 changes: 0 additions & 3 deletions assets/trimmed_video_office.mov

This file was deleted.

162 changes: 162 additions & 0 deletions bin/run-demo.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
#!/usr/bin/env bash
# Boot the guide-lite demo: dimos + webapp + tailscale serve.
# Retries dimos until the MCP race resolves (tools register within 30s).
#
# Usage: bin/run-demo.sh
# Stop: bin/run-demo.sh stop

set -uo pipefail
cd "$(git rev-parse --show-toplevel)"

BLUEPRINT="${BLUEPRINT:-unitree-go2-guide-lite}"
DIMOS_PORT="${DIMOS_PORT:-5555}"
WEBAPP_PORT="${WEBAPP_PORT:-3000}"
TAILNET_HTTPS_DIMOS="${TAILNET_HTTPS_DIMOS:-8443}"
TAILNET_HTTPS_WEBAPP="${TAILNET_HTTPS_WEBAPP:-443}"
TOOL_REGISTRATION_TIMEOUT="${TOOL_REGISTRATION_TIMEOUT:-30}"
MAX_RETRIES="${MAX_RETRIES:-10}"

RUN_DIR="${RUN_DIR:-/tmp/dimos_run}"
mkdir -p "$RUN_DIR"
DIMOS_LOG="$RUN_DIR/dimos.log"
WEBAPP_LOG="$RUN_DIR/webapp.log"
TOKEN_FILE="$RUN_DIR/api_token"

red() { printf "\033[31m%s\033[0m\n" "$*"; }
green() { printf "\033[32m%s\033[0m\n" "$*"; }
amber() { printf "\033[33m%s\033[0m\n" "$*"; }
blue() { printf "\033[34m%s\033[0m\n" "$*"; }

stop_all() {
amber "stopping demo..."
pkill -f "dimos.*run $BLUEPRINT" 2>/dev/null || true
pkill -f "next dev" 2>/dev/null || true
tailscale serve --https="$TAILNET_HTTPS_DIMOS" off 2>/dev/null || true
tailscale serve --https="$TAILNET_HTTPS_WEBAPP" off 2>/dev/null || true
green "stopped."
}

if [[ "${1:-}" == "stop" ]]; then
stop_all
exit 0
fi

# ---- 0) token
if [[ ! -s "$TOKEN_FILE" ]]; then
openssl rand -hex 16 > "$TOKEN_FILE"
fi
TOKEN="$(cat "$TOKEN_FILE")"
blue "API token: $TOKEN (file: $TOKEN_FILE)"

# ---- 1) preflight: required env, ports free, sudoers for route
command -v dimos >/dev/null 2>&1 || command -v .venv/bin/dimos >/dev/null 2>&1 \
|| { red "dimos CLI not found — activate venv or install"; exit 1; }
DIMOS_BIN="$(command -v dimos 2>/dev/null || echo .venv/bin/dimos)"

[[ -n "${OPENAI_API_KEY:-}" ]] || { red "OPENAI_API_KEY not set"; exit 1; }

if sudo -n route -h >/dev/null 2>&1; then
: # NOPASSWD configured
elif sudo -n true 2>/dev/null; then
: # cached sudo
else
amber "sudo may prompt for route configuration"
fi

# ---- 2) boot dimos in a retry loop until tools register
blue "booting dimos ($BLUEPRINT)..."
for attempt in $(seq 1 "$MAX_RETRIES"); do
pkill -f "dimos.*run $BLUEPRINT" 2>/dev/null || true
sleep 2

nohup env DIMOS_API_TOKEN="$TOKEN" "$DIMOS_BIN" --replay run "$BLUEPRINT" \
> "$DIMOS_LOG" 2>&1 &
DIMOS_PID=$!
echo "$DIMOS_PID" > "$RUN_DIR/dimos.pid"

amber " attempt $attempt: pid=$DIMOS_PID watching for 'Discovered tools'..."

start=$SECONDS
ok=""
while (( SECONDS - start < TOOL_REGISTRATION_TIMEOUT )); do
if ! kill -0 "$DIMOS_PID" 2>/dev/null; then
red " pid $DIMOS_PID died — see $DIMOS_LOG"
break
fi
if grep -q "Discovered tools from MCP server" "$DIMOS_LOG" 2>/dev/null; then
n=$(grep "Discovered tools from MCP server" "$DIMOS_LOG" | tail -1 \
| sed -E 's/.*n_tools=([0-9]+).*/\1/')
if [[ -n "$n" && "$n" -gt 0 ]]; then
green " ✓ tools registered (n_tools=$n) on attempt $attempt"
ok=1
break
fi
fi
sleep 1
done

if [[ "$ok" == "1" ]]; then
break
fi
amber " attempt $attempt failed — restarting"
done

if [[ "$ok" != "1" ]]; then
red "dimos never registered tools after $MAX_RETRIES attempts. Check $DIMOS_LOG"
exit 1
fi

# ---- 3) bring up tailscale serve (idempotent)
blue "configuring tailscale serve..."
TAILNET_HOST="$(tailscale status --self --json 2>/dev/null \
| python3 -c 'import json,sys;d=json.load(sys.stdin);print(d["Self"]["DNSName"].rstrip("."))' 2>/dev/null \
|| echo "$(hostname)")"

# serve dimos backend on :8443, webapp on :443
tailscale serve --https="$TAILNET_HTTPS_DIMOS" "$DIMOS_PORT" >/dev/null 2>&1 || true
tailscale serve --bg --https="$TAILNET_HTTPS_DIMOS" "$DIMOS_PORT" >/dev/null 2>&1 || true
tailscale serve --https="$TAILNET_HTTPS_WEBAPP" "$WEBAPP_PORT" >/dev/null 2>&1 || true
tailscale serve --bg --https="$TAILNET_HTTPS_WEBAPP" "$WEBAPP_PORT" >/dev/null 2>&1 || true

API_URL="https://${TAILNET_HOST}:${TAILNET_HTTPS_DIMOS}"
WEBAPP_URL="https://${TAILNET_HOST}"

# ---- 4) write webapp env + boot dev server if not already running
if ! lsof -nP -iTCP:"$WEBAPP_PORT" -sTCP:LISTEN >/dev/null 2>&1; then
blue "starting webapp dev server..."
cat > webapp/.env.local <<EOF
NEXT_PUBLIC_DIMOS_API=$API_URL
NEXT_PUBLIC_DIMOS_TOKEN=$TOKEN
EOF
( cd webapp && nohup npm run dev > "$WEBAPP_LOG" 2>&1 & echo $! > "$RUN_DIR/webapp.pid" )
disown
start=$SECONDS
while (( SECONDS - start < 30 )); do
grep -q "Ready in" "$WEBAPP_LOG" 2>/dev/null && break
sleep 1
done
fi

# ---- 5) sanity probe
TOKEN="$(cat "$TOKEN_FILE")"
unset HTTP_PROXY HTTPS_PROXY ALL_PROXY
streams=$(curl -fsS -H "Authorization: Bearer $TOKEN" "$API_URL/text_streams" 2>/dev/null || echo "FAIL")
if [[ "$streams" == *"agent_state"* ]]; then
green " ✓ dimos reachable via tailnet HTTPS"
else
amber " ! could not reach $API_URL/text_streams — check tailscale"
fi

# ---- 6) print connection details
echo
green "===================== READY ====================="
echo " webapp: $WEBAPP_URL"
echo " api: $API_URL"
echo " token: $TOKEN"
echo " log: $DIMOS_LOG"
echo " pid: $(cat "$RUN_DIR/dimos.pid")"
echo
echo " open the webapp URL in iPhone Safari (Tailscale ON)"
echo " stop with: bin/run-demo.sh stop"
echo " tail logs: tail -f $DIMOS_LOG"
green "================================================="
3 changes: 0 additions & 3 deletions data/.lfs/ab_lidar_frames.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/apartment.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/assets.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/astar_corner_min_cost.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/astar_min_cost.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/big_office.ply.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/big_office_height_cost_occupancy.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/big_office_simple_occupancy.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/cafe-smol.jpg.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/cafe.jpg.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/chair-image.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/command_center.html.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/drone.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/expected_occupancy_scene.xml.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/g1_wholebody_replay.json.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/g1_zed.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/go2_bigoffice.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/go2_china_office.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/go2_hongkong_office.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/go2_hongkong_office_twopass_map.pc2.lcm.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/go2_sf_office.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/go2_slamabuse1.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/go2_slamabuse2.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/gradient_simple.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/gradient_voronoi.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/hk_building_all_around.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/hk_building_elevator.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/hk_building_park.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/hk_village1.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/hk_village3.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/hk_village4.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/hk_village6.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/inflation_simple.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/lcm_msgs.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/make_navigation_map_mixed.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/make_navigation_map_simple.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/make_path_mask_full.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/make_path_mask_two_meters.png.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/markers_go2.db.tar.gz

This file was deleted.

3 changes: 0 additions & 3 deletions data/.lfs/models_contact_graspnet.tar.gz

This file was deleted.

Loading