A face for your AI agent.
A desktop VRM character that chats, speaks, listens, lip-syncs, and emotes — on macOS.
English · 繁體中文
小光 lives on your desktop as a 3D VRM avatar — she chats with you, speaks and listens (hands-free, and you can talk right over her), lip-syncs and emotes, and proactively strikes up conversations when you've been quiet.
AniCompanion doesn't ship an LLM; it's the character, voice, and presence layer in front of an agent you run. Any gateway that streams chat completions can drive it — backends are pluggable (Bring your own agent). Hermes Agent is the reference backend, validated end-to-end and runnable locally.
| English | 繁體中文 |
|---|---|
 |
 |
Status: functional, early-stage. Built and tested on macOS 15. Contributions welcome.
- 3D VRM character via three-vrm (WebGL in a WKWebView) — spring-bone physics (hair/skirt), idle breathing/blink, and skeletal gesture clips.
- Streaming chat through a pluggable agent backend: Hermes Agent (the validated reference) or a generic OpenAI-compatible one (Ollama, LM Studio, vLLM, OpenRouter, …).
- Talk & be talked to — she speaks with amplitude-driven lip sync, and you reply by voice: push-to-talk, hands-free (just talk), or full-duplex (interrupt her mid-sentence). See Voice setup.
- Pluggable voice providers — TTS: Apple on-device (default, no key), MiniMax, OpenAI, or local BlueMagpie. STT: Apple on-device (default) or cloud Whisper (Groq / OpenAI / OpenAI-compatible).
- Screen vision (opt-in, off by default) — let 小光 see your focused window (or whole screen) so she can react to what you're working on. Needs a vision-capable model + Screen Recording permission.
- 16 emotions — emotion tags from the LLM drive the avatar's facial expressions.
- Proactive companion — greets you on launch and speaks up after a quiet spell.
- Desktop Pet mode — pop 小光 out into a transparent, always-on-top desktop overlay; drag to move, scroll/pinch to resize. See Desktop Pet mode.
- Multilingual — ships in English and Traditional Chinese (繁體中文), switchable in Settings (interface and the language 小光 speaks).
Talk to 小光 and she talks back, hands-free — no clicking the mic, and you can interrupt her by voice just like a real conversation.
- 🍎 Apple on-device TTS — the new default. Zero setup: no API key, no network, fully private, and it works out of the box. (Cloud MiniMax/OpenAI and local BlueMagpie are still there if you prefer them.) Tip: download a higher-quality system voice for a big quality bump — see Voice setup → Apple TTS.
- 🎙️ Hands-free mode. She keeps listening and replies automatically — just talk, no button.
- 🗣️ Full-duplex barge-in (opt-in) — talk over her and she stops to listen, using acoustic echo cancellation so she doesn't hear herself. (She only takes over the audio device while she's actually speaking, so other apps aren't muted the rest of the time.)
Full history: CHANGELOG.md · Releases.
- macOS 15.0+, Apple Silicon
- Xcode 16 (Swift 6 toolchain)
- XcodeGen —
brew install xcodegen - A running agent gateway — Hermes Agent is the validated path
- Voice and vision work with the defaults (on-device, no accounts). Cloud providers are optional — see Voice setup.
# 1. Generate the Xcode project
xcodegen generate
# 2. Download the default VRM character model (not bundled — see ATTRIBUTION.md)
./scripts/download-model.sh
# 3. Build & run
open AniCompanion.xcodeproj # then Run (⌘R) in Xcode
# …or: xcodebuild -project AniCompanion.xcodeproj -scheme AniCompanion -destination 'platform=macOS' buildOn first launch, open Settings (⚙️) and set your Agent backend — its Endpoint (default
http://127.0.0.1:8642) and API Key — pointing at a gateway that's already running (see below).
Voice works out of the box on Apple's on-device engines; to use a cloud voice provider or tune the voice
modes, see Voice setup.
First launch needs internet — the three-vrm runtime loads from a CDN once, then caches. When it works you'll see 小光 appear and greet you. If she never shows, see Troubleshooting.
AniCompanion talks to an agent gateway you run yourself. Two backends are built in, under Settings → Agent backend:
- Hermes Agent — the reference backend, validated end-to-end.
- OpenAI-compatible — any gateway speaking
/v1/chat/completionsSSE: Ollama, LM Studio, vLLM, OpenRouter, and friends.
Hermes in brief — in ~/.hermes/.env set API_SERVER_ENABLED=true and API_SERVER_KEY=<your-key>
(openssl rand -hex 32), run hermes gateway (→ http://127.0.0.1:8642), and put the same
endpoint + key in Settings. Full walkthrough (incl. optional MCP tools):
docs/hermes-setup.md. Adding a new backend is a one-case change —
CONTRIBUTING.md.
Detach 小光 into a borderless, transparent, always-on-top companion that floats over your other apps. There's no chat panel — a small speech bubble shows what she's saying. Toggle with the 🐾 toolbar button, Character ▸ Desktop Pet Mode, or ⌘⇧D; double-click her to return. Drag to move, scroll/pinch to resize. Your conversation is untouched while she's out.
- Voice setup — TTS & STT providers, hands-free & full-duplex modes, downloading better voices
- VRM model guide — the default model, using your own, what a model needs
- Hermes setup — the reference agent gateway, MCP tools, diagnostics
- Privacy — exactly what stays local and what a cloud option sends
- Contributing — add a backend, a voice provider, or a language
- Architecture & developer notes — how the streaming voice pipeline fits together
- Changelog
| Symptom | Likely cause / fix |
|---|---|
xcodegen: command not found |
brew install xcodegen. |
| Window opens but the character never appears | First launch needs internet (three-vrm loads from a CDN); also confirm ./scripts/download-model.sh ran and a .vrm is in AniCompanion/Resources/VRMModel/. |
| You type and nothing happens | Your agent gateway isn't running / reachable. Start it and check the Settings connection indicator. For Hermes, a 401 means the API Key doesn't match API_SERVER_KEY. |
| She replies in text but doesn't speak | TTS is off, or her voice sounds robotic — both covered in Voice setup. |
| Voice input does nothing | Allow Microphone + Speech Recognition on first use (System Settings → Privacy & Security). Cloud Whisper: check endpoint/key/model. More in Voice setup. |
Application source code: MIT — see LICENSE. Bundled/downloaded assets (VRM model,
animation clips) are third-party works under their own terms — see ATTRIBUTION.md.
The default VRM model is not MIT-licensed and is not redistributed by this project.