Pegasus foundation — Expo data layer, Modal TRIBE ML, NVIDIA video

.claude/agents/dhruva-signals.md

@@ -0,0 +1,21 @@
+---
+name: dhruva-signals
+description: Dhruva's signals agent for Pegasus. Use for work in /signals — behavioral sentiment analysis (Claude), typing dynamics, the :8002 FastAPI service, Twilio SMS alerts, and the collector.js keystroke tracker. MUST stay inside signals/.
+tools: Read, Write, Edit, Bash, Grep, Glob
+---
+
+You are **Dhruva**, the signals engineer on the Pegasus team.
+
+**Hard boundary:** You may ONLY create/edit files inside `signals/`. Never touch
+`frontend/`, `backend/`, `ml/`, or `shared/`. Ask Jason for contract changes.
+
+**Your service** (FastAPI on :8002): `/health`, `/signals/analyze` (Claude
+`claude-sonnet-4-6` sentiment + energy + flags + `combined_signal_score`, with a
+lexical fallback when no API key), `/signals/{user_id}`, `/alert` (Twilio SMS on
+red; no-ops to console when creds are absent so the demo never crashes).
+
+You also own `signals/collector.js` (`KeystrokeTracker`), which the frontend
+imports — a synced copy lives at `frontend/src/keystroke.js`. Keep them in sync.
+
+Read `signals/AGENTS.md`, `AGENTS.md`, and `shared/contract.md` first.
+Stage only `signals/`. Commit to `feat/dhruva-sig`, PR into `dev`.

.claude/agents/imessage-bridge.md

@@ -0,0 +1,27 @@
+---
+name: imessage-bridge
+description: Agent for the standalone /imessage bridge. Use for work on the macOS iMessage delivery channel — AppleScript send, chat.db polling, the bridge CLI/loop, phone↔user registry. MUST stay inside imessage/ and only call the backend API.
+tools: Read, Write, Edit, Bash, Grep, Glob
+---
+
+You own the **`imessage/` bridge** — a standalone macOS delivery channel for
+Pegasus (owner TBD; not one of the four service lanes).
+
+**Hard boundary:** ONLY create/edit files inside `imessage/`. Never touch
+`frontend/`, `backend/`, `ml/`, `signals/`, or `shared/`. Talk to the **backend
+only** (:8001): `POST /users`, `GET /stimulus/today`, `POST /checkin`.
+
+**How it works (macOS, no API):** send iMessages via AppleScript driving
+Messages.app (`send.applescript` + `imessage.send`); receive by polling
+`~/Library/Messages/chat.db` read-only (`imessage.fetch_new`), decoding
+`attributedBody` when the `text` column is NULL. `bridge.py` provides
+`register` / `send` / `send-daily` / `listen`. `registry.py` maps phone ⇄
+user_id and auto-registers new senders.
+
+**Constraints to respect:** needs Full Disk Access + a signed-in iMessage
+account on the Mac; AppleScript send syntax is macOS-version-sensitive; no
+keystroke dynamics over iMessage (typing metrics sent as 0). Keep it runnable
+and never replay old chat history (start from current max ROWID).
+
+Read `imessage/AGENTS.md`, `AGENTS.md`, and `shared/contract.md` first.
+Stage only `imessage/`.

.claude/agents/jason-backend.md

@@ -0,0 +1,22 @@
+---
+name: jason-backend
+description: Jason's backend agent for Pegasus. Use for work in /backend and /shared — the :8001 orchestrator, SQLite persistence, daily stimuli, and the API contract. MUST stay inside backend/ and shared/.
+tools: Read, Write, Edit, Bash, Grep, Glob
+---
+
+You are **Jason**, the backend engineer and contract owner on the Pegasus team.
+
+**Hard boundary:** You may ONLY create/edit files inside `backend/` and
+`shared/`. Never touch `frontend/`, `ml/`, or `signals/` — call their HTTP APIs.
+
+**Your service** (FastAPI on :8001) is the orchestrator and the ONLY service
+the frontend talks to: `/health`, `/users`, `/stimulus/today`, `/checkin`
+(fans out to Signals :8002 + ML :8003, persists, alerts on red), `/status/{id}`.
+
+**You own the contract.** `shared/contract.md` is the source of truth. When you
+change it, note which service owner is affected so they stay shape-compatible.
+Keep `orchestrator.py` degrading gracefully when a downstream service is down —
+the demo must never hard-fail.
+
+Read `backend/AGENTS.md`, `AGENTS.md`, and `shared/contract.md` first.
+Stage only `backend/` and `shared/`. Commit to `feat/jason-api`, PR into `dev`.

.claude/agents/rishith-ml.md

@@ -0,0 +1,29 @@
+---
+name: rishith-ml
+description: Rishith's ML + Video agent for Pegasus. Use for work in /ml (TRIBE v2 brain prediction, the 4-stream combined burnout scorer, Claude interventions, :8003) and /video (MediaPipe facial + Whisper voice stress, :8004). MUST stay inside ml/ and video/.
+tools: Read, Write, Edit, Bash, Grep, Glob
+---
+
+You are **Rishith**, the ML / brain-pipeline / video engineer on the Pegasus team.
+
+**Hard boundary:** You may ONLY create/edit files inside `ml/` and `video/`.
+Never touch `frontend/`, `backend/`, `signals/`, or `shared/`. Call their HTTP
+API or read (never edit) `shared/contract.md`. If you need a contract change,
+report it for Jason — don't edit shared/.
+
+**ML service (:8003):** `/health`, `/predict` (TRIBE v2 healthy-brain prediction
++ per-region activations), `GET /baseline/{stimulus_id}` (cached, for Brain
+View), `/score` (merges imessage + typing + facial + voice + tribe streams via
+`combined_scorer.py`, renormalizing over whichever are present). Intervention
+(`claude_interpreter.py`, `claude-sonnet-4-6`, offline fallback) is part of `/score`.
+
+**Video service (:8004):** `/health`, `/analyze/video` (facial+voice),
+`/analyze/frame`. MediaPipe Face Mesh (`facial_analyzer.py`) + Whisper/librosa
+(`voice_analyzer.py`); heavy models load lazily so `/health` is instant.
+
+**Headline task:** `ml/tribe_inference.py` ships a seeded STUB (`TribeModel._infer`).
+Wire real TRIBE v2 inference there, keeping the return shape so the pipeline is
+unaffected. Keep both services' `/health` green and runnable at every commit.
+
+Read `ml/AGENTS.md`, `video/AGENTS.md`, `AGENTS.md`, and `shared/contract.md`
+first. Stage only `ml/` and `video/`. Commit to `feat/rishith-ml`, PR into `dev`.

.claude/agents/wesley-frontend.md

@@ -0,0 +1,23 @@
+---
+name: wesley-frontend
+description: Wesley's frontend agent for Pegasus. Use for work in /frontend — the React (Vite) UI on :3000, the check-in flow, keystroke capture, and the Check Engine Light component. MUST stay inside frontend/.
+tools: Read, Write, Edit, Bash, Grep, Glob
+---
+
+You are **Wesley**, the frontend engineer on the Pegasus team.
+
+**Hard boundary:** You may ONLY create/edit files inside `frontend/`. Never
+touch `backend/`, `ml/`, `signals/`, or `shared/`. You talk to the **backend
+only** (:8001) through the Vite `/api` proxy (already configured in
+`vite.config.js`) — never call ml/signals directly.
+
+**The UI flow:** create/get user → `GET /api/stimulus/today` → show stimulus →
+capture response with `KeystrokeTracker` (`src/keystroke.js`) → `POST
+/api/checkin` → render the Check Engine Light (🟢/🟡/🔴) + intervention via
+`EngineLight.jsx`. Keep `src/keystroke.js` in sync with Dhruva's
+`signals/collector.js`.
+
+Run with `npm install && npm run dev` (port 3000). Keep it building.
+
+Read `frontend/AGENTS.md`, `AGENTS.md`, and `shared/contract.md` first.
+Stage only `frontend/`. Commit to `feat/wesley-ui`, PR into `dev`.

.gitignore

@@ -40,3 +40,14 @@ wandb/
 
 # Logs
 *.log
+
+# iMessage bridge runtime (registry.json holds phone numbers — never commit)
+imessage/.state.json
+imessage/registry.json
+imessage/*.scpt
+
+# Expo / React Native
+.expo/
+frontend/.expo/
+frontend/web-build/
+*.tsbuildinfo

AGENTS.md

@@ -0,0 +1,59 @@
+# Pegasus — Agent Rules (root)
+
+Read this, then read the `AGENTS.md` in **your** folder. Every teammate runs
+their own coding agent; these files keep agents in their lane.
+
+## What Pegasus is
+A mental-health "check engine light." Four signal streams (SMS responses,
+typing biometrics, video facial/voice, and the in-app check-in) feed
+**TRIBE v2** (a brain foundation model deployed on Modal). We compare the user's
+real behavioral signals against TRIBE's healthy-brain prediction → deviation =
+burnout score → 🟢/🟡/🔴 + a Claude intervention → SMS alert if red.
+
+## Team & services
+| Owner   | Folder(s)                                   | Port(s)        |
+|---------|---------------------------------------------|----------------|
+| Rishith | frontend **data layer** + `ml/` + `video/`  | 3000, 8003, 8004 |
+| Wesley  | frontend **screens/components**             | 3000           |
+| Jason   | `backend/` + `shared/`                      | 8001           |
+| Dhruva  | `signals/` (Twilio SMS bot + alerts)        | 8002           |
+
+`frontend/` is an **Expo** app shared by Rishith (services/hooks/types) and
+Wesley (App.tsx/screens/components/navigation/utils) — see `frontend/AGENTS.md`.
+`imessage/` is a legacy macOS bridge, **superseded by Twilio SMS in `signals/`**.
+
+## The 5 rules (non-negotiable)
+1. **Stay in your files.** Only edit what you own (table above + per-folder
+   AGENTS.md). Need something from another area? **Call its API / import it — never edit it.**
+2. **The contract is law.** `shared/contract.md` is the source of truth. Need a
+   field changed? Jason changes it there first, then tells the owner.
+3. **Branch + PR.** Work on `feat/<you>-*`, PR into `dev`. Never push to `main`.
+   Stage only your files — never `git add .` / `git add -A`.
+4. **Stay runnable.** Your service must start on its port and answer `GET /health`.
+5. **Sync every ~45 min** (below) so integration never drifts far.
+
+## The 45-minute sync (everyone, every ~45 min)
+```bash
+# 1. Push your work + open/merge a PR into dev
+git add YOUR_FILES_ONLY/        # e.g. Rishith: ml/ video/ frontend/src/{services,hooks,types}/
+git commit -m "feat: what you did"
+git push origin feat/<you>
+#    → open PR feat/<you> → dev on GitHub, merge it
+
+# 2. Pull everyone else's merged work back into your branch
+git checkout dev && git pull origin dev
+git checkout feat/<you> && git merge dev
+# keep building
+```
+
+## Secrets
+Each service reads keys from a `.env` in its folder (gitignored). NEVER commit
+`.env` or put keys in code. Templates: `*/.env.example`.
+
+## Run locally
+```bash
+./dev.sh                 # ml + signals + backend (+ WITH_VIDEO=1 for video)
+cd frontend && npx expo start
+```
+The frontend (Expo) talks ONLY to the backend (:8001); backend fans out to
+signals + ml; ml calls TRIBE on Modal; video runs facial/voice on :8004.

README.md

@@ -1,28 +1,141 @@
-# pegasus
+# Pegasus
 
-## Branching model
+Pegasus is a multimodal burnout detection and mental wellness platform. It works like a check engine light for the mind — passively collecting behavioral signals throughout the day, fusing them into a single wellness score, and surfacing a grounded, personalized intervention when that score drops.
+
+## Architecture
+
+Pegasus is composed of four services that communicate over HTTP:
+
+| Service | Port | Stack | Owner |
+|---|---|---|---|
+| Backend orchestrator | 8001 | FastAPI, SQLite | Jason |
+| Signals processor | 8002 | FastAPI, HuggingFace | Dhruva |
+| ML / TRIBE scorer | 8003 | FastAPI, Modal, HuggingFace | Rishith |
+| Video analyzer | 8004 | FastAPI | Rishith |
+
+The Expo React Native mobile app connects to the backend at port 8001. The SMS bot runs alongside the signals service and polls iMessage via a local AppleScript bridge.
+
+## Features
+
+### Multimodal Wellness Scoring
+
+The scoring engine fuses up to five behavioral streams into a single wellness score from 0 to 100 (higher is better). Each stream contributes a burnout sub-score that is weighted and normalized depending on which signals are actually present in a given check-in.
+
+| Stream | Weight | Signal |
+|---|---|---|
+| iMessage / text sentiment | 25% | HuggingFace emotion model (j-hartmann/emotion-english-distilroberta-base) |
+| Typing biometrics | 20% | WPM, error rate, hesitation count, key hold time, flight time, correction loops |
+| Facial stress | 30% | Video frame analysis via the video service |
+| Voice stress | 15% | Audio analysis via the video service |
+| TRIBE deviation | 10% | Distance from the user's healthy brain-activity baseline |
+
+The score is inverted from a burnout reading so that rising numbers always mean the user is doing better. Scores above 70 are green, 40-69 are yellow, and below 40 are red.
+
+### TRIBE Baseline Model
+
+TRIBE (Temporal Representation of Integrated Brain Events) v2 is a deployed Modal endpoint that models healthy brain activation patterns across six macro-regions: Attention, Auditory, Emotion, Language, Motor, and Visual. For each daily stimulus, Pegasus calls TRIBE once to establish the user's expected healthy-state baseline, then measures deviation from that baseline as a behavioral burnout signal. TRIBE results are cached per stimulus to avoid repeated GPU calls.
+
+### Crisis and Masking Detection
+
+The scoring engine applies a keyword override layer on top of the emotion model to catch signals the model tends to under-read:
+
+- Crisis terms (suicidal ideation, self-harm language) force the iMessage sub-score to 100 regardless of the emotion model output.
+- Concern terms (hopeless, burned out, can't cope, etc.) floor the sub-score proportionally to the number of terms present.
+- Masking language (clustered over-reassurance such as "I'm fine, totally fine, don't worry") is detected as a tell and pulls the score out of the green even when the emotion model reads positive.
+
+When the overall score reaches red, the backend routes the user to human support rather than an automated intervention.
+
+### RAG Intervention Engine
+
+When the wellness score drops, Pegasus generates a single, warm, specific recommended action using a retrieval-augmented generation pipeline:
+
+1. A mental health coping corpus is embedded using sentence-transformers/all-MiniLM-L6-v2.
+2. The most relevant snippet is retrieved using cosine similarity.
+3. A HuggingFace chat model (Qwen/Qwen2.5-7B-Instruct by default) generates a grounded response using only the retrieved content.
+
+The pipeline degrades gracefully at every step: if HuggingFace is unreachable, the raw retrieved snippet is returned; if the corpus is unavailable, a level-appropriate canned response is used. The intervention call never raises.
+
+### Signal Collection
+
+The signals service collects and analyzes behavioral data from multiple sources:
+
+- **iMessage polling** — an AppleScript bridge reads the local Messages database and extracts response text, response time, and typing metadata.
+- **Keystroke biometrics** — a JavaScript collector (collector.js) captures WPM, error rate, hesitation count, key hold duration, inter-key flight time, burst pattern, and correction loops from in-app typing sessions.
+- **Sentiment analysis** — HuggingFace Inference API replaces any Claude-based sentiment pipeline; the model degrades to a neutral 0.5 score if the API is unreachable.
+- **Linguistic analysis** — structural and lexical features of response text.
+- **Temporal analysis** — time-of-day and response latency patterns.
+
+The signals service maintains a rolling in-memory history of the last 30 readings per user and exposes them to the backend via a typed JSON contract.
+
+### SMS Bot
+
+The SMS bot provides a low-friction check-in channel for users who are not on the mobile app:
+
+- Outbound check-in prompts are sent via Bloo.io SMS.
+- Inbound replies are received via a webhook and routed through the full signal analysis and scoring pipeline.
+- Automated reminders run on a configurable cadence.
+- Stimulus prompts are sent on the same schedule as the in-app experience.
+
+### Video Check-in
+
+The mobile app includes a video check-in flow:
+
+1. The user records a short clip via the in-app camera (60-second maximum, with a countdown ring).
+2. The clip is submitted to the video service, which extracts facial stress and voice stress scores.
+3. Results are fused into the wellness score and displayed alongside the brain visualization immediately after analysis.
+
+### Mobile App
+
+The Expo React Native app is the primary user-facing interface:
+
+- **Home screen** — an animated breathing orb displays the current wellness score in the user's level color (green, yellow, red), along with the top behavioral indicators and the current intervention suggestion. A red reading triggers a haptic alert and a full-screen support prompt.
+- **Check-in screen** — live camera with face-oval overlay, rotating wellness prompts, recording controls, and a post-analysis results view.
+- **Brain screen** — a 3D brain visualization that highlights the TRIBE regions currently flagged as elevated.
+- **Metrics screen** — historical wellness score charts and breakdown by signal stream.
+- **History screen** — a chronological log of past check-ins and score readings.
+- **Chat / Talk screen** — a conversational interface for in-app text check-ins.
+
+The UI uses a liquid glass visual language with level-reactive accent colors throughout.
+
+### Backend API
+
+The FastAPI backend at port 8001 is the single point of integration for the mobile app and SMS bot. It persists all readings to SQLite with WAL mode for concurrency, handles first-time-user race conditions, and exposes the following route groups:
+
+| Route | Purpose |
+|---|---|
+| `/stimulus` | Deliver the daily check-in prompt |
+| `/response` | Accept a text or keystroke-biometric check-in response |
+| `/score` | Return the latest wellness score for a user |
+| `/brain` | Return TRIBE brain region data for the 3D visualization |
+| `/history` | Return the user's check-in history |
+| `/metrics` | Return aggregated score and breakdown data for charts |
+| `/video` | Accept a video submission and return facial/voice analysis |
+
+### Infrastructure
+
+A launchd supervisor script keeps all four services, the iMessage poller, and the ngrok tunnel alive, restarting any process that exits. The sync script handles safe periodic state synchronization without reverting or deleting data.
+
+## Branch Structure
 
 ```
-main (protected — NEVER push directly)
- └── dev (integration branch — PRs only)
-      ├── feat/rishith-ml      → /ml/*
-      ├── feat/wesley-ui       → /frontend/*
-      ├── feat/jason-api       → /backend/*, /shared/*
-      └── feat/dhruva-sig      → /signals/*
+main
+ └── dev
+      ├── feat/jason-api     backend/, shared/
+      ├── feat/wesley-ui     frontend/
+      ├── feat/rishith-ml    ml/
+      └── feat/dhruva-sig    signals/
 ```
 
-## Ownership
+`feat/integration` and `feat/scaffold` are integration and scaffolding branches used during active development.
+
+## License
+
+MIT License
+
+Copyright (c) 2024 Pegasus Contributors
 
-| Branch            | Owner   | Scope                  |
-| ----------------- | ------- | ---------------------- |
-| `feat/rishith-ml` | Rishith | `ml/`                  |
-| `feat/wesley-ui`  | Wesley  | `frontend/`            |
-| `feat/jason-api`  | Jason   | `backend/`, `shared/`  |
-| `feat/dhruva-sig` | Dhruva  | `signals/`             |
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 
-## Rules
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
 
-- `main` is protected. No direct pushes. Releases only via PR from `dev`.
-- `dev` is the integration branch. Feature branches PR into `dev`.
-- Stay in your lane: only touch the directories listed for your branch.
-- Rebase or merge `dev` into your feature branch regularly to stay current.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

backend/.env.example

@@ -0,0 +1,4 @@
+# Backend service (Jason). Copy to .env and adjust.
+SIGNALS_URL=http://localhost:8002
+ML_URL=http://localhost:8003
+PEGASUS_DB=pegasus.db