PoseForge

PoseForge is an interactive workbench for analyzing and correcting human movement in sports using AI-assisted pose estimation and visual analytics. It enables coaches, analysts, and players to upload a single monocular video, reconstruct a temporally consistent 3D skeletal representation, and explore biomechanical metrics that quantify technique over time. Users can inspect motion through synchronized video and 3D views, identify performance deviations using graded metric feedback, and iteratively simulate corrections via direct manipulation or natural language instructions. By combining analysis, feedback, and editable simulation within a single interface, PoseForge supports a shift from subjective observation to quantitative, actionable coaching, while remaining accessible to grassroots and low-resource training environments.

It consists of:

• a React + Three.js frontend for frame review, 3D skeleton/mesh visualization, and metric feedback
• a FastAPI-based processing backend (SAM-3D workflow) for video upload, person tracking, frame extraction, and model/joint serving

The frontend expects a backend API that exposes people, frames, meshes, and joints data in real time after upload.

Repository docs

• Frontend setup and architecture: frontend/README.md
• Backend setup and API details: backend/README.md

Quick start

1) Frontend

npm install

Create a root .env file:

# Required: backend base URL
#1. Running on local server
REACT_APP_API_BASE=http://localhost:7680
#or
#2. Running on Colab/Jupyter Notebook
REACT_APP_NGROK_CHANNEL=https://your-ngrok-url.ngrok-free.app
REACT_APP_GROQ_API_KEY=your_groq_api_key

Run:

npm start

App URL: http://localhost:3000

2) Backend

See backend/README.md for:

• Colab/GPU setup
• local FastAPI serving options
• API contract expected by the frontend

Frontend-to-backend API contract

The frontend calls these endpoints:

• POST /upload - upload a video and trigger processing
• GET /people - list tracked people IDs (e.g., person_000)
• GET /person/{pid} - list model frame files for one person
• GET /person/{pid}/joints - per-frame joints timeline
• GET /models/{pid}/{filename} - serve .glb for a specific frame
• GET /frames/{filename} - serve source frame images