Component Resolver

Part of the AutoSocial platform. This service takes a topic or piece of text and turns it into a fully rendered, narrated video — automatically.

---

What it does

You send in a topic (e.g. "Explain how black holes form"). The service:

1. Writes a script using an LLM (Gemini / OpenRouter)
2. Generates narration — converts the script to SSML speech markup, then sends it to AWS Polly to produce audio files and precise word-level timing
3. Plans the visuals — an AI agent reads the script and decides what to show on screen at each moment: stock photos, AI-generated images, code blocks, diagrams, charts, website screenshots, speaker avatars, captions, background music, and more
4. Resolves each visual — fetches or renders every asset (Pixabay for stock images, Gemini/Imagen for AI art, Puppeteer for screenshots, Mermaid CLI for diagrams, etc.)
5. Hands off to the renderer — sends the fully resolved XML template to an external xml2video service, which composites everything into a final .mp4

The whole pipeline runs asynchronously. A job is submitted, a job ID is returned, and you poll for the result.

---

Tech stack

Layer	What's used
Runtime	Node.js + Express
Database	MongoDB (via Mongoose)
Speech / Audio	AWS Polly, Jamendo (stock music)
Images	Google Gemini, Imagen, Pixabay, Puppeteer (screenshots)
Diagrams	Mermaid CLI, KaTeX (math), custom physics renderer
Storage	Cloudflare R2 (all generated assets)
LLM	Gemini (default), OpenRouter (multi-key rotation fallback)
Video rendering	External xml2video microservice

---

Visual component types

The resolver supports 20+ component types, including:

• stock image — keyword search via Pixabay
• llm image — AI-generated image from a prompt
• mermaid illustration — flowcharts, sequence diagrams, ER diagrams
• code illustration — syntax-highlighted code snippets
• math equations illustration — rendered LaTeX/KaTeX
• website screenshot — live Puppeteer capture of a URL
• stock audio — background music from Jamendo
• stock video clip — stock video footage
• speaker-static / speaker-animated — presenter avatar
• text overlay, shape overlay, captions — UI overlays

---

API

Endpoint	Description
POST /generate-video-full	Submit a video job. Returns 202 with a requestId.
GET /status/:id	Poll job status (pending → processing → complete / error)
GET /details/:id	HTML page showing full job state
GET /logs/:id	HTML log viewer for a job
GET /health	Service health, version, uptime

All endpoints (except /health) require an x-api-key header matching API_KEY in your .env.

---

Getting started

Prerequisites

• Node.js 18+
• MongoDB instance
• AWS account (Polly + S3 or Cloudflare R2)
• At least one LLM API key (Gemini recommended)

Install

git clone <repo>
cd componentResolver
npm install --legacy-peer-deps

Configure

cp .env.sample .env
# Fill in the required keys (see below)

Run

npm run dev    # development (auto-reload)
npm start      # production

---

Environment variables

Variable	Purpose
GEMINI_API_KEY	Gemini LLM + image generation (default provider)
OPENROUTER1_KEY / OPENROUTER2_KEY / OPENROUTER3_KEY	Fallback LLM keys (rotated, retried on 429)
OPENROUTER_URL	OpenRouter API endpoint
PIXABAY_API_KEY	Stock image search
JAMENDO_CLIENT_ID / JAMENDO_CLIENT_SECRET	Stock music
AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_REGION	AWS Polly + S3
CLFR_ACCESS_KEY_ID / CLFR_SECRET_ACCESS_KEY / CLFR_ENDPOINTS_FOR_S3 / CLFR_PUB_BASE_URL / CLFR_BUCKET_NAME	Cloudflare R2 storage
MONGODB_URI	MongoDB connection string
XML2VIDEO_URL / XML2VIDEO_API_KEY	External video renderer
API_KEY	Auth token for this service's endpoints
PORT	Port to listen on (default: 5000)
LLM_PROVIDER	gemini (default) | openrouter | imagen
IMAGE_PROVIDER	gemini (default) | imagen

---

Testing

npm test                  # all tests
npm run test:resolvers    # resolver unit tests only

---

Docker

A Dockerfile is included. Build and run:

docker build -t component-resolver .
docker run --env-file .env -p 5000:5000 component-resolver

---

Project structure

server.js                        # Express app entry point
controllers/
  FullFlowController.js          # Orchestrates the full pipeline
  AgentCrewController.js         # Script → SSML → Polly → component plan
  ResolverController.js          # Resolves each component + audio suppression
services/
  PollyService.js                # AWS Polly narration + timing
  ComponentService.js            # LLM agents for component selection & layout
resolvers/
  ResolverFactory.js             # Maps component type strings → resolver classes
  BaseResolver.js                # Shared helpers (lifecycle, states, caching)
  [Type]Resolver.js              # One file per component type
utils/
  llm.js                         # Provider-agnostic LLM + image generation
  uploader.js                    # Cloudflare R2 uploads
  openrouter.js                  # Multi-key rotation + retry
prompts/
  agentcrew/                     # Markdown prompt templates for each LLM agent