MediaHub — Universal Media Downloader & Transcoding Engine

MediaHub is a high-performance, modular, enterprise-ready universal media downloading and audio/video processing platform. Built as a TypeScript monorepo, MediaHub provides instant URL stream analysis, format selection, progressive video streaming, FFmpeg transcoding, and REST API capabilities across top media platforms.

Overview

MediaHub solves the complexity of extracting, normalizing, and converting online media streams across distinct platforms. When a user or system submits a media URL, MediaHub automatically sanitizes and validates the link, performs SSRF security checks, routes the request through platform-specific provider normalizers (packages/downloader), and presents selectable output formats (native video streams, merged video + audio, or transcode-ready audio formats).

The platform decouples the frontend user experience from media stream extraction and background transcoding workers, ensuring fast responses and reliable stream delivery.

Key Features

Universal Media URL Analysis: Instant stream probe and metadata extraction for videos, audio tracks, shorts, and playlists.
Provider & Normalizer Architecture: Modular provider engine (packages/downloader) supporting specialized normalizers for YouTube, Instagram, X (Twitter), Reddit, and generic yt-dlp fallbacks.
Decoupled Application Shell: Next.js 15 App Router frontend paired with a high-throughput Hono REST API engine (apps/api) and background daemon workers (apps/worker).
FFmpeg Transcoding Pipeline: On-the-fly audio transcoding into MP3, FLAC, WAV, AAC, OPUS, and OGG formats with customizable bitrates (up to 320 kbps) and PCM bit depths (16/24/32-bit).
Progressive Stream Directing: Direct binary HTTP streaming with real-time AbortController cancellation and streaming status indicators.
Security & SSRF Shield: Input sanitization via Zod schemas, stripping tracking parameters (utm_*, igshid, fbclid), and blocking private IPv4/IPv6 ranges (localhost, 127.0.0.1, RFC1918, AWS metadata 169.254.169.254).
Quota & Sliding-Window Rate Limiting: IP-based rate limiting on sensitive /analyze and /download endpoints (packages/quota).
Observability & Health Probes: Dedicated diagnostic endpoints (/health, /health/ffmpeg, /health/extractors, /health/workers, /metrics) exposing Prometheus metrics and binary runtime status.
Lean PostgreSQL & Redis Caching: SHA-256 URL hash caching with TTL and fallback in-memory cache structures (packages/cache).
Developer API & Webhook Management: API key authentication, scope permissions (media.read, media.download, admin), idempotency middleware, and event webhooks (packages/events, packages/outbox).

Supported Platforms

Platform	Support Status	Extraction Engine	Supported Formats
YouTube	Fully Supported	`YouTubeNormalizer` / `yt-dlp`	Video (MP4, WebM, DASH Merged), Audio (MP3, FLAC, WAV, AAC)
YouTube Music	Fully Supported	`YouTubeMusicProvider`	Audio Streams, Transcoded MP3/FLAC/WAV/AAC
Instagram	Fully Supported	`InstagramProvider` / `GenericNormalizer`	Reels, Videos, Audio Extraction
X / Twitter	Fully Supported	`TwitterNormalizer`	Progressive MP4 Video Streams, Audio Extraction
TikTok	Fully Supported	`GenericNormalizer`	Watermark-free Video & Audio Extraction
Reddit	Fully Supported	`RedditProvider`	Video + Audio DASH Merge
Facebook	Fully Supported	`GenericNormalizer`	SD/HD MP4 Streams, Audio Extraction
Vimeo	Fully Supported	`GenericNormalizer`	HLS/MP4 Video Streams, Audio Extraction
Threads	Fully Supported	`GenericNormalizer`	Video & Audio Extraction
Pinterest	Fully Supported	`GenericNormalizer`	Video Clips & GIF Conversions
Generic HTTPS URLs	Fully Supported	`GenericYtDlpProvider`	Direct Video/Audio Stream Extraction

Output Formats & Codecs

Audio Formats

Format	Type	Supported Bitrates / PCM Depths	Container / Codec
MP3	Lossy Transcode	320 kbps, 256 kbps, 192 kbps, 160 kbps, 128 kbps	`libmp3lame` / `.mp3`
FLAC	Lossless Transcode	16-bit PCM, 24-bit PCM	`flac` / `.flac`
WAV	Uncompressed PCM	16-bit PCM, 24-bit PCM, 32-bit Float	`pcm_s16le` / `pcm_s24le` / `.wav`
AAC	High Efficiency	320 kbps, 256 kbps, 192 kbps, 128 kbps	`aac` / `.aac` / `.m4a`
OPUS	Low Latency	192 kbps, 160 kbps, 128 kbps, 96 kbps, 64 kbps	`libopus` / `.opus`
OGG	Open Source	Standard VBR	`libvorbis` / `.ogg`

Video Formats

Format	Quality / Resolution	Audio Stream Status	Container
MP4 (Combined)	1080p, 720p, 480p, 360p	Audio Stream Included	`.mp4` (H.264 / AAC)
MP4 (Video Only)	4K (2160p), 1440p, 1080p	Requires DASH Merge	`.mp4` (H.264 / AV1)
WebM	4K, 1080p, 720p	Native VP9 / AV1 Stream	`.webm` (VP9 / OPUS)

How MediaHub Works

flowchart TD
    User["User / Browser"] -->|"1. Paste Media URL"| Web["Next.js 15 Web App"]

    Web -->|"2. POST /api/v1/analyze"| API["Hono REST API Engine"]

    API -->|"3. Validate & SSRF Shield"| Utils["@mediahub/utils"]

    API -->|"4. Check Cache"| Cache["@mediahub/cache / Redis / PostgreSQL"]

    Cache -->|"Cache Miss"| Provider["@mediahub/downloader Provider Engine"]

    Provider -->|"5. Probe Stream Metadata"| Exec["yt-dlp Executable"]

    Exec -->|"6. Return Formats & Streams"| Provider

    Provider -->|"7. Normalize Formats"| API

    API -->|"8. Present Available Options"| Web

    User -->|"9. Select Format & Click Download"| Web

    Web -->|"10. POST /api/v1/download"| API

    API -->|"11. Transcode if Audio Format"| Audio["@mediahub/audio / FFmpeg Engine"]

    Audio -->|"12. Stream Binary File"| User

URL Submission: User pastes a media link into the 56px input field on the Next.js frontend or calls POST /api/v1/analyze.
Sanitization & SSRF Check: @mediahub/utils strips tracking query parameters, normalizes the hostname, and enforces HTTPS + non-private IP rules.
Cache Lookup: @mediahub/cache checks for cached stream metadata by URL SHA-256 hash.
Platform Normalization: @mediahub/downloader selects the appropriate provider/normalizer (YouTubeNormalizer, TwitterNormalizer, etc.) and invokes yt-dlp to extract available stream formats.
Format Selection: MediaHub categorizes streams into Video, Combined (Video + Audio), and Transcoded Audio options.
Streaming & Transcoding: Upon download request (POST /api/v1/download), @mediahub/audio invokes FFmpeg if transcoding is required (e.g. converting a video audio track into MP3 320 kbps or FLAC) and streams the result directly to the browser.

Architecture & Monorepo Structure

MediaHub/
├── apps/
│   ├── api/                  # Hono REST API Server (Port 4000)
│   ├── web/                  # Next.js 15 App Router Web App (Port 3000)
│   └── worker/               # Background Worker Daemon
├── packages/
│   ├── analytics/            # Event analytics & track collection
│   ├── audio/                # FFmpeg manager, transcoding pipeline, metadata writer
│   ├── billing/              # SaaS billing interfaces
│   ├── cache/                # Redis & PostgreSQL SHA-256 hash caching
│   ├── cli/                  # Command-line developer tools
│   ├── config/               # Zod environment variable validation
│   ├── downloader/           # YtDlpWrapper, ProviderFactory, and platform normalizers
│   ├── events/               # Event bus & publisher contracts
│   ├── flags/                # Feature flag management service
│   ├── metrics/              # Prometheus metrics registry
│   ├── notifications/        # System notification handlers
│   ├── orchestration/        # Workflow execution coordinator
│   ├── organizations/        # Multi-tenant SaaS organization logic
│   ├── outbox/               # Transactional outbox pattern implementation
│   ├── platform/             # Security headers, liveness/readiness probes, shutdown manager
│   ├── queue/                # Global queue manager & worker abstractions
│   ├── quota/                # IP sliding-window rate limiters
│   ├── rbac/                 # Role-based access control rules
│   ├── scheduler/            # Cron & background task scheduler
│   ├── sdk-ts/               # TypeScript SDK client library
│   ├── search/               # Media search indexing
│   ├── storage/              # StorageFactory (Local disk & S3 providers)
│   ├── telemetry/            # OpenTelemetry tracing primitives
│   ├── types/                # Shared TypeScript domain interfaces & DTOs
│   ├── utils/                # Platform detector, URL sanitizer, SSRF shield, formatters
│   └── workflows/            # Multi-step pipeline workflows
├── docker/                   # Standalone Dockerfiles for API, Web, and Worker
├── deployment/               # Deployment scripts & templates
├── nginx/                    # Nginx reverse proxy configuration (`nginx.conf`)
├── k8s/                      # Kubernetes Deployment, Service, and HPA manifests
├── docker-compose.yml        # Development Docker Compose configuration (PostgreSQL)
├── docker-compose.prod.yml   # Production Docker Compose configuration (Postgres, Redis, API, Web, Nginx)
├── package.json              # Monorepo root scripts & dev dependencies
├── pnpm-workspace.yaml       # pnpm workspace configuration
├── tsconfig.base.json        # Base TypeScript compiler settings
└── vitest.config.ts          # Vitest test suite configuration

Tech Stack

Component	Technology	Version
Frontend Framework	Next.js (App Router)	`15.5.22`
UI Library	React	`19.0.0`
Styling	TailwindCSS + Vanilla CSS Tokens	`3.4.17`
API Server	Hono (Node Server)	`4.7.2`
Runtime Language	TypeScript	`5.7.2`
Database ORM	Prisma ORM	`6.3.1`
Database	PostgreSQL	`16-alpine`
Cache & Queue	Redis + BullMQ	`7-alpine`
Stream Extractor	`yt-dlp`	Latest Binary / Python
Audio Transcoder	`FFmpeg`	6.x / 7.x Essentials Build
Validation	Zod	`3.24.2`
Logging	Pino Structured Logger	`9.6.0`
Test Runner	Vitest	`3.0.5`
Package Manager	pnpm	`>= 9.0.0`
Containerization	Docker & Docker Compose	Compose Specification v3.8

Requirements & Prerequisites

Before running MediaHub locally, ensure you have the following installed:

Node.js: >= 20.0.0 (Node.js Download)
pnpm: >= 9.0.0 (npm install -g pnpm)
yt-dlp: Must be accessible via PATH (yt-dlp) or Python (python -m yt_dlp)
FFmpeg: Must be installed and accessible on your system PATH (ffmpeg -version) or specified via FFMPEG_PATH.
PostgreSQL / Docker (Optional): Required for persistence. If PostgreSQL is absent, MediaHub runs in-memory.

Local Development & Setup

1. Clone the Repository

git clone https://github.com/me-hv/MediaHub.git
cd MediaHub

2. Install Workspace Dependencies

pnpm install

3. Build Monorepo Shared Packages

pnpm build

4. Start Infrastructure Containers (Optional PostgreSQL & Redis)

docker-compose up -d

5. Run Database Migrations (Optional)

pnpm db:generate

6. Start Concurrent Development Servers

pnpm dev

This launches:

Next.js Web Frontend: http://localhost:3000
Hono REST API Engine: http://localhost:4000
Background Worker Daemon: Concurrently active listening for queue jobs

Environment Variables

Copy .env.example or create an .env file in the root / app directories as needed:

Variable	Default Value	Description
`NODE_ENV`	`development`	Runtime mode (`development` / `production` / `test`)
`PORT`	`4000`	Port for Hono REST API server
`DATABASE_URL`	`postgresql://postgres:postgrespassword@localhost:5432/mediahub`	PostgreSQL database connection string
`REDIS_URL`	`redis://localhost:6379`	Redis connection URL for queues & caching
`NEXT_PUBLIC_API_URL`	`http://localhost:4000`	API endpoint used by Next.js frontend
`YTDLP_PATH`	`yt-dlp`	Path to custom `yt-dlp` executable
`FFMPEG_PATH`	`ffmpeg`	Path to custom `ffmpeg` binary

REST API Reference

1. Health & Diagnostics

`GET /health`

Returns system status, active queue stats, and enabled feature flags.

{
  "success": true,
  "status": "healthy",
  "version": "1.0.0",
  "queue": {
    "pendingCount": 0,
    "activeWorkers": 4
  },
  "timestamp": "2026-08-10T13:50:00.000Z",
  "requestId": "req-1786349000-abc12"
}

`GET /health/extractors`

Validates runtime availability of yt-dlp and FFmpeg binaries.

{
  "success": true,
  "status": "healthy",
  "extractors": {
    "ytDlp": { "available": true, "version": "2026.07.04" },
    "ffmpeg": { "available": true, "version": "7.0.1" }
  }
}

2. Media Operations

`POST /api/v1/analyze`

Analyzes a media URL and returns available video, combined, and audio stream formats.

Request Payload:

{
  "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
}

Response (200 OK):

{
  "success": true,
  "data": {
    "metadata": {
      "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
      "urlHash": "281691a56a6448408cf8eb47d174a72d73f4e24efd8c19958ee89ebecb631b12",
      "title": "Rick Astley - Never Gonna Give You Up",
      "uploader": "Rick Astley",
      "duration": 212,
      "platform": "youtube",
      "qualities": {
        "combined": [
          { "formatId": "18", "ext": "mp4", "resolution": "640x360", "hasVideo": true, "hasAudio": true }
        ],
        "video": [
          { "formatId": "137", "ext": "mp4", "resolution": "1920x1080", "hasVideo": true, "hasAudio": false }
        ],
        "audio": [
          { "formatId": "mp3-320", "ext": "mp3", "qualityLabel": "MP3 320 kbps", "requiresConversion": true }
        ]
      }
    }
  },
  "requestId": "req-1786349000-xyz89"
}

`POST /api/v1/download`

Streams binary media content or triggers an FFmpeg transcode download.

Request Payload:

{
  "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
  "formatId": "mp3-320"
}

Response: Binary file stream with HTTP header: Content-Disposition: attachment; filename="Rick_Astley_-_Never_Gonna_Give_You_Up.mp3"

Security & SSRF Protection

MediaHub implements zero-trust security checks for incoming media URLs in @mediahub/utils:

HTTPS Enforcement: Non-HTTPS protocols are automatically rejected.
Private Network Blocking: Prevents Server-Side Request Forgery (SSRF) by blocking:
- Loopback: localhost, 127.0.0.1, ::1
- RFC1918 Private Subnets: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
- Cloud Metadata APIs: 169.254.169.254
- Internal domains: *.local, *.internal
URL Sanitization: Strips tracking query parameters (utm_source, igshid, fbclid, si, ref) before execution.
Security Headers: Injects Strict-Transport-Security, X-Content-Type-Options, X-Frame-Options, and Content-Security-Policy headers into all HTTP responses.

Testing & Quality Assurance

MediaHub includes a comprehensive Vitest test suite covering normalizers, providers, executable resolvers, API controllers, worker daemons, and audio pipelines.

Run All Unit & Integration Tests

pnpm test

Run Monorepo TypeScript Verification

pnpm type-check

Production Deployment

1. Docker Compose Production Cluster

Run the complete production stack (PostgreSQL, Redis, API, Web, and Nginx reverse proxy):

docker-compose -f docker-compose.prod.yml up -d --build

2. Kubernetes Deployment

Kubernetes manifests are provided in k8s/:

kubectl apply -f k8s/service.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/hpa.yaml

Future Roadmap

The following architectural enhancements are planned for upcoming phases:

Cloud Storage S3 / R2 Handlers: Expand packages/storage for persistent cloud object caching.
Distributed Webhook Delivery: Automatic webhook retries with exponential backoff via BullMQ dead-letter queues.
OAuth2 & Multi-Tenant Organization Teams: Production auth provider integrations for team workspaces.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
apps		apps
components/media		components/media
deploy		deploy
deployment		deployment
docker		docker
docs		docs
k8s		k8s
nginx		nginx
packages		packages
tests/e2e		tests/e2e
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
README.md		README.md
docker-compose.prod.yml		docker-compose.prod.yml
docker-compose.yml		docker-compose.yml
package.json		package.json
playwright.config.ts		playwright.config.ts
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-lock.yaml.3944623656		pnpm-lock.yaml.3944623656
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.base.json		tsconfig.base.json

Latest commit

History

Folders and files

Repository files navigation