YouTube Media Downloader

Automated YouTube video downloader with channel monitoring, retention management, and a web interface.

Features

• Monitor YouTube channels and automatically download new videos
• Per-channel and per-video retention policies with cutoff dates
• Web UI for configuration and management
• Channels list sorted alphabetically with expandable downloaded-video panels
• Download status indicator for individually tracked videos
• Color-coded scoped logs with channel/video filtering
• REST API for programmatic control
• Cookie support for bypassing rate limits
• Automatic yt-dlp updates
• Docker support with multi-stage builds
• Concurrent downloads with configurable limits

Quick Start

Docker (Recommended)

# Using docker-compose
docker-compose up -d

# Or manually
docker build -t ytdm .
docker run -d -p 8080:8080 \
  -v $(pwd)/data:/app/data \
  -v $(pwd)/downloads:/app/downloads \
  ytdm

Access the web UI at http://localhost:8080

Local

Requirements: Go 1.21+, Python 3, yt-dlp, ffmpeg (optional)

# Quick start script
./run-local.sh

# Or manually
go build -o ytdm
./ytdm

Configuration

All configuration can be managed through the web UI or by editing data/config.json:

{
  "check_interval_seconds": "5m0s",
  "retention_days": 7,
  "disable_pruning": false,
  "download_dir": "../downloads",
  "file_name_pattern": "%(title)s-%(id)s.%(ext)s",
  "max_concurrent_downloads": 3,
  "yt_dlp": {
    "path": "yt-dlp",
    "update_interval_seconds": "24h0m0s",
    "cookies_browser": "firefox",
    "cookies_file": "data/cookies.txt",
    "extractor_sleep_interval_seconds": "0s",
    "download_throughput_limit": "",
    "restrict_filenames": false,
    "cache_dir": "data/yt-dlp-cache"
  }
}

Key Settings

Note: Duration fields in data/config.json use Go duration strings (e.g. "5m0s", "24h0m0s"). The web UI still accepts seconds and converts internally.

• check_interval_seconds: How often to check for new videos (default: 5m0s)
• retention_days: Default retention period in days (default: 7)
• disable_pruning: Disable all automatic pruning globally (default: false)
• max_concurrent_downloads: Number of simultaneous downloads (default: 3)
• yt_dlp: Settings for yt-dlp
  • path: Path to yt-dlp executable
  • update_interval_seconds: How often to auto-update yt-dlp (e.g. 24h0m0s, 0s to disable)
  • cookies_browser: Extract cookies from browser (firefox or chrome)
  • cookies_file: Path to Netscape format cookies file
  • extractor_sleep_interval_seconds: Sleep between extractor requests (e.g. 0s to disable)
  • download_throughput_limit: Limit download speed (e.g. 100K, 4.2M)
  • restrict_filenames: Use yt-dlp's filename restrictions
  • cache_dir: Cache directory used by yt-dlp

Per-Channel Settings

Each channel can override the global retention with its own retention period and cutoff date:

• Retention Days: Keep videos for N days (0 = use global setting)
• Keep indefinitely (disable pruning): Skip automatic pruning for this channel/video entry
• Cutoff Date: Only download videos published on or after this date

Channel monitoring discovery behavior:

• If cutoff_date is set: videos on/after cutoff_date are eligible for first-time download (cutoff-first backlog behavior)
• If cutoff_date is not set: discovery uses publish_date >= now - retention_days

Single-entry requested videos are always attempted when present in the list (publish date is not used as a download gate for single entries). Pruning is based on download age (now - retention_days), with per-entry No Prune still respected. Channel cutoff date is used for channel discovery eligibility and does not trigger pruning by publish date. For channel entries, videos already downloaded and later pruned are tracked in persistence to prevent re-download loops.

Cookie Support

YouTube may require authentication to avoid rate limiting. Two options:

1. Browser Cookies (Automatic)

Select browser in the Configuration tab. Requires the browser to be running and logged into YouTube.

2. Paste Cookies (Manual)

1. Export cookies using a browser extension (Cookie Editor, etc.)
2. Paste Netscape format cookies in the Configuration tab
3. Click "Save Pasted Cookies"

Example format:

# Netscape HTTP Cookie File
.youtube.com	TRUE	/	TRUE	1805237469	COOKIE_NAME	cookie_value

API Endpoints

Channels

• GET /api/channels - List all channels
• POST /api/channels - Add a channel
• DELETE /api/channels/{id} - Remove a channel

Videos

• GET /api/videos - List all videos
• POST /api/videos - Add a video
• DELETE /api/videos/{id} - Remove a video

Configuration

• GET /api/config - Get configuration
• PUT /api/config - Update configuration

Cookies

• POST /api/cookies - Save pasted cookies
• POST /api/cookies/clear - Clear all cookies

Status

• GET /api/status - Service status
• GET /api/logs - Recent logs
  • Optional query params: scope_type=channel|video, scope_id=<id>

Directory Structure

ytdm/
├── data/
│   ├── config.json      # Application configuration
│   ├── data.json        # Channel/video state
│   └── cookies.txt      # YouTube cookies
├── downloads/           # Downloaded videos (organized by channel)
├── static/
│   ├── index.html      # Web UI
│   └── app.js          # UI JavaScript
├── *.go                # Source files
├── *_test.go           # Test files
├── Dockerfile
├── docker-compose.yml
└── run-local.sh        # Local run script

Testing

# Run all tests
go test -v ./...

# Run specific test suite
go test -v -run TestStorage
go test -v -run TestConfig
go test -v -run TestVideoInfo

Development

Built with Go 1.21 using only the standard library (yt-dlp runs as subprocess).

Project structure:

• main.go - Entry point and lifecycle management
• config.go - Configuration with thread-safe operations
• storage.go - Persistent data management
• downloader.go - yt-dlp wrapper
• scheduler.go - Background task scheduling
• api.go - REST API and web server
• updater.go - yt-dlp auto-updater

Auto-Update Feature

yt-dlp automatically updates itself to the latest version:

• Default: Updates every 24 hours
• Configurable via web UI or API
• Set to 0 to disable auto-updates
• Uses yt-dlp's built-in self-update mechanism (yt-dlp -U)

ffmpeg updates require rebuilding the Docker image:

docker build -t ytdm:latest --no-cache .

API Endpoints

Channels

• GET /api/channels - List all channels
• POST /api/channels - Add a new channel

  {
    "name": "Channel Name",
    "url": "https://youtube.com/@channelname",
    "retention_days": 60,
    "cutoff_date": "2024-01-01T00:00:00Z"
  }

  • cutoff_date (optional): Don't download videos published before this date
• DELETE /api/channels/{id} - Remove a channel

Videos

• GET /api/videos - List all videos
• POST /api/videos - Add a new video

  {
    "title": "Video Title",
    "url": "https://youtube.com/watch?v=VIDEO_ID",
    "retention_days": 90
  }

  • added_date is set automatically when omitted.
• DELETE /api/videos/{id} - Remove a video

Configuration

• GET /api/config - Get current configuration
• PUT /api/config - Update configuration

  {
    "check_interval_seconds": 600,
    "retention_days": 60
  }

Status

• GET /api/status - Get service status
• GET /api/logs - Get recent logs (supports scope_type and scope_id filters)

Directory Structure

ytdm/
├── main.go              # Entry point
├── config.go            # Configuration management
├── storage.go           # Persistent data storage
├── downloader.go        # yt-dlp wrapper
├── scheduler.go         # Background task scheduler
├── api.go               # REST API handlers
├── static/
│   └── index.html       # Web interface
├── Dockerfile           # Docker build configuration
├── .dockerignore        # Docker ignore file
└── README.md           # This file

File Organization

Downloaded videos are organized by channel:

/downloads/
├── Channel_Name_1/
│   ├── video1.mp4
│   └── video2.mp4
└── Channel_Name_2/
    └── video3.mp4

Logging

All operations are logged to stdout, including:

• Video downloads
• Removal of old videos
• API requests
• Configuration changes
• Errors and warnings

The web UI logs tab supports:

• Color-coded channel/video scoped log lines
• Structured filtering by channel or individual video entry
• Jump-to-logs actions from the Channels and Videos views
• Channel-scoped lifecycle visibility including feed checks, download attempts, download outcomes, and channel prune events

Metadata lookups prioritize non-yt-dlp sources first (for example, YouTube oEmbed) and only fall back to yt-dlp when required.

Error Handling

The service is designed to handle errors gracefully:

• Failed downloads don't stop other downloads
• API errors return proper HTTP status codes
• Service continues running even if individual operations fail
• Automatic retry on transient failures

Graceful Shutdown

The application handles system signals (SIGINT/SIGTERM) gracefully:

• In-progress downloads: Always complete before shutdown
• Pending work: Skipped to speed up shutdown
• Timeout: 5-minute maximum wait for downloads to finish
• Clean exit: All resources properly released

When you press Ctrl+C or send a termination signal:

1. Service stops accepting new download tasks
2. In-progress downloads are allowed to complete
3. Once all downloads finish, service exits cleanly
4. If downloads take longer than 5 minutes, forces exit

Testing

The project includes comprehensive unit tests covering utility functions, storage operations, configuration management, and skip detection logic.

Running Tests

# Run all tests
go test ./...

# Run with verbose output
go test -v ./...

# Run specific tests
go test -v -run TestSanitizeFilename

# Run with coverage report
go test ./... -cover

# Generate detailed coverage report
go test ./... -coverprofile=coverage.out
go tool cover -func=coverage.out

# View coverage in browser
go tool cover -html=coverage.out

License

MIT License