SubHarvest

A self-hosted Reddit media harvester.

---

Features

• Track subreddits ("channels") and automatically sync media (images, GIFs, videos)
• Two Reddit modes: public JSON (no API key needed) or OAuth2 via snoowrap
• Cron-based scheduled syncing with BullMQ and Redis
• Masonry gallery with virtual scrolling (handles 10k+ assets at 60 fps)
• Client-side NSFW detection via NSFWJS and TensorFlow.js
• Collections, tags, and saved filters
• Subreddit discovery with NLP query expansion
• CSV/JSON export and bulk import
• Full-text search and advanced filtering
• Dark, light, and system theme support
• Keyboard shortcuts (Cmd+K palette, G-chord navigation)
• Docker Compose deployment

Quick Start

# 1. Clone the repository and copy the environment file
git clone https://github.com/your-org/subharvest.git && cd subharvest
cp .env.example .env

# 2. Edit .env with your secrets (JWT_SECRET at minimum)

# 3. Start all services
docker compose up -d

The web UI will be available at http://localhost:5173 and the API at http://localhost:3000.

Development Setup

Prerequisites

• Node.js 20+
• pnpm
• Redis

Steps

# Clone the repository
git clone https://github.com/your-org/subharvest.git
cd subharvest

# Install dependencies
pnpm install

# Copy and configure environment
cp .env.example .env
# Edit .env -- set JWT_SECRET and REDIS_URL at minimum

# Run database migrations
pnpm db:migrate

# Seed the database (optional)
pnpm db:seed

# Start all services in development mode
pnpm dev

Environment Variables

See .env.example for all available options.

Variable	Description	Default
DATABASE_URL	SQLite connection string	file:./dev.db
REDIS_URL	Redis connection URL	redis://localhost:6379
JWT_SECRET	Secret for signing tokens	(none -- must set)
REDDIT_CLIENT_ID	Reddit OAuth2 client ID (optional)	(none)
REDDIT_CLIENT_SECRET	Reddit OAuth2 client secret (optional)	(none)
REDDIT_MODE	public or oauth	public
CORS_ORIGIN	Allowed CORS origin	http://localhost:5173
SYNC_CRON	Cron expression for sync schedule	*/15 * * * *
PORT	API server port	3000

Architecture

SubHarvest is a pnpm monorepo with the following structure:

apps/
  api/          Fastify REST API (Node 20, TypeScript)
  web/          React 18 SPA (Vite, Tailwind, shadcn/ui)
packages/
  db/           Prisma ORM with SQLite
  types/        Shared Zod schemas
  config/       Shared build configs

The API handles Reddit data fetching, media downloads, and job scheduling. The web app provides the gallery UI, filtering, and management interface. Shared packages ensure type safety and consistent configuration across both apps.

Tech stack: Fastify, React 18, Vite, TanStack Query, Zustand, Prisma, SQLite, BullMQ, Redis, Zod, Tailwind CSS, Radix UI.

Scripts

Script	Description
pnpm dev	Start all apps in development mode
pnpm build	Build all packages and apps
pnpm lint	Lint the entire monorepo
pnpm test	Run all tests
pnpm db:migrate	Run Prisma migrations
pnpm db:seed	Seed the database
pnpm db:studio	Open Prisma Studio

Contributing

See CONTRIBUTING.md for guidelines on how to contribute to SubHarvest.

License

This project is licensed under the MIT License.