Skip to content

bauer-group/CS-MinIO

Repository files navigation

MinIO Object Storage

Production-ready MinIO S3-compatible object storage deployment with declarative JSON-based initialization, admin console, and full CI/CD automation.

The three core components (MinIO server, init container, admin console) are built from source using private forks for security patches and customization. An optional minio-worker helper (in-repo, off by default) adds webhook-driven jobs such as CDN cache purging.

Features

  • S3-Compatible API - Full Amazon S3 API compatibility via MinIO
  • Built from Source - All components compiled from private forks (security patches, custom features)
  • Declarative Init Container - Fully automatic infrastructure provisioning from a single JSON file:
    • Buckets with versioning, object-lock/WORM, hard quotas, retention policies, and anonymous access control
    • IAM Policies as fine-grained S3 policy documents (create or update)
    • Users & Groups with automatic group membership and direct policy attachment
    • Service Accounts with dynamic server-generated credentials, written as JSON files to a shared volume for consumption by other containers
    • Notifications as webhook targets with bucket/event bindings (e.g. to trigger CDN cache purging via the optional worker)
    • Environment Variable Resolution - ${VAR_NAME} syntax in JSON values keeps secrets out of config files
    • Fully Idempotent - Runs on every container start; safely skips already-existing resources
    • Two-Layer Config - Built-in defaults (admin policy, group, console user) + user-provided config, both processed independently
    • Pluggable Task System - Add new provisioning tasks by dropping numbered Python files into tasks/
  • Admin Console - Full management UI with complete admin functionality, built from a private fork (karlspace/MinIO-UI) with current security patches. Restores the admin capabilities that MinIO removed from its open-source release.
  • Optional Worker - Webhook-driven helper container (opt-in via Compose profile) that purges CDN edge caches (Cloudflare + Bunny) when objects change, with a durable queue, retry/backoff, and rate limiting
  • Multiple Deployment Modes - Direct port access, Traefik reverse proxy with HTTPS, or development mode with local builds
  • DNS-Style Bucket Access - Prepared virtual-host-style routing (e.g., bucket.s3.example.com)
  • CI/CD Automation - Semantic releases, Docker image builds, base image monitoring, auto-merge

Quick Start

  1. Clone the repository

    git clone https://github.com/bauer-group/CS-MinIO.git
    cd CS-MinIO
  2. Create environment file

    cp .env.example .env
  3. Edit .env - Set at minimum:

    • MINIO_ROOT_PASSWORD - Admin password
    • CONSOLE_PASSWORD - Console user password
  4. Edit config/minio-init.json - Configure buckets, policies, users as needed (copy from config/minio-init.example.json for a full example)

  5. Start MinIO

    # Single mode (direct port access)
    docker compose -f docker-compose-single.yml up -d
    
    # Or: Traefik mode (HTTPS via reverse proxy)
    docker compose -f docker-compose-single-traefik.yml up -d
    
    # Or: Development mode (local builds from source)
    docker compose -f docker-compose-development.yml up -d --build
  6. Access MinIO

    Mode S3 API Console
    Single http://localhost:9000 http://localhost:9001
    Traefik https://{S3_HOSTNAME} https://{S3_CONSOLE_HOSTNAME}
    Development http://localhost:9000 http://localhost:9001

Architecture

┌──────────────────────────────────────────────────────────┐
│                    Docker Compose Stack                   │
│                                                          │
│  ┌──────────────┐    ┌──────────────┐    ┌────────────┐  │
│  │ minio-server │◄───│  minio-init  │    │   admin-   │  │
│  │              │    │  (one-shot)  │    │  console   │  │
│  │  S3 API      │    │              │    │            │  │
│  │  :9000       │    │  Applies     │    │  :9090     │  │
│  │              │    │  JSON config │    │            │  │
│  │  Console     │    │  on start    │    │  Full      │  │
│  │  :9001 (off) │    └──────────────┘    │  admin UI  │  │
│  └──────────────┘                        └────────────┘  │
│         │                                      │         │
│         └──────────────────────────────────────┘         │
│                     Internal Network                     │
└──────────────────────────────────────────────────────────┘

The three boxes above are the always-on core. An optional minio-worker container (Compose profile worker, on the same internal network) can be enabled for webhook-driven jobs such as CDN cache purging — see Optional: CDN Cache Purging.

Deployment Modes

Mode Compose File Description
Single docker-compose-single.yml Direct port binding, pre-built GHCR images
Single + Traefik docker-compose-single-traefik.yml HTTPS via Traefik with Let's Encrypt certificates
Development docker-compose-development.yml Local builds from src/, for development and testing

Note: MinIO removed its built-in admin UI in RELEASE.2025-05-24T17-08-30Z and moved admin features to their commercial AIStor product. This deployment restores full admin functionality through a custom console image built from a private fork with current security patches. See docs/aistor-migration.md for details on MinIO's commercial successor.

Configuration

Environment Variables

All configuration is done via .env. See .env.example for the full reference with documentation.

Required settings:

Variable Description
MINIO_ROOT_PASSWORD MinIO admin password
CONSOLE_PASSWORD Console user password

Generate secrets:

openssl rand -base64 24

Init Container (JSON Configuration)

The init container processes two configuration files in order:

  1. Built-in default (baked into image) - Creates pAdministrators policy, gAdministrators group, and console user with full admin rights from CONSOLE_USER/CONSOLE_PASSWORD environment variables
  2. User config (mounted) - Your custom buckets, policies, users, service accounts, notifications

All operations are idempotent. The init container runs on every start.

Supported resources:

Resource Description
buckets Create buckets with versioning, object-lock/WORM, quotas, retention, lifecycle rules
policies Create or update custom IAM policy documents
groups Create groups and attach policies
users Create users, assign to groups, attach direct policies
service_accounts Create service accounts with dynamic server-generated credentials
notifications Configure webhook targets and bucket/event bindings (e.g. CDN cache purge)

Environment variable resolution: JSON values support ${VAR_NAME} syntax. Variables are resolved from the container's environment at runtime, keeping secrets out of config files.

Dynamic credentials: Service account credentials are generated by MinIO and written as JSON files to the minio-credentials volume (/data/credentials/*.json). Other containers can mount the same volume to read credentials.

Example configuration:

{
  "buckets": [
    {
      "name": "documents",
      "region": "eu-central-1",
      "versioning": true,
      "quota": { "type": "hard", "size": "10GB" },
      "policy": "private"
    }
  ],
  "policies": [
    {
      "name": "pDocumentsReadWrite",
      "statements": [
        {
          "Effect": "Allow",
          "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket"],
          "Resource": ["arn:aws:s3:::documents", "arn:aws:s3:::documents/*"]
        }
      ]
    }
  ],
  "users": [
    {
      "access_key": "${APP_USER}",
      "secret_key": "${APP_PASSWORD}",
      "groups": ["gApplications"]
    }
  ],
  "service_accounts": [
    {
      "user": "${APP_USER}",
      "name": "app-worker",
      "description": "Worker service account"
    }
  ],
  "notifications": [
    {
      "id": "cdnpurge",
      "type": "webhook",
      "endpoint": "http://minio-worker:8080/webhook",
      "auth_token": "${WEBHOOK_AUTH_TOKEN}",
      "buckets": ["public-assets"],
      "events": ["put", "delete"]
    }
  ]
}

See src/minio-init/README.md for the full JSON schema reference.

Optional: CDN Cache Purging (Worker)

The optional minio-worker container automatically purges CDN edge caches when MinIO objects change, so you can run a long CDN edge TTL and still serve fresh content. It receives MinIO bucket-notification webhooks and calls the Cloudflare and/or Bunny purge APIs, with a durable queue, retry/backoff, rate limiting, and dead-lettering.

It is disabled by default and gated behind the Docker Compose worker profile.

Enable it:

  1. In .env, set COMPOSE_PROFILES=worker and provide WEBHOOK_AUTH_TOKEN, S3_PUBLIC_BASE_URL, and Cloudflare (CF_PURGE_API_TOKEN + CF_ZONE_ID) and/or Bunny (BUNNY_API_KEY) credentials. A provider is used automatically when its credentials are present.
  2. Add a notifications block to your init config so MinIO forwards object events to the worker (see src/minio-init/README.md).
  3. Start any deployment mode as usual.

Configuring the CDN side — API tokens, zone ID, and serving MinIO through Cloudflare/Bunny: see Provider setup.

The worker is generic (a small plugin framework); CDN purge is its first job. See src/minio-worker/README.md.

Docker Images

The core images are built from source using private forks; the optional worker is a small in-repo Python service:

Image Source Repository Build
ghcr.io/bauer-group/cs-minio/minio karlspace/MinIO Go binary on Alpine
ghcr.io/bauer-group/cs-minio/minio-init karlspace/MinIO-CLI mc client + Python on Alpine
ghcr.io/bauer-group/cs-minio/minio-console karlspace/MinIO-UI React + Go on Alpine
ghcr.io/bauer-group/cs-minio/minio-worker (in-repo src/minio-worker) Python on Alpine

Base images (monitored for updates):

Base Image Used By
golang:1.26-alpine MinIO server, mc client, console backend
node:22-alpine Console frontend (React/TypeScript)
python:3.14-alpine Init container and worker runtime
alpine:3.23 Server and console runtime stages

Traefik Integration

The Traefik deployment mode provides:

  • Automatic HTTPS via Let's Encrypt certificate resolver
  • S3 API endpoint on ${S3_HOSTNAME} (path-style access)
  • Console endpoint on ${S3_CONSOLE_HOSTNAME} (admin console)
  • No-buffering middleware for large S3 uploads
  • HTTP to HTTPS redirect on all endpoints
  • Traefik v2 and v3 compatible

DNS-Style Bucket Access

Virtual-host-style bucket access (e.g., bucket.s3.example.com) is prepared but commented out in the Traefik compose file. To enable it:

  1. Configure wildcard certificates (requires DNS challenge) or explicit SANs (S3_BUCKET_SANS)
  2. Uncomment the HostRegexp rules in docker-compose-single-traefik.yml
  3. MINIO_DOMAIN is already configured in the template

Project Structure

.
├── .github/                           # GitHub CI/CD configuration
│   ├── CODEOWNERS                     # Pull request review assignments
│   ├── dependabot.yml                 # Automated dependency updates
│   ├── config/
│   │   ├── release/                   # Semantic release configuration
│   │   └── docker-base-image-monitor/ # Base image monitoring
│   └── workflows/
│       ├── docker-release.yml         # Build, release, push images
│       ├── docker-maintenance.yml     # Auto-merge Dependabot PRs
│       ├── check-base-images.yml      # Daily base image update check
│       ├── teams-notifications.yml    # Microsoft Teams notifications
│       └── ai-issue-summary.yml       # AI-powered issue summaries
├── src/
│   ├── minio/                         # MinIO server image (built from source)
│   │   ├── Dockerfile                 # Go build → Alpine runtime
│   │   └── .dockerignore
│   ├── minio-init/                    # Init container image
│   │   ├── Dockerfile                 # mc build + Python runtime
│   │   ├── main.py                    # Orchestrator (task discovery, config loading)
│   │   ├── README.md                  # Init container documentation
│   │   ├── config/
│   │   │   └── default.json           # Built-in default (admin policy/group/user)
│   │   └── tasks/
│   │       ├── 01_buckets.py          # Bucket creation and configuration
│   │       ├── 02_policies.py         # IAM policy create/update
│   │       ├── 03_users.py            # User creation and group assignment
│   │       ├── 04_groups.py           # Group policy attachment
│   │       ├── 05_service_accounts.py # Service accounts (dynamic credentials)
│   │       └── 06_notifications.py    # Webhook targets + bucket/event bindings
│   ├── minio-console/                 # Admin console image (built from source)
│   │   ├── Dockerfile                 # Node + Go build → Alpine runtime
│   │   └── .dockerignore
│   └── minio-worker/                  # Optional webhook worker (CDN purge; in-repo Python)
│       ├── Dockerfile                 # Python on Alpine runtime
│       ├── main.py                    # Entrypoint (config, queue, worker, server)
│       ├── handlers/                  # Event sources (cdn_purge)
│       ├── providers/                 # Purge targets (cloudflare, bunny)
│       └── README.md
├── config/
│   ├── minio-init.json                # Init container configuration (user-facing)
│   └── minio-init.example.json        # Full example with all resource types
├── docs/
│   └── aistor-migration.md            # MinIO AIStor (licensed successor) info
├── docker-compose-single.yml          # Single server, direct port access
├── docker-compose-single-traefik.yml  # Single server, Traefik HTTPS
├── docker-compose-development.yml     # Development mode, local source builds
├── .env.example                       # Environment configuration template
├── CHANGELOG.md                       # Release history (auto-generated)
├── LICENSE                            # MIT License
└── README.md                          # This file

CI/CD

The repository uses semantic-release with Conventional Commits:

Commit Prefix Version Bump Example
fix: Patch (0.1.0 -> 0.1.1) fix: correct health check endpoint
feat: Minor (0.1.0 -> 0.2.0) feat: add bucket lifecycle rules
BREAKING CHANGE: Major (0.1.0 -> 1.0.0) feat!: change init config format

Automated pipeline:

  1. Push to main triggers validation (compose files)
  2. Semantic release creates version tag and GitHub release
  3. All four Docker images (server, init, console, worker) are built and pushed to GHCR and Docker Hub
  4. Dependabot monitors base images weekly; auto-merges updates
  5. Daily base image monitor checks for new releases

Requirements

  • Docker Engine 24.0+
  • Docker Compose v2.20+
  • 512 MB RAM minimum (1 GB+ recommended)
  • For Traefik mode: Traefik v2/v3 reverse proxy on the ${PROXY_NETWORK} network

License

MIT License - BAUER GROUP

Maintainer

Karl Bauer - karl.bauer@bauer-group.com

About

Production-ready MinIO S3-compatible object storage deployment with declarative JSON-based initialization, optional admin console, and full CI/CD automation.

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors