diff --git a/docs/caddy.md b/docs/caddy.md index 49ec82e..2aa63b8 100644 --- a/docs/caddy.md +++ b/docs/caddy.md @@ -1,18 +1,37 @@ # Caddy and HAProxy in front of berth -Recommended public deployments use HAProxy on external `:443` as a TCP -SNI router: +This is how I run a public berth leader. The goal is to put the public site +and the cluster endpoint behind the same `:443`, on one VPS, without losing +the agent's mTLS client certificate on the way in. -- `leader.example.com:443` is passed through to Caddy on - `127.0.0.1:8443`; Caddy terminates public TLS and reverse-proxies to +The trick is that HAProxy owns `:443` and routes by SNI at the TCP layer. +It never terminates TLS. Caddy sits behind it on loopback and terminates +public TLS only for the public site. The cluster traffic goes straight +through to berth so the daemon still sees the agent cert. + +`scripts/setup-leader-vps.sh ` writes all of the config below +for you. If you'd rather wire it up by hand, the pieces are here. See +[../README.md](../README.md) for the project itself and +[deploy.md](deploy.md) for the broader deployment story. + +## How traffic flows + +Two names point at the same box, both on `:443`: + +- `leader.example.com:443` — HAProxy passes the TCP stream to Caddy on + `127.0.0.1:8443`. Caddy terminates public TLS and reverse-proxies to berth's plain-HTTP public listener on `127.0.0.1:11500`. -- `cluster.example.com:443` is passed through directly to berth's - cluster listener on `127.0.0.1:11501`; HAProxy does not terminate TLS, - so berth still sees the agent's mTLS client certificate. +- `cluster.example.com:443` — HAProxy passes the stream straight to berth's + cluster listener on `127.0.0.1:11501`. No TLS termination in the middle, + so berth still gets the agent's mTLS client certificate. + +Anything that arrives with neither SNI is dropped at HAProxy. ## Caddyfile -Recommended 443/SNI example: +This is `/etc/caddy/Caddyfile`. Replace `leader.example.com` and +`cluster.example.com` with your own names; `8443`, `11500` and `11501` are +the loopback ports. ```caddyfile { @@ -30,8 +49,8 @@ http://cluster.example.com { https://leader.example.com:8443 { bind 127.0.0.1 header { - # Caddy is behind HAProxy on :443; do not advertise loopback :8443 - # as an external HTTP/3 endpoint. + # Caddy is behind HAProxy on :443; do not advertise loopback + # :8443 as an external HTTP/3 endpoint. -Alt-Svc Strict-Transport-Security "max-age=31536000" } @@ -39,18 +58,63 @@ https://leader.example.com:8443 { header_up X-Forwarded-Proto https } } + +# Catch-all on the same loopback listener: any request whose Host header does +# not match the public site above gets 421 Misdirected Request instead of +# Caddy's empty 200. tls internal uses Caddy's local CA, which is fine here — +# this site only sees connections that HAProxy already routed via the leader +# SNI but with a wrong Host header. +https://:8443 { + bind 127.0.0.1 + tls internal + respond 421 +} ``` +A few things worth knowing: + +- `auto_https disable_redirects` keeps Caddy from standing up its own `:443` + redirector. HAProxy owns `:443`, not Caddy. +- The `http://cluster.example.com` site answers `404`. Plain HTTP to the + cluster name shouldn't reach berth's mTLS listener, so Caddy just turns it + away. +- `-Alt-Svc` strips the header that would otherwise tell clients to come back + on `:8443` over HTTP/3. That port is loopback-only; nothing external should + ever see it. + ## HAProxy +This is `/etc/haproxy/haproxy.cfg`. It's a pure TCP/SNI router — `mode tcp` +everywhere, no `mode http`. + ```haproxy +global + log /dev/log local0 + log /dev/log local1 notice + chroot /var/lib/haproxy + stats socket /run/haproxy/admin.sock mode 660 level admin + stats timeout 30s + user haproxy + group haproxy + daemon + +defaults + log global + mode tcp + option tcplog + timeout connect 5s + timeout client 1h + timeout server 1h + frontend berth_https bind *:443 tcp-request inspect-delay 5s tcp-request content accept if { req.ssl_hello_type 1 } + # Drop connections that don't carry one of our two expected SNI values. + # No default_backend: anything not matched here has already been rejected. + tcp-request content reject if !{ req.ssl_sni -i cluster.example.com leader.example.com } use_backend berth_cluster if { req.ssl_sni -i cluster.example.com } use_backend berth_public if { req.ssl_sni -i leader.example.com } - default_backend berth_public backend berth_public server caddy_public 127.0.0.1:8443 check @@ -59,9 +123,17 @@ backend berth_cluster server berth_cluster 127.0.0.1:11501 check ``` +The `reject if !{...}` line is the important one. There's no `default_backend`: +a connection has to present `cluster.example.com` or `leader.example.com` in +its SNI, or HAProxy closes it. The long `timeout client`/`timeout server` of +`1h` is for the cluster's long-lived mTLS WebSocket — agents hold that +connection open. + ## On the daemon side -Set the corresponding `~/.berth/config.toml`: +berth has to know it's behind a proxy: bind on loopback, speak plain HTTP on +the public listener, and trust the forwarded headers Caddy adds. Put this in +`~/.berth/config.toml`: ```toml [server] @@ -81,9 +153,23 @@ port = 11501 bind = "127.0.0.1" ``` -Set `BERTH_LEADER_URL=https://cluster.example.com` in the berth -systemd unit so enrollment URIs advertise the external 443 endpoint -instead of the loopback cluster port. +What these do: + +- `[server] leader_only` runs the leader as control-plane only — no local + Docker or NVIDIA required, every deploy targets a remote agent. Drop it if + the leader box also has GPUs. +- `[public] scheme = "http"` plus `trust_proxy_headers` makes berth bind plain + HTTP and honour the `X-Forwarded-Proto` Caddy sends. `forwarded_allow_ips` + restricts that trust to the loopback proxy, so a client can't spoof the + header. +- Both listeners bind `127.0.0.1` because HAProxy is the only thing that + should reach them. + +Then set `BERTH_LEADER_URL=https://cluster.example.com` in berth's systemd +unit. That's the URL berth puts in the enrollment URIs it hands out. Without +it, agents would be told to connect to the loopback cluster port instead of +the external `:443` endpoint. The leader installer and `berth deploy bootstrap +--sni-443` both set this for you. ## Verifying the path @@ -98,3 +184,9 @@ curl -i https://leader.example.com/healthz curl -k https://cluster.example.com/admin/ca.pem # → the CA PEM the daemon serves on the cluster listener ``` + +The first request proves the public path: HAProxy → Caddy → berth on +`:11500`. The second proves the cluster path goes straight through to berth on +`:11501` — `-k` because you don't have the CA yet, which is exactly what that +endpoint hands you. If either one hangs, check that the SNI name you're +hitting matches one of the two HAProxy `use_backend` rules. diff --git a/docs/deploy.md b/docs/deploy.md index e2520c1..d38becd 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -1,38 +1,50 @@ # Deploying berth on a public VPS -Operator guide for standing up a leader-only control plane on a small -Linux VPS with public DNS, Caddy for the public UI/API, HAProxy for -443/TLS passthrough routing, and one or more remote GPU hosts joining -as agents. - -The `berth deploy bootstrap` command (see [Bootstrap](#bootstrap) below) -automates most of this. This page documents the underlying recipe so -you can run it manually or audit what the bootstrap does. - -## Requirements - -**VPS**: -- 2 vCPU / 2 GB RAM is plenty (the leader doesn't run inference). -- ~5 GB disk for db / ca / logs. -- Public IPv4 (or IPv6). -- Ports 80 and 443 open inbound. Agents use 443, which works through - most corporate egress firewalls. The berth cluster listener stays on - loopback (`127.0.0.1:11501`) behind HAProxy TLS passthrough. -- Outbound to the agents' GPU hosts is *not* required — agents always - dial the leader. - -**DNS**: -- Two A records (or AAAA) pointing at the VPS: - - `leader.example.com` for the public UI/API. - - `cluster.example.com` for agent enrollment and the mTLS WebSocket. - -**Agents** (the GPU hosts): -- Outbound reachability to `cluster.example.com:443`. No inbound - needed — useful behind corporate VPNs that block client-to-client - traffic and non-standard outbound ports. +This is the operator guide for the leader: a small public VPS that serves +the OpenAI-compatible API and the UI, and that GPU agents dial back into +over mTLS. The leader is control-plane-only. No models run on it. The +weights live on the enrolled agent hosts, and the leader routes to them. + +If you just want it done, run [the one-command setup](#one-command-vps-setup) +and skip the rest. The manual recipe below documents what that script does, +so you can run the steps by hand or audit them. For the agent side and how +the WebSocket tunnel works, see [multi-node.md](multi-node.md). + +## What you need + +The leader does no inference, so it's cheap. + +- 2 vCPU / 2 GB RAM is plenty. +- ~5 GB disk for the DB, CA, and logs. +- A public IPv4 or IPv6. +- Ports 80 and 443 open inbound, nothing else. Agents connect on 443, + which gets through most corporate egress firewalls. The berth cluster + listener stays on loopback (`127.0.0.1:11501`) behind HAProxy doing TLS + passthrough. +- No outbound to the GPU hosts. Agents always dial the leader, never the + other way around. + +You need two DNS records, both pointing at the VPS (A or AAAA): + +- `leader.example.com` — the public UI and API. +- `cluster.example.com` — agent enrollment and the mTLS WebSocket. + +Set these up *before* you run the installer. The setup script checks the +domain is well-formed but it can't make DNS resolve for you, and HAProxy +routes by the SNI hostname. + +On the GPU hosts you only need outbound reachability to +`cluster.example.com:443`. Nothing listens for inbound. That's the whole +point: agents work fine behind corporate VPNs that block client-to-client +traffic and non-standard outbound ports. ## Topology +HAProxy owns external `:443` and routes by SNI. The public hostname goes +to Caddy, which terminates TLS and proxies to the daemon's loopback port. +The cluster hostname is passed straight through so the agent's client +certificate survives — Caddy never sees it. + ``` [ public internet ] ↓ :443 @@ -43,144 +55,259 @@ you can run it manually or audit what the bootstrap does. (TLS passthrough; agent mTLS preserved) ``` +The ports: + +| Port | Bind | What | +| --- | --- | --- | +| 443 | public | HAProxy, SNI router | +| 80 | public | Caddy, redirects to HTTPS | +| 8443 | `127.0.0.1` | Caddy public HTTPS, behind HAProxy | +| 11500 | `127.0.0.1` | berth public API/UI, behind Caddy | +| 11501 | `127.0.0.1` | berth cluster mTLS listener, behind HAProxy passthrough | + +Do not expose 11501 publicly in this setup. Only HAProxy should reach +`127.0.0.1:11501`. + ## One-command VPS setup -Clone berth on a fresh Ubuntu/Debian VPS, make sure DNS already points -at the box, then run: +Clone berth on a fresh Ubuntu or Debian box, confirm DNS already points at +it, then run: ```bash sudo ./scripts/setup-leader-vps.sh example.com ``` -This derives: +It derives the two hostnames from the base domain: + +```text +leader.example.com public UI/API on external 443 +cluster.example.com agent enrollment + mTLS WebSocket on external 443 +``` + +The script runs twelve steps and prints one status line each. Full output +of every step goes to `/var/log/berth-install.log`; the terminal stays +quiet unless something fails. It looks like this: ```text -leader.example.com public UI/API -cluster.example.com agent enrollment + mTLS WebSocket + berth leader installer example.com + ──────────────────────────────────────────────────────────── + [ 1/12] Installing OS packages ························· ok 14s + [ 2/12] Creating berth user and directories ·········· ok + [ 3/12] Copying checkout to /opt/berth/src ··········· ok + [ 4/12] Installing berth (hash-pinned deps) ·········· ok 39s + [ 5/12] Installing operator wrapper ·················· ok + [ 6/12] Bootstrapping config, CA, DB, key ············ ok + [ 7/12] Writing systemd unit ························· ok + [ 8/12] Writing Caddy + HAProxy config ··············· ok + [ 9/12] Validating service configs ··················· ok + [10/12] Firewall + unattended upgrades ··············· ok + [11/12] Kernel + SSH hardening + fail2ban ············ ok + [12/12] Starting services ···························· ok ``` -The script installs Caddy + HAProxy, creates the `berth` system user, -installs berth into `/opt/berth/venv`, writes a leader-only -`/var/lib/berth/config.toml`, bootstraps the DB/CA/key pepper/admin key, -writes systemd/Caddy/HAProxy configs, installs `/usr/local/bin/berth` as the -operator command, opens only `80/tcp` and `443/tcp` in UFW, and starts the -services. +If a step fails, the script stops, prints the last 20 lines of the install +log inline, and points you at the full log. Pass `-v` / `--verbose` to +stream every command live instead of logging it quietly. Pass `--force` to +overwrite an existing `config.toml`. + +At the end it prints a framed summary with the first admin key (shown once), +the two URLs, and the log path: + +```text + ──────────────────────────────────────────────────────────── + ✓ berth leader is up + + Admin key (shown once — save it now) + sk-... -After setup, use short commands: + Public UI/API https://leader.example.com + Agent endpoint https://cluster.example.com + + Verify berth status + Enroll an agent berth nodes enroll