Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Self-hosted backup of your Gmail to your Google Drive. Every email is rendered t

## Status

Live in production on a Hetzner k3s cluster (~€8/mo). The full pipeline — Gmail History polling → MIME fetch → PDF render (Gotenberg) → Drive upload — runs end-to-end, with KEDA scaling each worker pool 1–5 on queue depth and the reliable-queue pop (BLMOVE into per-worker processing lists, acked after the DB settles, reclaimed by the sweeper when a worker dies).
Live in production on a Hetzner k3s cluster (~$6.5/mo). The full pipeline — Gmail History polling → MIME fetch → PDF render (Gotenberg) → Drive upload — runs end-to-end, with KEDA scaling each worker pool 1–5 on queue depth and the reliable-queue pop (BLMOVE into per-worker processing lists, acked after the DB settles, reclaimed by the sweeper when a worker dies).

Captured evidence: [a chaos run](docs/chaos-demo-sample.txt) — worker killed mid-job, recovered in 21 s — and [a flood chart](docs/loadtest/flood-run.png) — 1,000-job burst, replicas 1→5, drained in ~50 s.

Expand Down Expand Up @@ -146,7 +146,7 @@ K8s manifests under `deploy/k8s/`. Validate offline:
make k8s-validate # uses kubeconform — `brew install kubeconform`
```

Production is k3s on a single Hetzner cpx21 (3 vCPU / 4 GB, ~€8/mo), Traefik ingress on :80. Moving providers is a 20-minute runbook ([OPERATIONS.md §3](docs/OPERATIONS.md)) — it ran DigitalOcean → Hetzner for real. KEDA install + autoscaling: §13; loadtest + chart: §14.
Production is k3s on a single Hetzner cx23 (2 vCPU / 4 GB, ~$6.5/mo), Traefik ingress on :80. Moving providers is a 20-minute runbook ([OPERATIONS.md §3](docs/OPERATIONS.md)) — it ran DigitalOcean → Hetzner for real. KEDA install + autoscaling: §13; loadtest + chart: §14.

## Repo layout

Expand Down
4 changes: 2 additions & 2 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Runbooks for operating DistTaskQueue in production. For *what* each component do

## 1. System inventory

What runs where. Current home: a Hetzner cpx21 (3 vCPU / 4 GB) running k3s.
What runs where. Current home: a Hetzner cx23 (2 vCPU / 4 GB, fsn1) running k3s.

| Component | Binary / image | Replicas | Health |
|---|---|---|---|
Expand Down Expand Up @@ -421,7 +421,7 @@ Guidelines based on measured behavior.

| Scenario | Recommended spec |
|---|---|
| Personal inbox, ~100 emails/day | 1 vCPU, 2 GB RAM, 30 GB disk minimum. In production: Hetzner cpx21 (3 vCPU / 4 GB, ~€8/mo) with headroom for KEDA scale-out. Runs cold. |
| Personal inbox, ~100 emails/day | 1 vCPU, 2 GB RAM, 30 GB disk minimum. In production: Hetzner cx23 (2 vCPU / 4 GB, ~$6.5/mo) with headroom for KEDA scale-out. Runs cold. |
| Personal inbox, ~1,000 emails/day | 2 vCPU, 4 GB RAM. Mostly for the Chromium renders during bursts. |
| Multi-user, 10 users × 100 emails/day | 2 vCPU, 4 GB RAM + managed Postgres. PVC for workers must be RWX. |
| Multi-user, 100+ users | Move Postgres + Redis to managed services (RDS + ElastiCache or equivalent). Scale render-worker to 3–5 replicas. Replace `/data` with object storage. |
Expand Down
226 changes: 226 additions & 0 deletions docs/superpowers/plans/2026-07-09-anki-deck-audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
# Anki Deck Audit Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Produce a current, atomic, import-safe Anki deck for DistTaskQueue and export an identical copy to Downloads.

**Architecture:** Treat the current repository as the factual source and the existing deck as an editable inventory. Audit facts before rewriting notes, then validate the completed tab-separated artifact structurally and against repository evidence before exporting it.

**Tech Stack:** Anki text import, tab-separated UTF-8 text, Go source and tests, Redis, Postgres, Kubernetes, shell validation

---

### Task 1: Capture the deck inventory and authoritative project facts

**Files:**
- Read: `All_Decks__System_Design__General.txt`
- Read: `INTERVIEW-BRIEF.md`
- Read: `README.md`
- Read: `docs/ARCHITECTURE.md`
- Read: `docs/OPERATIONS.md`
- Read: `docs/WAR-STORIES.md`
- Read: `internal/queue/queue.go`
- Read: `internal/worker/worker.go`
- Read: `internal/worker/backoff.go`
- Read: `internal/sweeper/sweeper.go`
- Read: `internal/store/store.go`
- Read: `internal/store/migrations/0001_init.up.sql`
- Read: `internal/scheduler/scheduler.go`
- Read: `internal/handler/fetch.go`
- Read: `internal/handler/render.go`
- Read: `internal/handler/upload.go`
- Read: `internal/drive/client.go`
- Read: `deploy/k8s/01-config.yaml`
- Read: `deploy/k8s/29-keda-scaledobjects.yaml`
- Read: `docs/loadtest/flood-run.csv`

- [ ] **Step 1: Record the current import shape and card count**

Run:

```bash
head -n 5 All_Decks__System_Design__General.txt
awk 'NR > 5 {count++} END {print count}' All_Decks__System_Design__General.txt
```

Expected: five Anki headers followed by the current note count.

- [ ] **Step 2: Locate all existing DistTaskQueue prompts and volatile claims**

Run:

```bash
rg -n 'Task Queue|DistTaskQueue|5K|5,000|5\.8s|2,400|heartbeat|TTL|sweep|attempt|KEDA|Hetzner|Gmail|Drive' All_Decks__System_Design__General.txt
```

Expected: every project-specific card and every measurement or deployment fact is visible for verification.

- [ ] **Step 3: Verify runtime constants and state transitions from source**

Run:

```bash
rg -n 'BLMove|LMove|LRem|Heartbeat|MaxAttempts|Compute|Sweep|MarkFailed|ClaimJob|AdvanceJob|MarkDone|time\.(Second|Minute)|Ticker|TTL|next_run_at' internal cmd deploy/k8s
```

Expected: code evidence for queue ordering, processing lists, heartbeats, retries, sweeps, claims, and stage transitions.

- [ ] **Step 4: Verify measured claims from checked-in artifacts**

Run:

```bash
rg -n '5000|5,000|jobs/sec|recovery|seconds|replica|1.*5|€8|Hetzner' README.md INTERVIEW-BRIEF.md docs scripts internal/loadtest
```

Expected: each retained metric has a checked-in source and its measurement context.

### Task 2: Rewrite the deck using minimum-information cards

**Files:**
- Modify: `All_Decks__System_Design__General.txt`

- [ ] **Step 1: Preserve the import contract**

Keep these lines unchanged at the top of the file:

```text
#separator:tab
#html:false
#notetype column:1
#deck column:2
#tags column:5
```

Each subsequent record must contain exactly these five fields:

```text
Basic<TAB>deck path<TAB>prompt<TAB>answer<TAB>space-separated tags
```

- [ ] **Step 2: Correct existing DistTaskQueue cards**

Use `apply_patch` to replace claims that conflict with Task 1 evidence. Keep one recall target per prompt. Split combined cards for timings, retry policy, recovery passes, delivery semantics, queue mechanics, scaling limits, and deployment facts. Add `as-of-2026-07` to volatile-fact tags and include the measurement context in the prompt.

- [ ] **Step 3: Add missing high-value project cards**

Add atomic prompts for:

```text
pipeline stages and why they scale separately
Postgres source-of-truth invariant
BLMOVE processing-list protocol and ACK behavior
conditional UPDATE as the only claim mutex
four independent sweeper recovery paths
fail-stop behavior after post-claim settlement failure
enqueue, claim, folder, and upload idempotency boundaries
Gmail History cursor and backfill safety net
filesystem metadata flow and shared PVC constraint
OAuth token encryption and refresh persistence
KEDA queue-depth scaling and its limits
Redis-loss recovery and simultaneous-store-loss boundary
synthetic demo isolation
real test strategy and load-test scope
current war stories and the design lesson from each
```

Each answer must begin with the shortest sufficient response. Retain a second sentence only when it supplies the causal reasoning expected in an interview.

- [ ] **Step 4: Ankify retained non-project cards**

Use `apply_patch` to split prompts that test multiple independently forgettable facts, shorten wordy answers, add context labels where similar concepts interfere, and remove exact duplicates. Do not alter personal metrics or stories unless repository evidence directly contradicts them.

### Task 3: Validate the Anki artifact

**Files:**
- Test: `All_Decks__System_Design__General.txt`

- [ ] **Step 1: Verify header and field structure**

Run:

```bash
awk -F '\t' 'NR <= 5 {next} NF != 5 {print NR ": " NF " fields"; bad=1} END {exit bad}' All_Decks__System_Design__General.txt
```

Expected: exit 0 with no output.

- [ ] **Step 2: Verify required fields are populated**

Run:

```bash
awk -F '\t' 'NR > 5 && ($1=="" || $2=="" || $3=="" || $4=="" || $5=="") {print NR; bad=1} END {exit bad}' All_Decks__System_Design__General.txt
```

Expected: exit 0 with no output.

- [ ] **Step 3: Verify normalized prompt uniqueness**

Run:

```bash
awk -F '\t' 'NR > 5 {key=tolower($3); gsub(/[[:space:]]+/," ",key); if (seen[key]++) print key}' All_Decks__System_Design__General.txt
```

Expected: no output.

- [ ] **Step 4: Review formulation hot spots**

Run:

```bash
awk -F '\t' 'NR > 5 && (length($3) > 140 || length($4) > 300) {print NR "\t" $3}' All_Decks__System_Design__General.txt
rg -n '\?|;| · .* · | and | vs ' All_Decks__System_Design__General.txt
```

Expected: every long or apparently multi-target note is manually justified or split.

- [ ] **Step 5: Run repository tests that support retained claims**

Run:

```bash
go test ./...
```

Expected: all unit-test packages report `ok` and the command exits 0.

Run when Docker is available:

```bash
make test-integration
```

Expected: the Redis/Postgres integration suite exits 0. If Docker is unavailable, report that exact limitation without claiming the integration suite passed.

### Task 4: Export and verify the final file

**Files:**
- Read: `All_Decks__System_Design__General.txt`
- Create: `~/Downloads/All_Decks__System_Design__General.txt`

- [ ] **Step 1: Copy the validated artifact to Downloads**

Run:

```bash
cp All_Decks__System_Design__General.txt ~/Downloads/All_Decks__System_Design__General.txt
```

Expected: the command exits 0.

- [ ] **Step 2: Verify the source and export are identical**

Run:

```bash
cmp All_Decks__System_Design__General.txt ~/Downloads/All_Decks__System_Design__General.txt
shasum -a 256 All_Decks__System_Design__General.txt ~/Downloads/All_Decks__System_Design__General.txt
```

Expected: `cmp` exits 0 and both SHA-256 values match.

- [ ] **Step 3: Report the completed audit**

Report the final note count, notes added/removed, corrected volatile facts, structural-validation output, test output, export path, and any unverifiable personal claims intentionally preserved.
70 changes: 70 additions & 0 deletions docs/superpowers/specs/2026-07-09-anki-deck-audit-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Anki Deck Audit Design

## Goal

Update `All_Decks__System_Design__General.txt` in place so its DistTaskQueue
material matches the current repository and every retained note follows effective
spaced-repetition formulation. Copy the verified import file to
`~/Downloads/All_Decks__System_Design__General.txt`.

## Sources of truth

Use current implementation and tests as the highest-priority source, followed by
current operational artifacts and project documentation. Use `INTERVIEW-BRIEF.md`
and the existing deck as inventories, not unquestioned authorities. Resolve any
conflict in favor of executable behavior and measured evidence.

Use the supplied SuperMemo twenty-rules article as the formulation standard:
minimum information, one recall target per prompt, context cues, avoidance of
sets and interference, useful redundancy, sources, date stamps for volatile
facts, and prioritization.

## Scope

- Preserve the Anki text-import headers, Basic note type, deck path, tab-separated
fields, and tags field.
- Audit every existing note for atomicity, prompt clarity, answer length,
duplication, interference, and import safety.
- Verify every DistTaskQueue claim against the current repository.
- Add missing high-value DistTaskQueue notes covering architecture, data flow,
invariants, recovery, retry behavior, idempotency, scaling, security,
operations, testing, limitations, and demonstrated incidents.
- Retain useful non-project interview and distributed-systems notes, improving
their formulation without attempting to invent evidence for personal claims
that the repository cannot verify.
- Remove exact duplicates and notes made obsolete by corrected replacements.
- Do not change application code, project behavior, or unrelated files.

## Card formulation

Each prompt will include the shortest context label needed to prevent
interference. Each answer will test one principal fact. Multi-part answers will
be split when their parts can be recalled independently. Explanatory material
will be kept only when it helps reconstruct the reasoning and will follow the
direct answer.

Stable concepts may appear from more than one useful retrieval direction.
Volatile measurements and deployment facts will carry an `as of 2026-07` cue or
tag. Source tags will identify repository evidence without requiring source
paths to be memorized.

## Verification

The final audit will check:

1. Every non-header row has exactly five tab-separated fields.
2. Note type, deck, prompts, answers, and tags are non-empty.
3. Prompts are unique after normalization.
4. No accidental multiline records or malformed quoting remain.
5. DistTaskQueue numbers and behavior match current code, tests, configuration,
documentation, or checked-in measurement artifacts.
6. Relevant Go tests and any existing deck-validation tooling pass.
7. The workspace file and Downloads export are byte-identical.

## Deliverables

- Updated `All_Decks__System_Design__General.txt` in the repository root.
- Identical export at `~/Downloads/All_Decks__System_Design__General.txt`.
- A concise completion report listing card counts, important factual changes,
validation results, and any claims left intentionally unchanged because the
repository cannot verify them.
Loading