From a229d1dc8e665620736ebd269a5ade16d3e579bc Mon Sep 17 00:00:00 2001 From: "google-labs-jules[bot]" <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Thu, 25 Jun 2026 10:18:50 +0000 Subject: [PATCH] docs: remove outdated Pub/Sub references from technical docs --- docs/ARCHITECTURE_ISSUES.md | 4 ++-- docs/DEPLOYMENT.md | 7 +++---- docs/MODE_SETUP.md | 21 ++++++++------------- docs/SECRETS_SETUP.md | 7 ++----- docs/STALE_SWEEPER.md | 5 ++--- 5 files changed, 17 insertions(+), 27 deletions(-) diff --git a/docs/ARCHITECTURE_ISSUES.md b/docs/ARCHITECTURE_ISSUES.md index 24db48cf..8c12450c 100644 --- a/docs/ARCHITECTURE_ISSUES.md +++ b/docs/ARCHITECTURE_ISSUES.md @@ -36,7 +36,7 @@ Aggregation is fire-and-forget from the sim PATCH response path. If it fails (ti - `api/app/api/jobs/[id]/simulations/[simId]/route.ts` — body fields destructured without type validation - Various PATCH endpoints accept `Record` without sanitization -**Mitigation:** Add zod schemas for request bodies at system boundaries. Internal service-to-service calls (worker → API) still benefit from validation since Pub/Sub redelivery can send malformed payloads. +**Mitigation:** Add zod schemas for request bodies at system boundaries. Internal service-to-service calls (worker → API) still benefit from validation to prevent malformed payloads. --- @@ -148,7 +148,7 @@ Users can submit a job with 4 identical decks. While technically valid (useful f These cross-cutting patterns contributed to past regressions: -1. **No state machine enforcement** (resolved in PR #78) — Invalid state transitions from Pub/Sub redelivery caused COMPLETED simulations to revert to RUNNING. +1. **No state machine enforcement** (resolved in PR #78) — Invalid state transitions caused COMPLETED simulations to revert to RUNNING. 2. **Silent error swallowing** (resolved in PR #78) — `.catch(() => {})` on fire-and-forget operations hid failures that later caused data inconsistency. diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index b2d4c103..8d8dfa4a 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -88,8 +88,8 @@ When prompted, type `gcloud` to use the legacy flow. The script will install `jq ### What the Worker Does -- Loads runtime config from Secret Manager (API_URL, GCS_BUCKET, PUBSUB_SUBSCRIPTION, WORKER_SECRET) -- Subscribes to Pub/Sub for job messages +- Loads runtime config from Secret Manager (API_URL, GCS_BUCKET, WORKER_SECRET) +- Polls API to claim jobs - Runs Forge simulations and POSTs results back to the API - Receives push commands from the API (config overrides, cancellation, job notifications, drain control) via port 9090 @@ -112,7 +112,7 @@ The `setup-worker.sh` script places a service account key at `worker/sa.json` an - **SSH/Tailscale access:** The worker accepts inbound push commands from the API on port 9090 (config updates, cancellation, job notifications). In local mode this is localhost-only; in GCP mode it uses VPC-internal networking. The worker also makes - outbound connections to Pub/Sub, API, and Secret Manager. + outbound connections to API and Secret Manager. ### Auto-Deploy with Watchtower @@ -155,7 +155,6 @@ The worker can be configured via environment variables (in `.env` or Secret Mana | `GOOGLE_CLOUD_PROJECT` | GCP Project ID (triggers GCP mode if set) | `magic-bracket-simulator` | | `API_URL` | Base URL of the API | `http://localhost:3000` or Cloud Run URL | | `WORKER_SECRET` | Shared secret for worker authentication | `some-secret-string` | -| `PUBSUB_SUBSCRIPTION` | Pub/Sub subscription name (GCP mode only) | `job-created-worker` | | `SIMULATION_IMAGE` | Docker image for the simulation container | `ghcr.io/tytaniumdev/magicbracketsimulator/simulation:latest` | | `WORKER_NAME` | Display name for the worker (visible in UI) | Hostname | | `WORKER_ID` | Unique ID for the worker (auto-generated if unset) | `worker-uuid-1234` | diff --git a/docs/MODE_SETUP.md b/docs/MODE_SETUP.md index d8339e87..34d6f2cd 100644 --- a/docs/MODE_SETUP.md +++ b/docs/MODE_SETUP.md @@ -8,21 +8,21 @@ This project supports two operational modes: **LOCAL** and **GCP**. This guide e |---------|------------|----------| | Database | SQLite | Firestore | | File Storage | Local filesystem | Cloud Storage (GCS) | -| Job Queue | Polling-based worker | Pub/Sub | +| Job Queue | Polling-based worker | Polling-based worker | | Analysis | Win rate / game stats | Win rate / game stats | -| Worker | worker/ (polling) | worker/ (Pub/Sub subscriber) | +| Worker | worker/ (polling) | worker/ (polling) | ## Mode Detection The system automatically detects the mode based on the `GOOGLE_CLOUD_PROJECT` environment variable: -- **Set**: GCP mode (Firestore, Pub/Sub, GCS) +- **Set**: GCP mode (Firestore, HTTP Polling, GCS) - **Not set**: LOCAL mode (SQLite, filesystem) At startup, you'll see log messages like: ``` [Job Store] Running in GCP mode [Job Store] Project: magic-bracket-simulator -[Job Store] Using: Firestore + Cloud Storage + Pub/Sub +[Job Store] Using: Firestore + Cloud Storage + HTTP Polling ``` ## Quick Mode Toggle @@ -50,8 +50,7 @@ These commands copy or remove `.env.local` override files in `api/` and `fronten 1. GCP Project with: - Firestore database - Cloud Storage bucket - - Pub/Sub topic and subscription -2. Service account key with permissions for Firestore, GCS, and Pub/Sub +2. Service account key with permissions for Firestore and GCS 3. Docker installed (for worker) ### Configuration Files @@ -86,7 +85,7 @@ WORKER_API_URL="http://:9090" # VPC-internal IP for API→worke npm run dev ``` -**Terminal 2: Local Worker (processes jobs via Pub/Sub)** +**Terminal 2: Local Worker** ```bash npm run worker:gcp ``` @@ -95,7 +94,7 @@ npm run worker:gcp | Service | Purpose | Where it runs | |---------|---------|---------------| -| api | API backend (Firestore/Pub/Sub) | Local or Cloud Run | +| api | API backend (Firestore/Polling) | Local or Cloud Run | | frontend | React UI | Local or Firebase Hosting | | worker | Node.js orchestrator that spawns simulation containers (Docker) | Docker on your machine | | simulation | Java + Forge engine container (ephemeral) | Spawns via `docker run` inside worker | @@ -167,10 +166,6 @@ The worker image contains the Node.js orchestrator. The simulation image (`magic - For local testing, use `http://localhost:3000` - For Cloud Run, use the deployed URL -### Pub/Sub messages not received -- Check `PUBSUB_SUBSCRIPTION` matches the subscription name in GCP -- Verify service account has `pubsub.subscriber` permission - ### Firestore permission errors (e.g. `7 PERMISSION_DENIED: User not authorized to perform this action`) - **Deployed API (Firebase App Hosting / Cloud Run):** The backend runs as a Cloud Run service account. That identity must have **Cloud Datastore User** so it can read/write Firestore (decks, jobs). Grant the role on the project: - GCP Console → **IAM & Admin** → **IAM** → find the service account used by your App Hosting backend (e.g. `PROJECT_NUMBER-compute@developer.gserviceaccount.com` or the one shown in **Firebase** → **App Hosting** → your backend → **Settings**). @@ -184,6 +179,6 @@ The worker image contains the Node.js orchestrator. The simulation image (`magic - **Local API:** Ensure service account key path is correct and the key’s account has Firestore read/write (e.g. Cloud Datastore User). ### Jobs stuck in QUEUED -- Ensure worker is running and connected to Pub/Sub +- Ensure worker is running and polling API - Check worker logs for errors - Verify `WORKER_SECRET` matches between API and worker diff --git a/docs/SECRETS_SETUP.md b/docs/SECRETS_SETUP.md index 0d48f18d..f384bd18 100644 --- a/docs/SECRETS_SETUP.md +++ b/docs/SECRETS_SETUP.md @@ -80,7 +80,6 @@ Use this URL when running `npm run populate-worker-secret`. The frontend always 1. In [GCP Console](https://console.cloud.google.com/iam-admin/serviceaccounts), select your project. 2. Create a service account (e.g. `magic-bracket-worker`) or select an existing one used for the worker. 3. Grant roles (e.g.): - - **Pub/Sub Subscriber** (to pull job messages), - **Storage Object Admin** (or appropriate GCS role for the artifacts bucket), - **Cloud Datastore User** (or **Firestore** roles if using Firestore). 4. Open the service account → **Keys** → **Add key** → **Create new key** → **JSON**. Download the JSON file. @@ -103,8 +102,6 @@ The **Provision Worker** workflow reads secrets from Doppler and syncs them into | `WORKER_SECRET` | Shared secret between worker and API | | `API_URL` | Production API URL (see §0) | | `GCS_BUCKET` | Cloud Storage bucket name (e.g. `magic-bracket-simulator-artifacts`) | - | `PUBSUB_SUBSCRIPTION` | Pub/Sub subscription for jobs (e.g. `job-created-worker`) | - | `PUBSUB_WORKER_REPORT_IN_SUBSCRIPTION` | Pub/Sub subscription for worker report-in (e.g. `worker-report-in-worker`) | | `GHCR_USER` | GitHub username (for Watchtower to pull worker images) | | `GHCR_TOKEN` | GitHub PAT with `read:packages` scope (for Watchtower) | @@ -131,7 +128,7 @@ Use `--defaults` for non-interactive mode: `npm run populate-worker-secret -- -- #### IAM requirements -The identity the worker uses (ADC or service account key) must have **Secret Manager Secret Accessor** on the project. Same key can have Pub/Sub, GCS, Firestore roles as before. For the provision workflow, the SA key also needs **Secret Manager Admin** (to create/update secrets). +The identity the worker uses (ADC or service account key) must have **Secret Manager Secret Accessor** on the project. Same key can have GCS, Firestore roles as before. For the provision workflow, the SA key also needs **Secret Manager Admin** (to create/update secrets). --- @@ -178,7 +175,7 @@ Firebase client config (API key, project ID, etc.) is still set at **build time* Set at least: -- `GOOGLE_CLOUD_PROJECT`, `GCS_BUCKET`, `PUBSUB_TOPIC`, and any secrets already used. +- `GOOGLE_CLOUD_PROJECT`, `GCS_BUCKET`, and any secrets already used. - **WORKER_SECRET** (optional but recommended): a shared secret string. Set the same value in: - Cloud Run (API) env, - worker config (in Secret Manager via `npm run populate-worker-secret`, or in `.env` if not using Secret Manager). diff --git a/docs/STALE_SWEEPER.md b/docs/STALE_SWEEPER.md index 33a4b9ba..85feae4e 100644 --- a/docs/STALE_SWEEPER.md +++ b/docs/STALE_SWEEPER.md @@ -17,7 +17,7 @@ For each `QUEUED` or `RUNNING` job, the endpoint: conditional writes so a worker completing a sim at the last millisecond always wins the race. 3. **Runs the existing recovery path** (`recoverStaleJob`) to re-publish - stale-PENDING Pub/Sub messages and re-trigger aggregation. + stale-PENDING messages and re-trigger aggregation. 4. **Explicitly triggers aggregation** for local-mode jobs where every sim is now terminal (Firestore mode's recovery already does this). @@ -83,5 +83,4 @@ and redeploy. See `docs/superpowers/specs/2026-04-10-stuck-job-prevention-design.md` and the incident writeup for job `uxBSYQvYB4JNoycuLSzz`, which sat stuck at -24/25 sims for ~12 hours because of a worker restart plus Pub/Sub -starvation. +24/25 sims for ~12 hours because of a worker restart plus starvation.