Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,15 @@ build:ci-remote-base --remote_timeout=60

build:ci-cached --config=ci-remote-base
build:ci-cached --remote_cache_compression
# Shared-cache actions execute inside a bounded ARC runner container.
# Bound both action parallelism and per-process Node heap so one local action
# plus Bazel overhead remains inside that hard memory limit. Executor-backed
# proofs retain their separately managed capacity.
build:ci-cached --jobs=1
build:ci-cached --action_env=NODE_OPTIONS=--max-old-space-size=1024
# Browser-backed test actions execute inside the same ARC pod in this mode.
# Keep them serial so concurrent Chromium processes cannot OOM the runner.
test:ci-cached --local_test_jobs=1

build:executor-backed --config=ci-remote-base
build:executor-backed --action_env=HOME=/var/lib/gf-reapi-cell/home
Expand Down
117 changes: 71 additions & 46 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,28 @@ jobs:
bazel-remote-gates:
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
# Per-ref dedupe only: a newer push to the same ref supersedes its own
# stale gates run. Cross-PR quota pressure on the gf-reapi-cell tenant
# (maxConcurrentExecutions=4; runs 28732755094, 28847693875) is handled
# by the exit-34 retry loop in scripts/bazel-cache-backed.sh — a shared
# repo-wide group was tried first, but GitHub evicts (cancels) middle
# pending jobs in a group, which breaks required checks (run
# 28849318814 + main 09c1dd0 were cancelled, not queued).
# stale gates run without canceling unrelated pull requests.
concurrency:
group: gf-reapi-tenant-gates-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
id-token: write
runs-on: tinyland-dind
timeout-minutes: 60
# Shared-cache actions execute locally inside a fleet-managed ARC pod.
# Keep the repo process envelope bounded independently of fleet sizing.
timeout-minutes: 90
env:
GF_BAZEL_SUBSTRATE_MODE: shared-cache-backed
GF_BAZEL_REMOTE_UPLOAD: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'true' || 'false' }}
GF_BAZEL_HOST_JVM_MAX_HEAP_MB: '2560'
GF_REAPI_TOKEN_EXCHANGE_REQUEST: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'cache-write' || 'cache-read' }}
GF_REAPI_TOKEN_EXCHANGE_TTL: 45m
TECTONIC_CACHE_DIR: /tmp/tectonic-cache-v33
# The GF consumer registry records this repo as shared-cache-backed.
# Clear any executor endpoint inherited from the generic ARC pod.
BAZEL_REMOTE_EXECUTOR: ''
BAZEL_REMOTE_EXECUTOR_CACHE: ''
steps:
- uses: actions/checkout@v6
with:
Expand All @@ -55,62 +63,53 @@ jobs:
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: npm

- name: Cache Playwright browsers
uses: actions/cache@v5
with:
path: ~/.cache/ms-playwright
key: ms-playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

- name: Provision Chromium for browser gates
- name: Cache Tectonic resources
uses: actions/cache@v5
with:
path: /tmp/tectonic-cache-v33
key: tectonic-v33-${{ runner.os }}-${{ hashFiles('MODULE.bazel') }}
restore-keys: tectonic-v33-${{ runner.os }}-

- name: Provision package-lock-pinned Chromium
run: |
set -euo pipefail
# Executor images ship /bin/chromium; the dind runner image ships no
# browser, so the puppeteer smoke and playwright e2e need one
# provisioned here. Playwright's installer also pulls the shared
# system libraries the binary links against.
for c in /bin/chromium /usr/bin/chromium /usr/bin/chromium-browser /usr/bin/google-chrome; do
if [ -x "$c" ]; then
echo "System chromium present at $c"
exit 0
fi
done
PLAYWRIGHT_VERSION="$(node -p "require('./package.json').devDependencies['@playwright/test'].replace(/^[^0-9]*/, '')")"
npx --yes "playwright@${PLAYWRIGHT_VERSION}" install --with-deps chromium
BIN="$(ls -d "${HOME}/.cache/ms-playwright"/chromium-*/chrome-linux*/chrome 2>/dev/null | head -1)"
test -x "${BIN}"
echo "GF_RBE_CHROMIUM_EXECUTABLE=${BIN}" >> "$GITHUB_ENV"

- name: Resolve GloriousFlywheel substrate lane
# Install the revision pinned by @playwright/test outside Bazel,
# before token minting and SSH setup, then pass its absolute path
# into browser tests.
npm ci --ignore-scripts
npx --no-install playwright install --with-deps chromium
chromium_path="$(node --input-type=module -e "import { chromium } from '@playwright/test'; process.stdout.write(chromium.executablePath())")"
if [[ "${chromium_path}" != /* || ! -x "${chromium_path}" ]]; then
echo "::error::Playwright Chromium resolution produced an unusable path"
exit 1
fi
echo "GF_RBE_CHROMIUM_EXECUTABLE=${chromium_path}" >> "${GITHUB_ENV}"

- name: Require shared-cache-backed GloriousFlywheel lane
run: |
set -euo pipefail
if [ -z "${BAZEL_REMOTE_CACHE:-}" ]; then
echo "::error::BAZEL_REMOTE_CACHE is missing. This job must run on the GloriousFlywheel ARC lane with cache hints attached."
exit 1
fi

# This repo's GF consumer-registry entry is substrate_mode=
# shared-cache-backed (latch: remote_processes=0), so executor
# tokens are not ours to mint — the exchange grants cache-write
# only to trusted default-branch pushes and cache-read to any
# enrolled ref. Strip the executor attachment so the wrapper and
# cache-attachment-contract agree on shared-cache mode.
echo "GF_BAZEL_SUBSTRATE_MODE=shared-cache-backed" >> "$GITHUB_ENV"
echo "BAZEL_REMOTE_EXECUTOR=" >> "$GITHUB_ENV"
echo "BAZEL_REMOTE_EXECUTOR_CACHE=" >> "$GITHUB_ENV"
if [ "${GITHUB_EVENT_NAME}" = "push" ] && [ "${GITHUB_REF}" = "refs/heads/main" ]; then
echo "GF_REAPI_TOKEN_EXCHANGE_REQUEST=cache-write" >> "$GITHUB_ENV"
else
echo "GF_REAPI_TOKEN_EXCHANGE_REQUEST=cache-read" >> "$GITHUB_ENV"
echo "GF_BAZEL_CACHE_ROLE=read-only" >> "$GITHUB_ENV"
if [ -n "${BAZEL_REMOTE_EXECUTOR:-}" ]; then
echo "::error::BAZEL_REMOTE_EXECUTOR must be empty for this shared-cache-backed consumer."
exit 1
fi

- name: Mint GloriousFlywheel REAPI token
env:
GF_REAPI_TOKEN_EXCHANGE_TTL: 45m
run: bash scripts/mint-gf-reapi-token-from-exchange.sh

- name: Validate GloriousFlywheel RBE attachment
- name: Validate GloriousFlywheel cache attachment
run: bash scripts/cache-attachment-contract.sh --strict

- name: Set up SSH for spear-resumes
Expand Down Expand Up @@ -153,14 +152,40 @@ jobs:
sudo mv "${tmp}" /usr/local/bin/bazel
fi

- name: Refresh cache token for Tectonic warm
run: bash scripts/mint-gf-reapi-token-from-exchange.sh

- name: Warm Tectonic resources serially
run: |
bash scripts/bazel-cache-backed.sh build --jobs=1 \
@spear_resumes//generic:resume \
@spear_resumes//generic:precis \
@spear_resumes//generic:cv
bash scripts/bazel-cache-backed.sh shutdown

- name: Refresh cache token for remote check
run: bash scripts/mint-gf-reapi-token-from-exchange.sh

- name: Remote check
run: npm run remote:check
run: |
npm run remote:check
bash scripts/bazel-cache-backed.sh shutdown

- name: Refresh cache token for remote test
run: bash scripts/mint-gf-reapi-token-from-exchange.sh

- name: Remote test
run: npm run remote:test
run: |
npm run remote:test
bash scripts/bazel-cache-backed.sh shutdown

- name: Refresh cache token for remote e2e
run: bash scripts/mint-gf-reapi-token-from-exchange.sh

- name: Remote e2e
run: npm run remote:e2e
run: |
npm run remote:e2e
bash scripts/bazel-cache-backed.sh shutdown

build-and-test:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -210,7 +235,7 @@ jobs:
- name: Test authority
run: |
echo "check/test/e2e are intentionally owned by bazel-remote-gates."
echo "The GloriousFlywheel RBE lane runs remote:check, remote:test, and remote:e2e."
echo "The GloriousFlywheel shared-cache lane runs remote:check, remote:test, and remote:e2e."

- name: Blog agent production dependency audit (report-only)
run: |
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ These rules apply before the writing-style guidance below.

## Bazel And GloriousFlywheel

- GloriousFlywheel-backed validation runs through `scripts/bazel-cache-backed.sh`. The remote scripts must fail closed when `BAZEL_REMOTE_CACHE`, `BAZEL_REMOTE_EXECUTOR`, or the expected executor-backed substrate is missing.
- GloriousFlywheel-backed validation runs through `scripts/bazel-cache-backed.sh`. This repository is registered as `shared-cache-backed`; the remote scripts must fail closed when `BAZEL_REMOTE_CACHE` or that expected substrate mode is missing. CI must clear generic-runner executor hints. Executor-backed mode remains an explicit opt-in only after a reviewed GloriousFlywheel consumer-registry promotion.
- `npm run remote:check`, `npm run remote:test`, and `npm run remote:e2e` are the authoritative Bazel entrypoints. They cover SvelteKit check/build smoke, package checks, CV PDF sync, Vitest, blog-agent tests, browser smoke, Playwright e2e, and graph hygiene.
- `package-lock.json` is the npm authority. `pnpm-lock.yaml` is the generated `rules_js` lock consumed by Bazel; do not casually hand-edit it.
- Browser-backed Bazel tests use the pinned worker Chromium path. Do not add lifecycle downloads or host-local browser assumptions to the RBE path.
- Shared-cache CI provisions the package-lock-pinned Playwright Chromium outside Bazel actions and passes its absolute path explicitly. Browser-backed RBE uses the pinned worker Chromium path; do not add lifecycle downloads or host-local browser assumptions inside the RBE action path.
- Broad `bazel query //...` and `bazel test //...` must not traverse agent scratch trees. `.bazelignore` intentionally excludes `.claude`, `.gstack`, `.worktrees`, build outputs, caches, and `workers/dns-guard/.wrangler`.
- If you touch Bazel structure, scratch-tree ignores, or broad query behavior, run `npm run test:bazel-graph-hygiene` and, when possible, `bazelisk --output_user_root=/tmp/jess-ghio-bazel-codex test //:bazel_graph_hygiene`.

Expand Down
13 changes: 13 additions & 0 deletions BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,18 @@ sh_test(
visibility = ["//visibility:public"],
)

sh_test(
name = "bazel_cache_backed_contract",
srcs = ["scripts/test-bazel-cache-backed.sh"],
data = [
".bazelrc",
".github/workflows/ci.yml",
"scripts/bazel-cache-backed.sh",
"scripts/cache-attachment-contract.sh",
],
visibility = ["//visibility:public"],
)

exports_files(
[
"package-lock.json",
Expand All @@ -312,6 +324,7 @@ exports_files(
"scripts/run-sveltekit-check-bazel.mjs",
"scripts/run-sveltekit-vite-build-bazel.mjs",
"scripts/serve-static-build.mjs",
"scripts/test-bazel-cache-backed.sh",
"scripts/run-vitest-bazel.mjs",
"vitest.bazel.config.ts",
],
Expand Down
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,29 +100,31 @@ flowchart LR
PrivateMirror --> Tailnet["jesssullivan-blog-shadow.taila4c78d.ts.net"]
```

## GloriousFlywheel Bazel/RBE Pilot Surface
## GloriousFlywheel Bazel Substrate Surface

This repo still uses the npm/SvelteKit workflow for normal local development and deployment. CI also carries a blocking GloriousFlywheel Bazel lane on the `tinyland-dind` ARC runner so check, test, and Chromium e2e coverage are not proven only by local `npm`/`npx` commands.

- `npm run remote:check`, `npm run remote:test`, and `npm run remote:e2e` route through `scripts/bazel-cache-backed.sh`, which refuses to run without a valid `BAZEL_REMOTE_CACHE` and the expected GloriousFlywheel substrate mode.
- Local developer shells attach through the endpoint-free GloriousFlywheel front-door kit (`justfile.flywheel` plus `.bazelrc.flywheel`). The managed Nix/Home Manager profile is the preferred source of `BAZEL_REMOTE_CACHE` and auth metadata; a gitignored `.env.flywheel.local` generated by `just flywheel-enroll ...` is only the fallback fixture, and is sourced by both `.envrc` and `scripts/bazel-cache-backed.sh`.
- GitHub CI runs the Bazel lane on `tinyland-dind` and refuses to run the RBE gate unless `BAZEL_REMOTE_EXECUTOR` and `BAZEL_REMOTE_CACHE` are attached by the ARC runner environment. Before the strict gate, CI exchanges GitHub Actions OIDC with the GF token exchange and writes the short-lived cell JWT to `GF_REAPI_CREDENTIAL_HELPER_TOKEN_FILE`.
- In executor-backed mode, the wrapper uses the executor endpoint as Bazel's effective REAPI CAS/cache by default so remote Execute can read uploaded action/input digests. This profile disables remote cache compression for the current GF proof cell and gives remote actions writable GF worker cache paths. `BAZEL_REMOTE_EXECUTOR_CACHE` can override the cache endpoint only when a separate executor-readable CAS is wired.
- GitHub CI runs the Bazel lane on `tinyland-dind` as the `shared-cache-backed` consumer recorded in the GloriousFlywheel registry. Pull requests mint cache-read tokens and disable result uploads; trusted `main` pushes mint cache-write tokens. Generic-runner executor hints are cleared so the workflow cannot silently claim executor-backed behavior.
- Shared-cache CI runs one local Bazel action at a time, caps the Bazel host JVM at 2560 MiB and each Node action at 1024 MiB, and serializes local test actions. It shuts the Bazel server down between the Tectonic warm, check, test, and e2e phases so retained analysis state cannot grow across the entire job. This keeps the repo process envelope below fleet-managed ARC capacity while leaving enough JVM heap to analyze the full workspace graph, without changing the independently managed capacity of explicit executor-backed proofs. The hosted gate allows 90 minutes because this reliability-first envelope deliberately trades wall-clock speed for runner survival.
- The lane restores one v33 Tectonic resource cache, warms the three public CV targets serially, and explicitly makes that cache writable inside local Bazel sandboxes. This avoids concurrent bundle fetches; the runner-local path is rejected in executor-backed mode.
- Executor-backed mode remains available in the wrapper only as a separate opt-in contract. It requires a reviewed GloriousFlywheel registry promotion plus an explicit executor endpoint; this repository's normal CI does not use it.
- `gf-reapi-cell` endpoints also require scoped Bazel credential-helper auth. `scripts/bazel-cache-backed.sh` attaches `scripts/gf-reapi-bazel-credential-helper.mjs` only for the GF REAPI host, and the helper reads a short-lived JWT from `GF_REAPI_CREDENTIAL_HELPER_TOKEN_FILE`, `GF_REAPI_CREDENTIAL_HELPER_TOKEN`, or the projected-token file at `/var/run/secrets/tokens/gf-reapi-cell-token`.
- The CI lane sets `BAZEL_REMOTE_INSTANCE_NAME=default` for the current GF tenant. Literal shell placeholders are rejected before Bazel starts.
- The token exchange supplies the repository-scoped `BAZEL_REMOTE_INSTANCE_NAME`. Literal shell placeholders are rejected before Bazel starts.
- `//:sveltekit_check` runs the SvelteKit check path under Bazel.
- `//static/cv:pdfs_synced_test` byte-compares the checked-in resume/CV PDFs against the Bazel-built `spear_resumes` outputs; `.bazelrc` pins `SOURCE_DATE_EPOCH` and `TZ` so Tectonic output stays reproducible across local sync, CI, and GF RBE.
- `//static/cv:pdfs_synced_test` byte-compares the checked-in resume/CV PDFs against the Bazel-built `spear_resumes` outputs; `.bazelrc` pins `SOURCE_DATE_EPOCH` and `TZ` so Tectonic output stays reproducible across local sync, shared-cache CI, and explicit executor proofs.
- `//:vitest_unit_tests` wraps the root and Pulse Vitest suites through `vitest.bazel.config.ts`.
- `//:blog_agent_node_tests` wraps the blog-agent `node:test` suite through `tsx --test`.
- `//:sveltekit_vite_build_smoke` runs a copied-workdir SvelteKit/Vite production build smoke. It proves the build target class, not the full npm prebuild/postbuild publication chain.
- `//:playwright_chromium_e2e` runs the Chromium Playwright e2e suite through Bazel against the pinned GloriousFlywheel Chromium runtime path.
- `//:playwright_chromium_e2e` runs the Chromium Playwright e2e suite through Bazel. Shared-cache CI provisions the package-lock-pinned Playwright browser before Bazel starts and passes its absolute executable path into the local test action.
- `//:playwright_chromium_smoke` remains a narrow diagnostic target for browser runtime authority, not the remote e2e gate.
- `//:puppeteer_chromium_smoke` launches Puppeteer against the same pinned Chromium runtime path. It proves Puppeteer can consume browser runtime authority without lifecycle downloads.
- `//:puppeteer_chromium_smoke` launches Puppeteer against the same pinned Chromium runtime path. It proves Puppeteer and Playwright consume one explicit browser authority rather than relying on an undeclared host path.
- `package-lock.json` remains the npm dependency authority for the app. `pnpm-workspace.yaml` makes the package importers explicit for Bazel, and `pnpm-lock.yaml` is the generated `rules_js` lock consumed by Bazel.
- Bazel npm lifecycle hooks skip Playwright and Puppeteer browser downloads. Browser-backed RBE must use the pinned worker Chromium path rather than downloading browsers during proof actions.
- Bazel npm lifecycle hooks skip Playwright and Puppeteer browser downloads. Shared-cache CI provisions the lockfile-pinned Playwright browser outside Bazel actions; browser-backed RBE continues to use the pinned worker Chromium path and never downloads a browser inside proof actions.
- GloriousFlywheel proof runs use the external GF REAPI proof harness or the repository `remote:*` scripts against this public repo checkout; remote cache hits, hosted runners, and shared-cache-only execution do not count as RBE.

Current boundary: this gates Bazel check/test/Chromium-e2e and CV PDF sync drift under GloriousFlywheel. Deployment still publishes the SvelteKit static artifact through the existing Pages workflows.
Current boundary: this gates Bazel check/test/Chromium-e2e and CV PDF sync drift with GloriousFlywheel shared-cache attachment. It does not claim remote action execution. Deployment still publishes the SvelteKit static artifact through the existing Pages workflows.

## Production DNS And Health

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"check": "npm run remote:check",
"icons:generate": "tsx scripts/generate-favicon-kit.mts",
"remote:check": "bash scripts/bazel-cache-backed.sh test //:sveltekit_check //:sveltekit_vite_build_smoke //:workspace_package_checks //static/cv:pdfs_synced_test",
"remote:test": "bash scripts/bazel-cache-backed.sh test //:bazel_graph_hygiene //:vitest_unit_tests //:blog_agent_node_tests //:puppeteer_chromium_smoke",
"remote:test": "bash scripts/bazel-cache-backed.sh test //:bazel_graph_hygiene //:bazel_cache_backed_contract //:vitest_unit_tests //:blog_agent_node_tests //:puppeteer_chromium_smoke",
"remote:e2e": "bash scripts/bazel-cache-backed.sh test //:playwright_chromium_e2e",
"remote:all": "npm run remote:check && npm run remote:test && npm run remote:e2e",
"ingest:tinyland-posts": "tsx scripts/ingest-tinyland-posts.mts --adopt-reviewed",
Expand Down
Loading
Loading