diff --git a/.cf-studio/config/AGENTS.md b/.cf-studio/config/AGENTS.md index 330d92ada..2b7435549 100644 --- a/.cf-studio/config/AGENTS.md +++ b/.cf-studio/config/AGENTS.md @@ -19,4 +19,6 @@ ALWAYS open and follow `.cf-studio/config/rules/architecture.md` WHEN modifying ALWAYS open and follow `.cf-studio/config/rules/patterns.md` WHEN documenting a new connector or defining new data source tables ALWAYS open and follow `.cf-studio/config/rules/code-conventions.md` WHEN writing or reviewing shell scripts, Python helpers, Argo/Kubernetes YAML, dbt macros, deploy scripts, or any imperative code + +ALWAYS open and follow deploy/compose/keycloak/README.md WHEN configuring or debugging local Keycloak auth for the compose stack diff --git a/.env.compose.example b/.env.compose.example index 09bb2fc25..67d3b6279 100644 --- a/.env.compose.example +++ b/.env.compose.example @@ -41,6 +41,19 @@ INSIGHT_FRONT_PATH=../insight-front # the default (ghcr.io/constructorfabric/insight-front:latest). FRONTEND_IMAGE= +# ── Auth mode ────────────────────────────────────────────────────────── +# Which OIDC backend the authenticator BFF logs in against: +# fakeidp : the in-repo fake OIDC provider — no login screen, binds to +# VITE_DEV_USER_EMAIL. Default; zero setup. +# keycloak : a real Keycloak container (:8085) with a login form + the +# custom claims (tenant_id/org_unit/groups/roles). The gateway +# ENFORCES the JWT; the frontend auto-switches to the ghcr build +# (OIDC-capable). +# This is the switch — `./dev-compose.sh up` reads it every run. A per-run +# `--auth=fakeidp|keycloak` flag can override it without editing this file. +# See deploy/compose/keycloak/README.md. +AUTH_MODE=fakeidp + # ── Backend image overrides (per service) ───────────────────────────── # When set, that service pulls the named image instead of building # locally — dev-compose.sh up detects these and skips the corresponding diff --git a/.gitignore b/.gitignore index 941fac362..a011cade3 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ up-orb.sh # docker-compose dev stack /deploy/compose/build/ /deploy/compose/override.generated.yml +/deploy/compose/keycloak/realm-insight.generated.json /deploy/seed/.venv/ /deploy/seed/__pycache__/ /deploy/seed/.mypy_cache/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d84236115..f7cdd8aa3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -20,6 +20,7 @@ files under `docs/components//specs/`. - [First-run wizard + re-runs](#first-run-wizard--re-runs) - [External MariaDB / ClickHouse](#external-mariadb--clickhouse) - [Frontend modes](#frontend-modes) + - [Local dev auth backend (fakeidp / Keycloak)](#local-dev-auth-backend-fakeidp--keycloak) - [Backend image fallback (ghcr)](#backend-image-fallback-ghcr) - [Settings reference (`.env.compose`)](#settings-reference-envcompose) 6. [Daily workflow](#daily-workflow) @@ -261,6 +262,35 @@ wizard. To use it, hand-edit `FRONTEND_MODE=built` in `.env.compose`, ./dev-compose.sh up --no-frontend # backend-only ``` +### Local dev auth backend (fakeidp / Keycloak) + +The `authenticator` service's login always runs the same BFF code path — only +the IdP behind it changes. Select it with **`AUTH_MODE` in `.env.compose`** +(a persisted setting like `FRONTEND_MODE`): + +- `fakeidp` (default) — a tiny in-repo test double, no login screen, no setup. +- `keycloak` — a real Keycloak container with an actual login form, for + exercising the genuine OIDC code path. + +```dotenv +# .env.compose +AUTH_MODE=keycloak +``` + +```bash +./dev-compose.sh up # reads AUTH_MODE from .env.compose +./dev-compose.sh up --auth=keycloak # optional per-run override +``` + +In keycloak mode the api-gateway enforces auth (validates the Keycloak JWT); the +frontend auto-switches to the OIDC-capable `ghcr` build, and Keycloak uses a +`localhost` issuer so the browser reaches it via the published port. See +[`deploy/compose/keycloak/README.md`](deploy/compose/keycloak/README.md) for +login creds, admin console, enforcement, and the custom-claims contract. This is +distinct from [switching the gateway to real +OIDC](#switch-the-gateway-to-real-oidc) below (a production-style toggle against +an external IdP). + ### Backend image fallback (ghcr) Skip the local Rust/dotnet build for one or more services: diff --git a/deploy/compose/insight-init.sh b/deploy/compose/insight-init.sh index 03e59518f..5d1dc803e 100755 --- a/deploy/compose/insight-init.sh +++ b/deploy/compose/insight-init.sh @@ -427,6 +427,27 @@ write_compose() { done echo "" >&2 + # ── Auth mode (compose-only) ────────────────────────────────────── + echo "--- Auth ---" >&2 + local auth_mode + echo " Which auth backend should the authenticator log in against?" >&2 + echo " 1) fakeidp — no login screen, binds to VITE_DEV_USER_EMAIL (default)" >&2 + echo " 2) keycloak — real Keycloak login form + custom claims (:8085)" >&2 + local auth_choice + while true; do + auth_choice=$(ask " Choice" "1") + case "$auth_choice" in + 1|fakeidp) + auth_mode="fakeidp" + break ;; + 2|keycloak) + auth_mode="keycloak" + break ;; + *) echo " Please answer 1 or 2." >&2 ;; + esac + done + echo "" >&2 + # ── Seeding decision for external DBs ───────────────────────────── local seed_external=false if [[ "$MARIADB_EXTERNAL" == "true" || "$CLICKHOUSE_EXTERNAL" == "true" ]]; then @@ -457,6 +478,7 @@ write_compose() { update_env_var "$env_file" VITE_DEV_USER_EMAIL "$DEV_USER_EMAIL" update_env_var "$env_file" FRONTEND_MODE "$fe_mode" update_env_var "$env_file" INSIGHT_FRONT_PATH "$fe_path" + update_env_var "$env_file" AUTH_MODE "$auth_mode" # SEEDED_LOCAL_* gates the first-run auto-seed in dev-compose.sh. if [[ "$MARIADB_EXTERNAL" == "true" && "$seed_external" != "true" ]]; then diff --git a/deploy/compose/keycloak/README.md b/deploy/compose/keycloak/README.md new file mode 100644 index 000000000..7242e9ea2 --- /dev/null +++ b/deploy/compose/keycloak/README.md @@ -0,0 +1,150 @@ +# Keycloak auth mode (compose stack) + +How the compose stack authenticates is selected by **`AUTH_MODE` in +`.env.compose`** (a persisted setting, exactly like `FRONTEND_MODE`; a `--auth` +flag is only an optional per-run override). The mode changes what the +**frontend** and **api-gateway** do — the frontend either impersonates a seeded +user or runs a real OIDC login, and the gateway either bypasses auth or enforces +it. Two modes: + +| `AUTH_MODE` | IdP | Login | Use it for | +| --- | --- | --- | --- | +| `fakeidp` (default) | [`fakeidp`](../../../src/backend/services/fakeidp/README.md) — a tiny in-repo test double | No login screen: `/authorize` mints a code and 302s straight back | Day-to-day backend / frontend work. Fast, no setup. | +| `keycloak` | A real [Keycloak](https://www.keycloak.org/) 26.4 container | An actual Keycloak login form — username/password, real session | Exercising the real OIDC code path: login-form UX, real token claims, admin-console poking, anything that must behave like a customer's IdP. | + +## Auth flow + +```text +fakeidp browser → frontend (impersonates a seeded user, unsigned dev token) + → gateway (no-auth.yaml, no validation) → analytics / identity + +keycloak browser ⇄ Keycloak (redirect to the login form; OIDC code + PKCE) + frontend (SPA via oidc-client-ts; holds the access token) + → gateway (keycloak.yaml, validates the Bearer JWT) → analytics / identity +``` + +In **keycloak** mode the frontend runs the OIDC authorization-code + PKCE flow +**directly** (a SPA, using `oidc-client-ts`): the browser is redirected to the +Keycloak login form, the SPA receives and holds the access token, and sends it +as a `Bearer` header on every `/api` call. The api-gateway validates that JWT +(issuer + JWKS signature + audience). In **fakeidp** mode there is no login — the +frontend binds requests to an impersonated seeded user and the gateway does not +validate. (The `authenticator` service is not in the compose login path; the +frontend talks OIDC directly.) + +## Configure it (in `.env.compose`) + +Set the mode in `.env.compose` — the same file that holds `FRONTEND_MODE`, DB +settings, etc.: + +```dotenv +AUTH_MODE=keycloak +``` + +Then bring the stack up normally: + +```bash +./dev-compose.sh up +``` + +Any `up` reads `AUTH_MODE` from `.env.compose` and, for `keycloak`, wires the +whole stack for real login: + +- generates the realm from the seed roster and starts Keycloak (single + container, `:8085`, profile `auth-keycloak`); +- configures the **frontend** to run the OIDC code+PKCE flow directly (SPA), + using the public `insight` client — auto-switching `FRONTEND_MODE` to `ghcr` + (the only frontend that injects `window.__OIDC_CONFIG__` at runtime); +- switches the **api-gateway** to `keycloak.yaml` (`auth_disabled: false`) so it + **validates** the Keycloak-issued JWT on every request. + +On a fresh checkout the first-run wizard (`deploy/compose/insight-init.sh`) asks +for the mode and writes it to `AUTH_MODE` for you. + +**Per-run override (optional):** `./dev-compose.sh up --auth=keycloak` (or +`--auth=fakeidp`) overrides `AUTH_MODE` for that one run without editing +`.env.compose`. + +Switch back to the bypass by setting `AUTH_MODE=fakeidp` (or `--auth=fakeidp`); +switching modes stops the other IdP so only the active one runs. + +## Auth enforcement (fakeidp bypasses, keycloak enforces) + +The api-gateway config is selected by mode: + +| Mode | Gateway config | Behavior | +| --- | --- | --- | +| `fakeidp` | `no-auth.yaml` (`auth_disabled: true`) | No JWT required. Every request gets a default context; the frontend supplies the impersonated identity. This is the **bypass**. | +| `keycloak` | `keycloak.yaml` (`auth_disabled: false`) | The gateway **validates** the Bearer JWT (issuer + JWKS signature + `audience: insight`). No/invalid token → `401`; a valid Keycloak token → the caller resolves to that persona. | + +## Logging in + +Any seeded persona logs in with password `insight-dev`: + +- Your dev-lead identity — `VITE_DEV_USER_EMAIL` (from `.env.compose`). +- Any other seeded person — `email_*@company.nonpresent`. + +### Dev impersonation vs. real login + +`VITE_DEV_USER_EMAIL` does double duty: it seeds the realm roster's dev-lead +persona **and** it's the **frontend** dev-impersonation trigger (a non-empty +value makes the frontend skip OIDC and bind requests to that person; the +api-gateway runs `no-auth.yaml` and takes the caller identity from the +frontend, so impersonation is purely a frontend concern). Those two roles +conflict in keycloak mode — you need the value to build the realm, but if the +frontend also sees it, real login never happens. + +Keycloak mode resolves this automatically. On `up` with `AUTH_MODE=keycloak`, +`dev-compose.sh`: + +- passes the email to the realm generator explicitly + (`gen-realm.py --dev-email …`), so the roster anchor is independent of the + impersonation trigger; +- blanks `VITE_DEV_USER_EMAIL`/`DEV_USER_EMAIL` **only on the frontend services** + (`insight-front-dev`/`-built`/`-ghcr`) via the generated compose override + (`deploy/compose/override.generated.yml`) — the shell variable is left intact, + so the **seed step and the realm roster still get the real value** (blanking + it globally would break seeding); +- exports `OIDC_ISSUER`/`OIDC_CLIENT_ID=insight`/`OIDC_SCOPES` for the frontend + and auto-switches `FRONTEND_MODE` to `ghcr` (the frontend that injects the + runtime OIDC config). + +So dev-impersonation is off in keycloak mode and the dev lead logs in like +anyone else — as a real Keycloak user in the generated realm. Your +`.env.compose` is untouched. + +> One bypass is *not* auto-cleared: if `AUTH_DISABLED=true`, auth is skipped +> regardless. Keycloak mode warns when it sees it — unset it to exercise the +> real login flow. + +### Admin console + +`http://localhost:8085/kc/admin/` — `admin` / `admin`. + +## Custom claims + +Every seeded user's tokens (id token, access token, userinfo) carry these +claims, on both the `insight` and `insight-authenticator` clients: + +| Claim | Source | +|-------|--------| +| `tenant_id` | static seed tenant UUID | +| `org_unit` | user attribute = team (`executive` for CEO) | +| `groups` | `/development /sales /hr /support /executive` | +| `roles` | realm role (`insight-admin`/`insight-lead`/`insight-member`) | +| `aud` += `insight` | gateway audience check | + +## Realm is generated — don't hand-edit the JSON + +`deploy/compose/keycloak/realm-insight.generated.json` is an ephemeral, +gitignored artifact rebuilt from the seed roster on every `up --auth=keycloak`. +To change realm shape (users, clients, mappers, roles), edit the generator's +inputs instead: + +- [`deploy/seed/profiles.py`](../../seed/profiles.py) — the roster + (`build_roster`), team/role assignments, and the dev-lead email resolution. +- [`gen-realm.py`](./gen-realm.py) — the realm generator itself (clients, + protocol mappers, role mapping). + +Hand-editing the generated `.json` only lasts until the next `up +--auth=keycloak`, which overwrites it. diff --git a/deploy/compose/keycloak/api-gateway.keycloak.yaml b/deploy/compose/keycloak/api-gateway.keycloak.yaml new file mode 100644 index 000000000..7d59e8344 --- /dev/null +++ b/deploy/compose/keycloak/api-gateway.keycloak.yaml @@ -0,0 +1,100 @@ +# Insight API Gateway — Local Development (Keycloak, auth ENFORCED) +# +# Compose-only config. docker-compose.yml always bind-mounts this file into the +# api-gateway at /app/config-compose/keycloak.yaml; the gateway's `-c` flag is an +# env switch (${API_GATEWAY_CONFIG:-/app/config/no-auth.yaml}) that dev-compose.sh +# points here when AUTH_MODE=keycloak. It is NOT in the backend service's own +# config dir. +# +# Unlike no-auth.yaml, auth is ENABLED here: the gateway validates the +# Keycloak-issued +# JWT the SPA frontend sends as a Bearer token. +# +# Why a static, fully-resolved file (vs insight.yaml's ${OIDC_*} placeholders): +# gears does NOT interpolate ${ENV} in the mounted config (that substitution is +# a Helm render-time step in the k8s path), and hyphenated APP__gears__ +# env-override keys are invalid env-var names. So compose needs concrete values. +# +# Split-horizon (public issuer vs internal JWKS): +# - issuer_url = the PUBLIC browser-facing issuer (localhost:8085). It must +# equal the token `iss`, which the browser SPA obtains over the published +# port. +# - jwks_url = fetched by the gateway over the compose network at +# keycloak:8085 (Keycloak runs with KC_HOSTNAME_BACKCHANNEL_DYNAMIC=true so +# it serves the certs endpoint on the internal host). + +server: + home_dir: "~/.insight" + +logging: + default: + console_level: debug + +gears: + api-gateway: + config: + bind_addr: "0.0.0.0:8080" + enable_docs: true + cors_enabled: true + prefix_path: "/api" + openapi: + title: "Insight API (dev, keycloak)" + version: "0.1.0-dev" + description: "Insight Platform API — auth enforced via Keycloak" + auth_disabled: false + + auth-info: + config: + issuer_url: "http://localhost:8085/kc/realms/insight" + client_id: "insight" + redirect_uri: "http://localhost:3000/callback" + scopes: "openid profile email" + + oidc-authn-plugin: + config: + vendor: "hyperspot" + priority: 50 + issuer_url: "http://localhost:8085/kc/realms/insight" + audience: "insight" + jwks_url: "http://keycloak:8085/kc/realms/insight/protocol/openid-connect/certs" + jwks_refresh_interval_seconds: 300 + tenant_claim: "tenant_id" + subject_type: "user" + leeway_seconds: 60 + + gear-orchestrator: + config: {} + + grpc-hub: + config: + listen_addr: "uds:///tmp/insight-grpc" + + authn-resolver: + config: + vendor: "hyperspot" + + authz-resolver: + config: + vendor: "hyperspot" + + static-authz-plugin: + config: + vendor: "hyperspot" + priority: 100 + + tenant-resolver: + config: + vendor: "hyperspot" + + single-tenant-tr-plugin: + config: + vendor: "hyperspot" + priority: 20 + + proxy: + config: + routes: + - prefix: "/analytics" + upstream: "http://analytics:8081" + - prefix: "/identity" + upstream: "http://identity:8082" diff --git a/deploy/compose/keycloak/gen-realm.py b/deploy/compose/keycloak/gen-realm.py new file mode 100644 index 000000000..b7a58748e --- /dev/null +++ b/deploy/compose/keycloak/gen-realm.py @@ -0,0 +1,223 @@ +#!/usr/bin/env python3 +"""Generate the `insight` Keycloak realm from the seeded 25-person org. + +Reads the same roster builder the DB seeder uses +(`deploy/seed/profiles.py::build_roster`) so every user in the realm +matches a row in `identity.persons`, then emits an importable Keycloak +realm JSON: 25 users, the `insight` + `insight-authenticator` clients, +their 5 shared protocol mappers, the 4 team groups + `executive`, and +the 3 realm roles. + +Usage: + python3 gen-realm.py --out deploy/compose/keycloak/realm-insight.generated.json +""" + +from __future__ import annotations + +import argparse +import json +import os +import sys +from pathlib import Path + +# `deploy/seed` is a sibling package (not installed), so it has to be put on +# sys.path explicitly to import `profiles` from this script's location. +_SEED_DIR = Path(__file__).resolve().parents[2] / "seed" +sys.path.insert(0, str(_SEED_DIR)) + +from profiles import Person, build_roster, get_dev_user_email # noqa: E402 + +REALM_NAME = "insight" +DEV_PASSWORD = "insight-dev" + +TEAMS = ["development", "sales", "hr", "support", "executive"] + +REALM_ROLES = ["insight-admin", "insight-lead", "insight-member"] + +# Person.role -> realm roles the user is granted. +_ROLE_TO_REALM_ROLES: dict[str, list[str]] = { + "ceo": ["insight-admin", "insight-lead"], + "lead": ["insight-lead"], + "ic": ["insight-member"], +} + + +def _org_unit(person: Person) -> str: + """CEO has no team (Person.team is None) → the literal 'executive'.""" + return person.team if person.team is not None else "executive" + + +def _protocol_mappers(tenant_id: str) -> list[dict]: + """The 5 shared mappers, identical on both clients (Input table).""" + common = { + "id.token.claim": "true", + "access.token.claim": "true", + "userinfo.token.claim": "true", + } + return [ + { + "name": "tenant_id", + "protocol": "openid-connect", + "protocolMapper": "oidc-hardcoded-claim-mapper", + "consentRequired": False, + "config": { + **common, + "claim.name": "tenant_id", + "claim.value": tenant_id, + "jsonType.label": "String", + }, + }, + { + "name": "org_unit", + "protocol": "openid-connect", + "protocolMapper": "oidc-usermodel-attribute-mapper", + "consentRequired": False, + "config": { + **common, + "user.attribute": "org_unit", + "claim.name": "org_unit", + "jsonType.label": "String", + }, + }, + { + "name": "groups", + "protocol": "openid-connect", + "protocolMapper": "oidc-group-membership-mapper", + "consentRequired": False, + "config": { + **common, + "claim.name": "groups", + "full.path": "false", + }, + }, + { + "name": "roles", + "protocol": "openid-connect", + "protocolMapper": "oidc-usermodel-realm-role-mapper", + "consentRequired": False, + "config": { + **common, + "claim.name": "roles", + "jsonType.label": "String", + "multivalued": "true", + }, + }, + { + "name": "aud-insight", + "protocol": "openid-connect", + "protocolMapper": "oidc-audience-mapper", + "consentRequired": False, + "config": { + "id.token.claim": "false", + "access.token.claim": "true", + "included.client.audience": "insight", + }, + }, + ] + + +def _client_insight(tenant_id: str) -> dict: + return { + "clientId": "insight", + "publicClient": True, + "protocol": "openid-connect", + "standardFlowEnabled": True, + "directAccessGrantsEnabled": True, + "serviceAccountsEnabled": False, + # localhost:3000 = the compose frontend's callback (SPA does the OIDC + # code+PKCE flow directly). localhost:8080 kept for the orbstack/SPA + # host that shares this realm. + "redirectUris": [ + "http://localhost:3000/callback", + "http://localhost:8080/callback", + ], + "webOrigins": ["+"], + "attributes": {"pkce.code.challenge.method": "S256"}, + "protocolMappers": _protocol_mappers(tenant_id), + } + + +def _client_insight_authenticator(tenant_id: str) -> dict: + return { + "clientId": "insight-authenticator", + "publicClient": False, + "protocol": "openid-connect", + "secret": "insight-authenticator-dev-secret", + "standardFlowEnabled": True, + "directAccessGrantsEnabled": False, + "serviceAccountsEnabled": False, + "redirectUris": ["http://localhost:8083/auth/callback"], + "webOrigins": [], + "protocolMappers": _protocol_mappers(tenant_id), + } + + +def _user(person: Person) -> dict: + org_unit = _org_unit(person) + return { + "id": person.uuid, + "username": person.email, + "email": person.email, + "firstName": person.first_name, + "lastName": person.last_name, + "enabled": True, + "emailVerified": True, + "credentials": [ + {"type": "password", "value": DEV_PASSWORD, "temporary": False} + ], + "attributes": {"org_unit": [org_unit]}, + "groups": [f"/{org_unit}"], + "realmRoles": _ROLE_TO_REALM_ROLES[person.role], + } + + +def build_realm(dev_user_email: str, tenant_id: str) -> dict: + roster = build_roster(dev_user_email) + return { + "realm": REALM_NAME, + "enabled": True, + "sslRequired": "none", + "roles": {"realm": [{"name": name} for name in REALM_ROLES]}, + "groups": [{"name": team, "path": f"/{team}"} for team in TEAMS], + "users": [_user(person) for person in roster], + "clients": [ + _client_insight(tenant_id), + _client_insight_authenticator(tenant_id), + ], + } + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--out", required=True, help="Output path for the realm JSON") + parser.add_argument( + "--dev-email", + default=None, + help=( + "Roster-anchor email for the dev-lead persona. Decouples the realm's " + "roster anchor from VITE_DEV_USER_EMAIL (which the frontend/gateway also " + "read as the impersonation trigger). Falls back to VITE_DEV_USER_EMAIL " + "when omitted." + ), + ) + args = parser.parse_args() + + # Explicit --dev-email wins; otherwise fall back to VITE_DEV_USER_EMAIL via + # get_dev_user_email() (which fail-fasts if that is also unset). + dev_user_email = args.dev_email if args.dev_email else get_dev_user_email() + # Same fallback value as deploy/seed/identity.py's run() — that script's + # own TENANT_DEFAULT_ID lookup carries the identical default, so this + # mirrors (rather than introduces) that convention. + tenant_id = os.environ.get( # RULE-DEFAULTS-OK: mirrors deploy/seed/identity.py's TENANT_DEFAULT_ID default, so the realm's tenant_id claim converges with an un-configured seed run + "TENANT_DEFAULT_ID", "00000000-df51-5b42-9538-d2b56b7ee953" + ) + + realm = build_realm(dev_user_email, tenant_id) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + out_path.write_text(json.dumps(realm, indent=2, sort_keys=True) + "\n") + + +if __name__ == "__main__": + main() diff --git a/dev-compose.sh b/dev-compose.sh index ee9098fd3..f867db138 100755 --- a/dev-compose.sh +++ b/dev-compose.sh @@ -105,6 +105,9 @@ Options: --build-only=svc1,svc2 Build only these; everything else from ghcr. --frontend-mode=MODE Override FRONTEND_MODE for this run. (dev | built | ghcr) + --auth=MODE Override AUTH_MODE (fakeidp|keycloak) from + .env.compose for this run only. + (fakeidp | keycloak, default: fakeidp) --no-frontend Don't start any frontend variant. --skip-build Don't rebuild artefacts — reuse what's already in deploy/compose/build/. @@ -169,6 +172,7 @@ cmd_up() { local from_ghcr_csv="" local build_only_csv="" local frontend_mode_override="" + local auth_mode_override="" local skip_build=false local no_frontend=false @@ -182,6 +186,8 @@ cmd_up() { --build-only) build_only_csv="$2"; shift 2 ;; --frontend-mode=*) frontend_mode_override="${1#*=}"; shift ;; --frontend-mode) frontend_mode_override="$2"; shift 2 ;; + --auth=*) auth_mode_override="${1#*=}"; shift ;; + --auth) auth_mode_override="$2"; shift 2 ;; --skip-build) skip_build=true; shift ;; --no-frontend) no_frontend=true; shift ;; --start-airbyte|--start-argo) @@ -209,6 +215,24 @@ cmd_up() { [[ -n "$frontend_mode_override" ]] && FRONTEND_MODE="$frontend_mode_override" FRONTEND_MODE="${FRONTEND_MODE:-dev}" + [[ -n "$auth_mode_override" ]] && AUTH_MODE="$auth_mode_override" + AUTH_MODE="${AUTH_MODE:-fakeidp}" # RULE-DEFAULTS-OK: fakeidp is the documented default auth mode (bypass) + case "$AUTH_MODE" in + fakeidp|keycloak) ;; + *) echo "ERROR: AUTH_MODE must be fakeidp|keycloak (got: $AUTH_MODE)" >&2; return 1 ;; + esac + + # Keycloak mode needs a frontend that injects window.__OIDC_CONFIG__ at + # runtime from OIDC_* env — only the published FE image (front-ghcr) does + # that via its entrypoint. front-dev (Vite) and front-built (plain nginx) + # can't, so they'd render "Nothing to display" with no login. Auto-switch to + # ghcr and say so, rather than silently serving a broken login. + if [[ "$AUTH_MODE" == keycloak && "$FRONTEND_MODE" != "ghcr" && "$no_frontend" != "true" ]]; then + echo "NOTE: AUTH_MODE=keycloak needs the OIDC-injecting frontend image;" >&2 + echo " switching FRONTEND_MODE=$FRONTEND_MODE -> ghcr for this run." >&2 + FRONTEND_MODE="ghcr" + fi + # ── Resolve which services go to ghcr ──────────────────────────── local all_backend="api-gateway analytics identity" local ghcr_list="" @@ -242,10 +266,13 @@ cmd_up() { # ── Generate per-run override ──────────────────────────────────── local override="deploy/compose/override.generated.yml" mkdir -p compose + local want_overrides=false + { [[ -n "$ghcr_list" ]] || [[ "$AUTH_MODE" == keycloak ]]; } && want_overrides=true { echo "# Auto-generated by dev-compose.sh — DO NOT EDIT BY HAND." - echo "# Per-run override that flips selected services to ghcr mode." - if [[ -z "$ghcr_list" ]]; then + echo "# Per-run override: flips selected services to ghcr mode, and in" + echo "# keycloak auth mode turns off frontend dev-impersonation." + if [[ "$want_overrides" != true ]]; then echo "services: {}" else echo "services:" @@ -267,12 +294,76 @@ cmd_up() { YML fi done + # Keycloak mode: turn OFF frontend dev-impersonation so real Keycloak + # login engages. Scoped to the frontend services — the ONLY consumers of + # the impersonation vars — so seed-sample and the realm roster keep the + # real VITE_DEV_USER_EMAIL. (Compose merges these env keys over the base.) + if [[ "$AUTH_MODE" == keycloak ]]; then + local fe + for fe in insight-front-dev insight-front-built insight-front-ghcr; do + cat < "$override" # Ensure the authenticator's dev signing key exists before bring-up. ensure_authenticator_dev_key + # Keycloak mode: generate the realm import file and repoint the + # authenticator's BFF at Keycloak. This must run before `up -d` — the + # keycloak service read-only-mounts the generated file, and if it's + # missing at container-create time Docker creates an empty directory + # at the mount path instead, so --import-realm silently imports nothing. + if [[ "$AUTH_MODE" == keycloak ]]; then + # Hand the roster-anchor email to the generator explicitly (--dev-email) so + # the realm's dev-lead persona is decoupled from VITE_DEV_USER_EMAIL's other + # role (the frontend dev-impersonation trigger). VITE_DEV_USER_EMAIL stays + # set here — the realm roster and the seed step both still need it. The + # frontend impersonation is turned off separately, per-service, in the + # generated override above (NOT by blanking the shell var, which would also + # break seed-sample). + local dev_lead_email="${VITE_DEV_USER_EMAIL:?VITE_DEV_USER_EMAIL must be set (roster anchor for the Keycloak realm; e.g. dev@company.nonpresent — see .env.compose)}" + + echo "=== Generating Keycloak realm import (deploy/compose/keycloak/realm-insight.generated.json) ===" + python3 deploy/compose/keycloak/gen-realm.py \ + --dev-email "$dev_lead_email" \ + --out deploy/compose/keycloak/realm-insight.generated.json + + # Frontend OIDC config. The compose FE does the OIDC code+PKCE flow + # directly (SPA) using the PUBLIC `insight` client; its entrypoint injects + # window.__OIDC_CONFIG__ from these. The issuer is the browser-facing + # localhost URL (reached via the published port). Consumed by + # front-ghcr's ${OIDC_ISSUER}/${OIDC_CLIENT_ID}/${OIDC_SCOPES}. + export OIDC_ISSUER="http://localhost:8085/kc/realms/insight" + export OIDC_CLIENT_ID="insight" + export OIDC_SCOPES="openid profile email" + + # Enforce auth at the gateway: select the compose-only keycloak config via + # the API_GATEWAY_CONFIG env switch the base docker-compose.yml command + # reads (the file is always bind-mounted there). fakeidp mode leaves this + # unset, so the command falls back to no-auth.yaml (bypass). + export API_GATEWAY_CONFIG="/app/config-compose/keycloak.yaml" + # NOTE: the `authenticator` BFF is NOT in the compose login path (the FE + # talks OIDC directly), so we do not repoint it here. + + # AUTH_DISABLED is a separate, blunter bypass; if it's on, real login is + # still skipped regardless. + [[ "${AUTH_DISABLED:-false}" == "true" ]] && { # RULE-DEFAULTS-OK: purely a cosmetic warn-or-not check, not a config value + echo "WARN: AUTH_DISABLED=true forces an auth bypass — unset it to" >&2 + echo " exercise the real Keycloak login flow." >&2 + } + fi + local compose_cmd=(docker compose --env-file "$env_file" -f docker-compose.yml -f "$override") local profiles=() # Pull local DB services into scope unless the user pointed at an @@ -286,6 +377,7 @@ YML *) echo "ERROR: FRONTEND_MODE must be dev|built|ghcr (got: $FRONTEND_MODE)" >&2; return 1 ;; esac fi + profiles+=(--profile "auth-$AUTH_MODE") # ── Build phase ────────────────────────────────────────────────── if [[ "$skip_build" != "true" ]]; then @@ -329,6 +421,15 @@ YML contains "$ghcr_list" "$svc" && mkdir -p "deploy/compose/build/$svc" done + # Stop the OTHER auth mode's IdP if it lingers from a prior in-place `up`. + # Compose profiles decide what to START, not what to stop, so switching auth + # modes without a `down` in between would otherwise leave both IdPs running + # (e.g. fakeidp still up after switching to keycloak). Pass both auth + # profiles so the target service is in scope for `stop`. + local other_idp + [[ "$AUTH_MODE" == keycloak ]] && other_idp=fakeidp || other_idp=keycloak + "${compose_cmd[@]}" --profile auth-fakeidp --profile auth-keycloak stop "$other_idp" >/dev/null 2>&1 || true + echo "=== docker compose up ===" "${compose_cmd[@]}" ${profiles[@]+"${profiles[@]}"} up -d --remove-orphans @@ -363,7 +464,7 @@ YML local frontend_up=true [[ "$no_frontend" == "true" ]] && frontend_up=false - report_service_urls "$frontend_up" + report_service_urls "$frontend_up" "$AUTH_MODE" echo echo "Service URLs: ./dev-compose.sh urls" @@ -384,6 +485,7 @@ YML # caller (arg 1 = "true" when a front-* profile is active). report_service_urls() { local frontend_up="${1:-true}" + local auth_mode="${2:-fakeidp}" local h="localhost" echo "=== Service URLs (exposed host ports) ===" if [[ "$frontend_up" == "true" ]]; then @@ -393,7 +495,18 @@ report_service_urls() { printf ' %-18s %s\n' "Analytics API" "http://$h:${ANALYTICS_PORT:-8081}" printf ' %-18s %s\n' "Identity API" "http://$h:${IDENTITY_PORT:-8082}" printf ' %-18s %s\n' "Authenticator" "http://$h:${AUTHENTICATOR_PORT:-8083}" - printf ' %-18s %s\n' "Fake IdP" "http://$h:${FAKEIDP_PORT:-8084}" + if [[ "$auth_mode" == keycloak ]]; then + printf ' %-18s %s\n' "Keycloak" \ + "http://$h:${KEYCLOAK_PORT:-8085}/kc/admin/ (admin console: admin/admin)" # RULE-DEFAULTS-OK: display-only port default, mirrors the pre-existing per-service *_PORT lines above + # Default login to show testers: a stable ENGINEERING persona (development + # team). Any seeded persona works (email_{development,sales,hr,support}_NN@ + # company.nonpresent, the *_lead variants, or email_ceo@…); all use the same + # password. Independent of VITE_DEV_USER_EMAIL so the hint never drifts. + printf ' %-18s %s\n' " ↳ login" \ + "${KEYCLOAK_DEFAULT_LOGIN:-email_development_01@company.nonpresent} / insight-dev (engineering; any seeded persona works)" + else + printf ' %-18s %s\n' "Fake IdP" "http://$h:${FAKEIDP_PORT:-8084}" + fi if [[ "${CLICKHOUSE_EXTERNAL:-false}" != "true" ]]; then printf ' %-18s %s\n' "ClickHouse HTTP" \ "http://$h:${CLICKHOUSE_HTTP_PORT:-8123} (native $h:${CLICKHOUSE_NATIVE_PORT:-9000}, user ${CLICKHOUSE_USER:-insight})" @@ -422,9 +535,10 @@ cmd_urls() { done env_file="$(resolve_env_file "$env_file")" || return $? set -a; source "$env_file"; set +a + AUTH_MODE="${AUTH_MODE:-fakeidp}" # RULE-DEFAULTS-OK: fakeidp is the documented default auth mode (bypass) # FRONTEND_MODE is always dev|built|ghcr (cmd_up enforces it), so the # frontend is assumed up; report_service_urls defaults to showing it. - report_service_urls + report_service_urls true "$AUTH_MODE" } # ────────────────────────────────────────────────────────────────────── @@ -465,6 +579,7 @@ cmd_down() { "${compose_cmd[@]}" \ --profile front-dev --profile front-built --profile front-ghcr \ + --profile auth-fakeidp --profile auth-keycloak \ --profile build --profile seed \ down $([[ "$wipe" == "true" ]] && echo "--volumes --remove-orphans") @@ -668,6 +783,7 @@ EOF echo "=== docker compose down --volumes --remove-orphans ===" "${compose_cmd[@]}" \ --profile front-dev --profile front-built --profile front-ghcr \ + --profile auth-fakeidp --profile auth-keycloak \ --profile build --profile seed \ --profile local-mariadb --profile local-clickhouse \ down --volumes --remove-orphans || true diff --git a/docker-compose.yml b/docker-compose.yml index 8786e572f..da49ccdab 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -193,8 +193,6 @@ services: identity: {condition: service_started} environment: !!merge <<: *backend-env - # Service config is baked at /app/config/. Compose `command:` below - # selects which one; see CONTRIBUTING.md for the OIDC swap. volumes: # Bind-mount the host-built Linux binary over the baked one. The # ./deploy/compose/build/api-gateway path is populated by `./dev-compose.sh up` @@ -208,12 +206,19 @@ services: # Mount the service's config dir so YAML edits don't require a # full image rebuild. - ./src/backend/services/api-gateway/config:/app/config:ro + # Compose-only Keycloak-mode config (auth ENFORCED). Always mounted but + # only SELECTED when API_GATEWAY_CONFIG points at it (keycloak mode); + # inert otherwise. Kept out of the service's own config dir above. + - ./deploy/compose/keycloak/api-gateway.keycloak.yaml:/app/config-compose/keycloak.yaml:ro + # Which gateway config to run is an env switch: default no-auth.yaml (auth + # bypass, fakeidp mode); dev-compose.sh sets API_GATEWAY_CONFIG to the + # keycloak config when AUTH_MODE=keycloak. See CONTRIBUTING.md. command: - "/app/insight-api-gateway" # - "--" - "/app/insight-api-gateway" - "-c" - - "/app/config/no-auth.yaml" + - "${API_GATEWAY_CONFIG:-/app/config/no-auth.yaml}" - "run" ports: - "${API_GATEWAY_PORT:-8080}:8080" @@ -310,7 +315,9 @@ services: dockerfile: services/authenticator/Dockerfile depends_on: redis: {condition: service_healthy} - fakeidp: {condition: service_started} + # required: false → compose skips this dependency when the fakeidp + # profile is inactive (--auth=keycloak runs auth-keycloak instead). + fakeidp: {condition: service_started, required: false} identity: {condition: service_started} environment: !!merge <<: *backend-env @@ -352,9 +359,11 @@ services: # a throwaway keypair generated at startup and exposes /_control/* test hooks. Builds # entirely from its own Dockerfile (no host-built-binary bind mount), so it is # self-contained. See src/backend/services/fakeidp/README.md and - # cf/NGINX_BFF.md §10 G6. On the default profile — comes up with the stack. + # cf/NGINX_BFF.md §10 G6. Gated behind --profile auth-fakeidp (the default + # auth mode); --profile auth-keycloak brings up `keycloak` instead. fakeidp: !!merge <<: *backend-common + profiles: ["auth-fakeidp"] image: ${FAKEIDP_IMAGE:-insight-fakeidp:dev} container_name: insight-fakeidp build: @@ -381,6 +390,46 @@ services: ports: - "${FAKEIDP_PORT:-8084}:8084" + # ── Keycloak (dev/e2e realm-import IdP — the real-OIDC counterpart to + # fakeidp) ─────────────────────────────────────────────────────────────── + # + # Single container, embedded H2, imports the roster-generated realm at + # deploy/compose/keycloak/realm-insight.generated.json (produced by + # deploy/compose/keycloak/gen-realm.py, gitignored). Gated behind + # --profile auth-keycloak; mutually exclusive with fakeidp's auth-fakeidp + # profile. + keycloak: + image: quay.io/keycloak/keycloak:26.4 + container_name: insight-keycloak + command: ["start-dev", "--import-realm", "--http-relative-path=/kc"] + environment: + KC_BOOTSTRAP_ADMIN_USERNAME: admin + KC_BOOTSTRAP_ADMIN_PASSWORD: admin + # KC_HOSTNAME is the PUBLIC (browser-facing) issuer. The compose + # frontend does the OIDC code+PKCE flow directly from the host browser, + # which reaches Keycloak via the published port at localhost:8085 — so + # the issuer must be localhost. It MUST include /kc: KC26 hostname-v2 + # drops --http-relative-path from the advertised issuer, so a path-less + # value would yield iss=.../realms/insight while endpoints live under /kc. + KC_HOSTNAME: http://localhost:8085/kc + # The api-gateway validates tokens by fetching JWKS over the compose + # network at keycloak:8085 (a different host than the public issuer). + # backchannel-dynamic lets Keycloak serve those back-channel requests on + # the internal host while keeping the public issuer as localhost. + KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true" + KC_HTTP_ENABLED: "true" + KC_HTTP_PORT: "8085" + JAVA_OPTS_APPEND: "-Xms256m -Xmx512m" + ports: ["8085:8085"] + volumes: + - ./deploy/compose/keycloak/realm-insight.generated.json:/opt/keycloak/data/import/realm-insight.json:ro + # `insight`, not `default` — every other service in this stack (see + # x-backend-common / front-* blocks) attaches to the named `insight` + # network; a bare `default` network would leave keycloak unreachable + # by service name from authenticator/gateway. + networks: [insight] + profiles: ["auth-keycloak"] + # ── Gateway (OpenResty edge) ─────────────────────────────────────────── # # Compiles the mounted routes.yaml (same file the chart ships) into nginx.conf diff --git a/docs/components/deployment/specs/DESIGN.md b/docs/components/deployment/specs/DESIGN.md index 0e2df7979..4e7d97f12 100644 --- a/docs/components/deployment/specs/DESIGN.md +++ b/docs/components/deployment/specs/DESIGN.md @@ -531,6 +531,7 @@ Per-tag artifacts are immutable; the Chart Publishing CI does not overwrite. GHC |----------|-------------|-----------| | `ENABLE_AUTO_RELOAD` | Wraps each backend entrypoint in `watchexec --restart` for ~1s reload. Compose-only — never set in a Kubernetes manifest. | stable | | `FRONTEND_MODE` | `ghcr` (published image, default), `dev` (Vite HMR from `INSIGHT_FRONT_PATH`), or `built` (host-built dist). | stable | +| `AUTH_MODE` | `fakeidp` (default, in-repo test IdP) or `keycloak` (real login via a bundled Keycloak container — see [`deploy/compose/keycloak/README.md`](../../../../deploy/compose/keycloak/README.md)). Persisted here; a per-run `--auth` flag overrides it. | stable | | `_IMAGE` | Pull a published image for a backend service instead of building it (e.g. `API_GATEWAY_IMAGE`). | stable | | `*_PORT` | Host port for each published service (Frontend :3000, gateway :8080, …); override on conflict. | stable | | `MARIADB_EXTERNAL` / `_HOST` / `_INTERNAL_PORT`, ClickHouse equivalents | Point the stack at an external DB instead of the bundled container. | stable | diff --git a/src/backend/services/fakeidp/README.md b/src/backend/services/fakeidp/README.md index 1a9da586f..085fe242c 100644 --- a/src/backend/services/fakeidp/README.md +++ b/src/backend/services/fakeidp/README.md @@ -13,6 +13,11 @@ token-endpoint outages). > production, ship in a production image, or be referenced by a production > chart.** See `cf/NGINX_BFF.md` §10 G6 for the decision. +fakeidp is the **default** IdP behind the compose stack's `authenticator` BFF +(`AUTH_MODE=fakeidp` in `.env.compose`). For the real-login alternative +(`AUTH_MODE=keycloak`), see +[`deploy/compose/keycloak/README.md`](../../../../deploy/compose/keycloak/README.md). + ## Why this is NOT a gears-rust toolkit gear (by intent, not by mistake) Every other backend service here is an idiomatic gears-rust gear