Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .cf-studio/config/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,6 @@ ALWAYS open and follow `.cf-studio/config/rules/architecture.md` WHEN modifying
ALWAYS open and follow `.cf-studio/config/rules/patterns.md` WHEN documenting a new connector or defining new data source tables

ALWAYS open and follow `.cf-studio/config/rules/code-conventions.md` WHEN writing or reviewing shell scripts, Python helpers, Argo/Kubernetes YAML, dbt macros, deploy scripts, or any imperative code

ALWAYS open and follow deploy/compose/keycloak/README.md WHEN configuring or debugging local Keycloak auth for the compose stack
<!-- auto-config:rules:end -->
13 changes: 13 additions & 0 deletions .env.compose.example
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,19 @@ INSIGHT_FRONT_PATH=../insight-front
# the default (ghcr.io/constructorfabric/insight-front:latest).
FRONTEND_IMAGE=

# ── Auth mode ──────────────────────────────────────────────────────────
# Which OIDC backend the authenticator BFF logs in against:
# fakeidp : the in-repo fake OIDC provider — no login screen, binds to
# VITE_DEV_USER_EMAIL. Default; zero setup.
# keycloak : a real Keycloak container (:8085) with a login form + the
# custom claims (tenant_id/org_unit/groups/roles). The gateway
# ENFORCES the JWT; the frontend auto-switches to the ghcr build
# (OIDC-capable).
# This is the switch — `./dev-compose.sh up` reads it every run. A per-run
# `--auth=fakeidp|keycloak` flag can override it without editing this file.
# See deploy/compose/keycloak/README.md.
AUTH_MODE=fakeidp

# ── Backend image overrides (per service) ─────────────────────────────
# When set, that service pulls the named image instead of building
# locally — dev-compose.sh up detects these and skips the corresponding
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ up-orb.sh
# docker-compose dev stack
/deploy/compose/build/
/deploy/compose/override.generated.yml
/deploy/compose/keycloak/realm-insight.generated.json
/deploy/seed/.venv/
/deploy/seed/__pycache__/
/deploy/seed/.mypy_cache/
Expand Down
30 changes: 30 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ files under `docs/components/<area>/specs/`.
- [First-run wizard + re-runs](#first-run-wizard--re-runs)
- [External MariaDB / ClickHouse](#external-mariadb--clickhouse)
- [Frontend modes](#frontend-modes)
- [Local dev auth backend (fakeidp / Keycloak)](#local-dev-auth-backend-fakeidp--keycloak)
- [Backend image fallback (ghcr)](#backend-image-fallback-ghcr)
- [Settings reference (`.env.compose`)](#settings-reference-envcompose)
6. [Daily workflow](#daily-workflow)
Expand Down Expand Up @@ -261,6 +262,35 @@ wizard. To use it, hand-edit `FRONTEND_MODE=built` in `.env.compose`,
./dev-compose.sh up --no-frontend # backend-only
```

### Local dev auth backend (fakeidp / Keycloak)

The `authenticator` service's login always runs the same BFF code path — only
the IdP behind it changes. Select it with **`AUTH_MODE` in `.env.compose`**
(a persisted setting like `FRONTEND_MODE`):

- `fakeidp` (default) — a tiny in-repo test double, no login screen, no setup.
- `keycloak` — a real Keycloak container with an actual login form, for
exercising the genuine OIDC code path.

```dotenv
# .env.compose
AUTH_MODE=keycloak
```

```bash
./dev-compose.sh up # reads AUTH_MODE from .env.compose
./dev-compose.sh up --auth=keycloak # optional per-run override
```

In keycloak mode the api-gateway enforces auth (validates the Keycloak JWT); the
frontend auto-switches to the OIDC-capable `ghcr` build, and Keycloak uses a
`localhost` issuer so the browser reaches it via the published port. See
[`deploy/compose/keycloak/README.md`](deploy/compose/keycloak/README.md) for
login creds, admin console, enforcement, and the custom-claims contract. This is
distinct from [switching the gateway to real
OIDC](#switch-the-gateway-to-real-oidc) below (a production-style toggle against
an external IdP).

### Backend image fallback (ghcr)

Skip the local Rust/dotnet build for one or more services:
Expand Down
22 changes: 22 additions & 0 deletions deploy/compose/insight-init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -427,6 +427,27 @@ write_compose() {
done
echo "" >&2

# ── Auth mode (compose-only) ──────────────────────────────────────
echo "--- Auth ---" >&2
local auth_mode
echo " Which auth backend should the authenticator log in against?" >&2
echo " 1) fakeidp — no login screen, binds to VITE_DEV_USER_EMAIL (default)" >&2
echo " 2) keycloak — real Keycloak login form + custom claims (:8085)" >&2
local auth_choice
while true; do
auth_choice=$(ask " Choice" "1")
case "$auth_choice" in
1|fakeidp)
auth_mode="fakeidp"
break ;;
2|keycloak)
auth_mode="keycloak"
break ;;
*) echo " Please answer 1 or 2." >&2 ;;
esac
done
echo "" >&2

# ── Seeding decision for external DBs ─────────────────────────────
local seed_external=false
if [[ "$MARIADB_EXTERNAL" == "true" || "$CLICKHOUSE_EXTERNAL" == "true" ]]; then
Expand Down Expand Up @@ -457,6 +478,7 @@ write_compose() {
update_env_var "$env_file" VITE_DEV_USER_EMAIL "$DEV_USER_EMAIL"
update_env_var "$env_file" FRONTEND_MODE "$fe_mode"
update_env_var "$env_file" INSIGHT_FRONT_PATH "$fe_path"
update_env_var "$env_file" AUTH_MODE "$auth_mode"

# SEEDED_LOCAL_* gates the first-run auto-seed in dev-compose.sh.
if [[ "$MARIADB_EXTERNAL" == "true" && "$seed_external" != "true" ]]; then
Expand Down
150 changes: 150 additions & 0 deletions deploy/compose/keycloak/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
# Keycloak auth mode (compose stack)

How the compose stack authenticates is selected by **`AUTH_MODE` in
`.env.compose`** (a persisted setting, exactly like `FRONTEND_MODE`; a `--auth`
flag is only an optional per-run override). The mode changes what the
**frontend** and **api-gateway** do — the frontend either impersonates a seeded
user or runs a real OIDC login, and the gateway either bypasses auth or enforces
it. Two modes:

| `AUTH_MODE` | IdP | Login | Use it for |
| --- | --- | --- | --- |
| `fakeidp` (default) | [`fakeidp`](../../../src/backend/services/fakeidp/README.md) — a tiny in-repo test double | No login screen: `/authorize` mints a code and 302s straight back | Day-to-day backend / frontend work. Fast, no setup. |
| `keycloak` | A real [Keycloak](https://www.keycloak.org/) 26.4 container | An actual Keycloak login form — username/password, real session | Exercising the real OIDC code path: login-form UX, real token claims, admin-console poking, anything that must behave like a customer's IdP. |

## Auth flow

```text
fakeidp browser → frontend (impersonates a seeded user, unsigned dev token)
→ gateway (no-auth.yaml, no validation) → analytics / identity

keycloak browser ⇄ Keycloak (redirect to the login form; OIDC code + PKCE)
frontend (SPA via oidc-client-ts; holds the access token)
→ gateway (keycloak.yaml, validates the Bearer JWT) → analytics / identity
```

In **keycloak** mode the frontend runs the OIDC authorization-code + PKCE flow
**directly** (a SPA, using `oidc-client-ts`): the browser is redirected to the
Keycloak login form, the SPA receives and holds the access token, and sends it
as a `Bearer` header on every `/api` call. The api-gateway validates that JWT
(issuer + JWKS signature + audience). In **fakeidp** mode there is no login — the
frontend binds requests to an impersonated seeded user and the gateway does not
validate. (The `authenticator` service is not in the compose login path; the
frontend talks OIDC directly.)

## Configure it (in `.env.compose`)

Set the mode in `.env.compose` — the same file that holds `FRONTEND_MODE`, DB
settings, etc.:

```dotenv
AUTH_MODE=keycloak
```

Then bring the stack up normally:

```bash
./dev-compose.sh up
```

Any `up` reads `AUTH_MODE` from `.env.compose` and, for `keycloak`, wires the
whole stack for real login:

- generates the realm from the seed roster and starts Keycloak (single
container, `:8085`, profile `auth-keycloak`);
- configures the **frontend** to run the OIDC code+PKCE flow directly (SPA),
using the public `insight` client — auto-switching `FRONTEND_MODE` to `ghcr`
(the only frontend that injects `window.__OIDC_CONFIG__` at runtime);
- switches the **api-gateway** to `keycloak.yaml` (`auth_disabled: false`) so it
**validates** the Keycloak-issued JWT on every request.

On a fresh checkout the first-run wizard (`deploy/compose/insight-init.sh`) asks
for the mode and writes it to `AUTH_MODE` for you.

**Per-run override (optional):** `./dev-compose.sh up --auth=keycloak` (or
`--auth=fakeidp`) overrides `AUTH_MODE` for that one run without editing
`.env.compose`.

Switch back to the bypass by setting `AUTH_MODE=fakeidp` (or `--auth=fakeidp`);
switching modes stops the other IdP so only the active one runs.

## Auth enforcement (fakeidp bypasses, keycloak enforces)

The api-gateway config is selected by mode:

| Mode | Gateway config | Behavior |
| --- | --- | --- |
| `fakeidp` | `no-auth.yaml` (`auth_disabled: true`) | No JWT required. Every request gets a default context; the frontend supplies the impersonated identity. This is the **bypass**. |
| `keycloak` | `keycloak.yaml` (`auth_disabled: false`) | The gateway **validates** the Bearer JWT (issuer + JWKS signature + `audience: insight`). No/invalid token → `401`; a valid Keycloak token → the caller resolves to that persona. |

## Logging in

Any seeded persona logs in with password `insight-dev`:

- Your dev-lead identity — `VITE_DEV_USER_EMAIL` (from `.env.compose`).
- Any other seeded person — `email_*@company.nonpresent`.

### Dev impersonation vs. real login

`VITE_DEV_USER_EMAIL` does double duty: it seeds the realm roster's dev-lead
persona **and** it's the **frontend** dev-impersonation trigger (a non-empty
value makes the frontend skip OIDC and bind requests to that person; the
api-gateway runs `no-auth.yaml` and takes the caller identity from the
frontend, so impersonation is purely a frontend concern). Those two roles
conflict in keycloak mode — you need the value to build the realm, but if the
frontend also sees it, real login never happens.

Keycloak mode resolves this automatically. On `up` with `AUTH_MODE=keycloak`,
`dev-compose.sh`:

- passes the email to the realm generator explicitly
(`gen-realm.py --dev-email …`), so the roster anchor is independent of the
impersonation trigger;
- blanks `VITE_DEV_USER_EMAIL`/`DEV_USER_EMAIL` **only on the frontend services**
(`insight-front-dev`/`-built`/`-ghcr`) via the generated compose override
(`deploy/compose/override.generated.yml`) — the shell variable is left intact,
so the **seed step and the realm roster still get the real value** (blanking
it globally would break seeding);
- exports `OIDC_ISSUER`/`OIDC_CLIENT_ID=insight`/`OIDC_SCOPES` for the frontend
and auto-switches `FRONTEND_MODE` to `ghcr` (the frontend that injects the
runtime OIDC config).

So dev-impersonation is off in keycloak mode and the dev lead logs in like
anyone else — as a real Keycloak user in the generated realm. Your
`.env.compose` is untouched.

> One bypass is *not* auto-cleared: if `AUTH_DISABLED=true`, auth is skipped
> regardless. Keycloak mode warns when it sees it — unset it to exercise the
> real login flow.

### Admin console

`http://localhost:8085/kc/admin/` — `admin` / `admin`.

## Custom claims

Every seeded user's tokens (id token, access token, userinfo) carry these
claims, on both the `insight` and `insight-authenticator` clients:

| Claim | Source |
|-------|--------|
| `tenant_id` | static seed tenant UUID |
| `org_unit` | user attribute = team (`executive` for CEO) |
| `groups` | `/development /sales /hr /support /executive` |
| `roles` | realm role (`insight-admin`/`insight-lead`/`insight-member`) |
| `aud` += `insight` | gateway audience check |

## Realm is generated — don't hand-edit the JSON

`deploy/compose/keycloak/realm-insight.generated.json` is an ephemeral,
gitignored artifact rebuilt from the seed roster on every `up --auth=keycloak`.
To change realm shape (users, clients, mappers, roles), edit the generator's
inputs instead:

- [`deploy/seed/profiles.py`](../../seed/profiles.py) — the roster
(`build_roster`), team/role assignments, and the dev-lead email resolution.
- [`gen-realm.py`](./gen-realm.py) — the realm generator itself (clients,
protocol mappers, role mapping).

Hand-editing the generated `.json` only lasts until the next `up
--auth=keycloak`, which overwrites it.
100 changes: 100 additions & 0 deletions deploy/compose/keycloak/api-gateway.keycloak.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Insight API Gateway — Local Development (Keycloak, auth ENFORCED)
#
# Compose-only config. docker-compose.yml always bind-mounts this file into the
# api-gateway at /app/config-compose/keycloak.yaml; the gateway's `-c` flag is an
# env switch (${API_GATEWAY_CONFIG:-/app/config/no-auth.yaml}) that dev-compose.sh
# points here when AUTH_MODE=keycloak. It is NOT in the backend service's own
# config dir.
#
# Unlike no-auth.yaml, auth is ENABLED here: the gateway validates the
# Keycloak-issued
# JWT the SPA frontend sends as a Bearer token.
#
# Why a static, fully-resolved file (vs insight.yaml's ${OIDC_*} placeholders):
# gears does NOT interpolate ${ENV} in the mounted config (that substitution is
# a Helm render-time step in the k8s path), and hyphenated APP__gears__<gear>
# env-override keys are invalid env-var names. So compose needs concrete values.
#
# Split-horizon (public issuer vs internal JWKS):
# - issuer_url = the PUBLIC browser-facing issuer (localhost:8085). It must
# equal the token `iss`, which the browser SPA obtains over the published
# port.
# - jwks_url = fetched by the gateway over the compose network at
# keycloak:8085 (Keycloak runs with KC_HOSTNAME_BACKCHANNEL_DYNAMIC=true so
# it serves the certs endpoint on the internal host).

server:
home_dir: "~/.insight"

logging:
default:
console_level: debug

gears:
api-gateway:
config:
bind_addr: "0.0.0.0:8080"
enable_docs: true
cors_enabled: true
prefix_path: "/api"
openapi:
title: "Insight API (dev, keycloak)"
version: "0.1.0-dev"
description: "Insight Platform API — auth enforced via Keycloak"
auth_disabled: false

auth-info:
config:
issuer_url: "http://localhost:8085/kc/realms/insight"
client_id: "insight"
redirect_uri: "http://localhost:3000/callback"
scopes: "openid profile email"

oidc-authn-plugin:
config:
vendor: "hyperspot"
priority: 50
issuer_url: "http://localhost:8085/kc/realms/insight"
audience: "insight"
jwks_url: "http://keycloak:8085/kc/realms/insight/protocol/openid-connect/certs"
jwks_refresh_interval_seconds: 300
tenant_claim: "tenant_id"
subject_type: "user"
leeway_seconds: 60

gear-orchestrator:
config: {}

grpc-hub:
config:
listen_addr: "uds:///tmp/insight-grpc"

authn-resolver:
config:
vendor: "hyperspot"

authz-resolver:
config:
vendor: "hyperspot"

static-authz-plugin:
config:
vendor: "hyperspot"
priority: 100

tenant-resolver:
config:
vendor: "hyperspot"

single-tenant-tr-plugin:
config:
vendor: "hyperspot"
priority: 20

proxy:
config:
routes:
- prefix: "/analytics"
upstream: "http://analytics:8081"
- prefix: "/identity"
upstream: "http://identity:8082"
Loading
Loading