fix(retrieval): enforce group ACL in RAG retrieval (fail closed)

[yuzushi-dev]

Context

Two negative prod feedbacks (2026-07-14) reported RAG answers whose sources 404'd on "View Document". Investigation showed the users (group Engineering, tenant default, groups_enforced=true) had no folder grants, yet retrieval surfaced and answered from documents the direct document endpoint (correctly) denies via RLS. The missing prod grants were fixed operationally; this PR fixes the underlying code defect: group document ACL was enforced on direct document access but not in RAG retrieval.

Security impact

On a groups_enforced tenant, any non-admin user could retrieve and receive answers from any document in their tenant via RAG (chat/stream, non-stream, DRIFT, agent tool), regardless of their group grants. Fail-open authorization bypass.

Root causes fixed

1. Group state never reached retrieval. AuthMiddleware computed group_ids / groups_enforced but built QueryScopes via resolve_query_scopes(tenant_id), leaving enforce_groups=False. resolve_query_scopes now accepts and carries them; auth threads them through. Tenant admins and root/super_admin platform keys are exempted (mirrors the RLS documents policy; preserves root tenant-impersonation).
2. Own-tenant retrieval failed open. _resolve_vector_targets / _resolve_graph_targets applied the group allowlist only with a candidate set or a cross-tenant scope; the common own-tenant, no-candidate query left the target filter None → backends (tenant-filter only) returned everything. Now they resolve the allowlist for the viewer's own tenant when enforcement is on and skip the target when empty (fail closed).
3. Scope not propagated to every entry point. The SSE /query/stream handler, DRIFT's inner retrieve() calls, and the agent create_retrieval_tool (stream + non-stream agent branches) called retrieve() without query_scopes. All now pass the authenticated scopes.
4. Result cache ignored the ACL scope. The vector ResultCache key omitted the per-viewer allowlist → same-tenant users with different grants could share an entry. The resolved ACL document scope is now part of the key.

Behaviour is unchanged when group enforcement is off.

Peer review

Adversarially reviewed with Codex (GPT-5.5, xhigh) across 5 rounds; each round's findings (scope threading gaps, admin/root regressions, a broken test double) were fixed until the final review reported no regressions.

Tests

• tests/unit/test_retrieval_group_acl.py (new): own-tenant enforced applies the allowlist without a candidate; empty allowlist yields no target (fail closed); enforcement-off keeps open scope; graph path mirrors vector; resolve_query_scopes threads group state.
• Fixed tests/unit/test_query_stream_agent_mode.py stub to accept the new kwarg.
• Retrieval/visibility integration tests need a database and run in CI.

Known follow-ups (out of scope)

Non-interactive/system contexts with no request scope do not carry query_scopes yet: the worker retrieval path (src/workers/tasks.py) and the eval harness. They need an explicit scope source and are tracked separately.

Deploy

No data migration. Group enforcement must already be intended for the tenant. Requires api restart to take effect (not performed).

[yuzushi-dev]

Follow-up commit 1e0e1f41 — closes the merge blocker found on the prod-data mirror.

CRITICAL fix: QueryUseCase.execute ran the STRUCTURED fast-path (structured_executor) as step 1, before any ACL, with tenant_id only. Its Cypher filters on tenant alone (no group join) and Neo4j has no RLS backstop, so a group-restricted non-admin asking "list all documents" / "how many documents" / "list entities|relationships" could enumerate filenames/entities/relationships from documents their groups were never granted — the same leak this PR closes for vector/graph.

Fix: skip the structured fast-path when query_scopes.enforce_groups is on (fall through to the ACL-enforced RAG path). Unchanged when enforcement is off. Regression test added.

Open follow-up (not fixed here): resolve_query_scopes defaults enforce_groups=False, so any retrieval call site that forgets to pass query_scopes fails OPEN; the reachable API routes set it via auth, but the default should be hardened.

Deploy: pure code, no new deps → per the prod runbook, git reset --hard origin/main + docker restart of api (uvicorn runs without --reload). No image recreate needed.