diff --git a/.github/workflows/reference-implementation.yml b/.github/workflows/reference-implementation.yml
index 664b68a93..58a1e0a21 100644
--- a/.github/workflows/reference-implementation.yml
+++ b/.github/workflows/reference-implementation.yml
@@ -72,7 +72,6 @@ jobs:
packages/polyfill-connectors/*|packages/polyfill-connectors/**|\
packages/reference-contract/*|packages/reference-contract/**|\
openspec/changes/complete-mcp-slvp-surface/*|openspec/changes/complete-mcp-slvp-surface/**|\
- docs/research/mcp-slvp-*.md|\
reference-implementation/*|reference-implementation/**|\
scripts/check-console-health-surfaces.ts|\
scripts/check-console-health-surfaces.test.ts|\
diff --git a/docs/research/ephemeral-runtime-health-separates-control-plane-capability-from-warm-instance-state-2026-07-16.md b/docs/research/ephemeral-runtime-health-separates-control-plane-capability-from-warm-instance-state-2026-07-16.md
deleted file mode 100644
index 1f929866b..000000000
--- a/docs/research/ephemeral-runtime-health-separates-control-plane-capability-from-warm-instance-state-2026-07-16.md
+++ /dev/null
@@ -1,76 +0,0 @@
----
-title: "Scale-to-zero runtimes separate service capability from warm-instance readiness"
-date: 2026-07-16
-topic: distributed-systems
-tags: [health, scale-to-zero, allocator, runtime, browser-surface]
-status: final
-sources: [cloud-run-autoscaling, cloud-run-min-instances, aws-lambda-environments, kubernetes-probes, chatgpt-session-loss-rootcause, chatgpt-session-discriminator, retained-credential-boundary-surface-process]
----
-
-## CLAIMS
-
-- Cloud Run services default to zero minimum instances and scale a revision to zero when it receives no traffic; a later request creates a new instance. [cloud-run-autoscaling]
-- Cloud Run describes minimum instances as an optional latency optimization that keeps instances warm and incurs idle cost. A service does not need a warm instance merely to remain deployable and callable. [cloud-run-min-instances]
-- AWS Lambda creates an execution environment when an on-demand function is first invoked, may reuse it after the invocation, and advises applications not to depend on an environment being long-lived. [aws-lambda-environments]
-- Kubernetes readiness is an instance-level condition: a readiness probe decides whether a running Pod can receive Service traffic. It does not establish whether a separate control plane can allocate a future instance. [kubernetes-probes]
-- Two isolated ChatGPT connections completed successful runs and then both reported `chatgpt_session_required` immediately after replacement Chromium containers were created. Stable profile keys and, for one connection, a reused profile mount rule out a different or empty profile as the common explanation. [chatgpt-session-loss-rootcause]
-- A read-only check of the surviving replacement surface reached `/api/auth/session` with HTTP 200 but `hasUser=false`, while DOM login and app/account markers were simultaneously present. The connector's authenticated-session probe is therefore the correct discriminator; a reachable page, persisted profile, URL, title, or logged-looking DOM is not proof of authentication. [chatgpt-session-discriminator]
-- The retained-surface contract deliberately preserves a credential-boundary browser process during routine idle and capacity events. After reconciliation with the replacement evidence, its current OpenSpec treats genuine process/container loss as non-green continuity uncertainty with no owner action unless a typed provider invalidation proof exists; it still does not define portable authenticated state that survives process replacement. [retained-credential-boundary-surface-process]
-
-## SOURCES
-
-**cloud-run-autoscaling**
-
-URL: https://docs.cloud.google.com/run/docs/about-instance-autoscaling
-
-Accessed: 2026-07-16
-
-**cloud-run-min-instances**
-
-URL: https://docs.cloud.google.com/run/docs/configuring/min-instances
-
-Accessed: 2026-07-16
-
-**aws-lambda-environments**
-
-URL: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html
-
-Accessed: 2026-07-16
-
-URL: https://docs.aws.amazon.com/lambda/latest/dg/foundation-progmodel.html
-
-Accessed: 2026-07-16
-
-**kubernetes-probes**
-
-URL: https://kubernetes.io/docs/concepts/workloads/pods/probes/
-
-Accessed: 2026-07-16
-
-**chatgpt-session-loss-rootcause**
-
-Path: internal workstream report `chatgpt-session-loss-rootcause-0716.md` (not tracked in this repo)
-
-Accessed: 2026-07-16
-
-**retained-credential-boundary-surface-process**
-
-Paths: `openspec/changes/retain-credential-boundary-surface-process/{proposal.md,design.md,tasks.md,specs/polyfill-runtime/spec.md}`
-
-Accessed: 2026-07-16
-
-**chatgpt-session-discriminator**
-
-Path: internal workstream report `chatgpt-session-discriminator-0716/report.md` (not tracked in this repo)
-
-Accessed: 2026-07-16
-
-## SYNTHESIS
-
-An allocate-on-demand runtime has at least two independent health questions: whether its control plane is currently reachable enough to accept and describe work, and whether a concrete warm instance is currently ready. Scale-to-zero systems intentionally allow the second answer to be "none" while the service remains operable. Requiring a warm instance to keep connection health green converts an optional latency/cost tradeoff into a false availability requirement.
-
-PDPP's dynamic browser runtime should therefore treat a successful, bounded allocator `listSurfaces()` read as current evidence that the allocator API is reachable and returned a valid inventory response. That read does not prove free capacity, successful future container creation, profile-specific startup, CDP readiness, or provider collectability; those deeper facts are established only by lease acquisition, allocator `ensureSurface`, and the run-scoped readiness probe. Active lease/surface failure remains current fail-closed evidence. A prior ready-to-released receipt remains historical proof of a particular run, never a substitute for current allocator capability and never a reason to revive a retired surface row.
-
-Static runtimes differ: when there is no allocation control plane that can create a replacement on demand, current instance readiness can legitimately remain load-bearing. Non-browser connectors have neither concern and must not inherit browser-runtime uncertainty.
-
-Credential-boundary continuity is adjacent but not reducible to health projection. A successful allocator probe, a replacement lifecycle receipt, and a prior successful run cannot prove that provider authentication crossed a browser-process boundary. The current connector/runtime interface has no generic, provider-safe export/import contract for portable authenticated state; the active retention design instead avoids ordinary replacement, while unexpected process loss remains non-green and creates no owner action without provider proof. The replacement incidents therefore require two separate outcomes: PDPP can make every replacement causally auditable without secrets, while durable session transfer remains open until the connector-runtime credential boundary owns an explicit restoration contract and proves it with forced replacement of two isolated connections. That restoration must run and pass the connector's authenticated-session probe before owner action; DOM or profile-presence heuristics cannot substitute. Health must not turn green by treating either a profile mount or an old success as that proof.
diff --git a/docs/research/external-tool-connector-adapter-prior-art-2026-07-09.md b/docs/research/external-tool-connector-adapter-prior-art-2026-07-09.md
deleted file mode 100644
index c50b53b54..000000000
--- a/docs/research/external-tool-connector-adapter-prior-art-2026-07-09.md
+++ /dev/null
@@ -1,60 +0,0 @@
-# External-tool connector adapter prior art
-
-Date: 2026-07-09
-
-## Question
-
-What can a PDPP Collection Profile connector safely delegate to an existing
-personal-data command-line tool, and what must remain explicit at the adapter
-boundary?
-
-## Primary-source check
-
-The review pinned HPI source revision
-[`13685d8`](https://github.com/karlicoss/HPI/tree/13685d8ac432cdeeb09382aa9258cb241eeffeff).
-The repository was active when checked, but its modules do not share one
-record identity or field vocabulary:
-
-- [`my.reddit.common.RedditBase`](https://github.com/karlicoss/HPI/blob/13685d8ac432cdeeb09382aa9258cb241eeffeff/src/my/reddit/common.py)
- exposes `id` and `text`; it does not establish a generic `body` field.
-- [`my.coding.commits.Commit`](https://github.com/karlicoss/HPI/blob/13685d8ac432cdeeb09382aa9258cb241eeffeff/src/my/coding/commits.py)
- exposes `sha` and repository/message/date fields; it does not expose a
- generic `id` field.
-
-An adapter that requires `id` for every module or copies arbitrary upstream
-objects into a loose record schema therefore does not prove a conforming HPI
-connector. Each mapped module needs an explicit normalization function,
-manifest-conformant output, and fixtures pinned to the upstream shape it
-claims to support.
-
-HPI remains useful prior art for delegating source parsing and local export
-access to an established tool. Repository activity is not evidence that every
-module is production-ready, that schemas are uniform, or that mutable source
-state is reconciled.
-
-## Boundary conclusions
-
-1. The external-tool process runner may be shared. Source-specific module
- selection, normalization, record identity, cursor choice, and mutable-state
- reconciliation remain connector responsibilities.
-2. Runtime mappings must be a closed, manifest-declared set. Arbitrary
- environment-provided module/function names cannot create undeclared streams
- or bypass the consent surface.
-3. Output handling must be incremental and bounded. A nominal byte ceiling on
- a fully buffered stdout string is not a streaming design.
-4. Cancellation must terminate the complete child process tree and integrate
- with the connector run's cancellation signal. Timeout-only cleanup is not
- sufficient.
-5. Malformed output, missing required fields, unsupported modules, and partial
- stream failures need distinct Collection Profile outcomes. Invalid JSONL
- cannot disappear as if it were harmless log output.
-6. Mutable streams such as saved-item collections need snapshot or deletion
- reconciliation. An incremental cursor alone cannot prove that removals were
- observed.
-
-## Implication for PDPP
-
-An external-tool adapter is an earned reference-implementation seam, not a new
-Core or Collection Profile abstraction. It is worth implementing when a real
-connector proves the common process boundary end to end. The adapter should
-not land first as a generic framework with no conforming consumer.
diff --git a/docs/research/heb-auth-session-and-passive-collection-2026-07-14.md b/docs/research/heb-auth-session-and-passive-collection-2026-07-14.md
deleted file mode 100644
index 85708fb81..000000000
--- a/docs/research/heb-auth-session-and-passive-collection-2026-07-14.md
+++ /dev/null
@@ -1,54 +0,0 @@
----
-title: "H-E-B account login supports passkeys and verification codes, and unauthenticated my-account traffic lands on an Incapsula-fronted OIDC login page; passive collection therefore needs a browser-session-first path plus an optional saved-sign-in-details repair path"
-date: 2026-07-14
-topic: connectors
-tags: [heb, login, passkey, verification-code, incapsula, oidc, browser-session, credential-capture]
-status: draft
-sources: [heb-help-passkey, heb-help-passkey-fallback, heb-help-phone-code, heb-help-forgot-password, heb-help-new-account, heb-my-account-redirect]
----
-
-## CLAIMS
-
-- H-E-B's help pages confirm passkey support, including adding a passkey from account settings and using a passkey instead of a password or verification code. That means H-E-B login is not password-only, and any automation that claims to handle the whole login surface needs a browser-session handoff for passkey cases.
-- H-E-B's help pages also document a 6-digit phone verification code flow for new or unrecognized devices, password problems, and phone-number updates. That is a manual challenge path, not a background-safe automatic login.
-- The "forgot password" flow is email-code-driven and culminates in creating a new password. That is a distinct recovery path, not a hidden static-secret login API.
-- A public unauthenticated visit to `https://www.heb.com/my-account/your-orders` redirected to an `accounts.heb.com/oidc/auth?...prompt=login...` URL and showed an Incapsula incident page. So the browser-bound entry point is real, and the login surface is protected by bot-challenge infrastructure.
-- The right product model for PDPP is therefore two honest owner choices: session-only secure browser login, or saved encrypted sign-in details for automatic session repair.
-
-## SOURCES
-
-**heb-help-passkey**
-URL: https://www.heb.com/help/account/how-do-i-add-a-passkey
-Accessed: 2026-07-14
-
-**heb-help-passkey-fallback**
-URL: https://www.heb.com/help/account/if-i-add-a-passkey-can-i-still-use-a-password-or-code-to-log-in
-Accessed: 2026-07-14
-
-**heb-help-phone-code**
-URL: https://www.heb.com/help/account/what-are-phone-verification-security-codes
-Accessed: 2026-07-14
-
-**heb-help-forgot-password**
-URL: https://www.heb.com/help/account/i-forgot-my-password
-Accessed: 2026-07-14
-
-**heb-help-new-account**
-URL: https://www.heb.com/help/account/how-do-i-create-a-new-account
-Accessed: 2026-07-14
-
-**heb-my-account-redirect**
-URL: https://www.heb.com/my-account/your-orders
-Observed on: 2026-07-14
-Observed landing URL: https://accounts.heb.com/oidc/auth?client_id=...&prompt=login...
-
-## SYNTHESIS
-
-H-E-B behaves like a browser-bound, challenge-prone consumer account surface with a real identity-provider login front door. Passkeys and verification codes are first-class account recovery/login mechanisms, so a connector should not pretend the whole auth problem is "fill a password once and forget it." The operationally honest PDPP shape is:
-
-1. probe and reuse a live browser session first,
-2. if the session is dead, use encrypted sign-in details only when the owner opted into that path,
-3. on passkey / code / CAPTCHA / Incapsula / unknown-UI cases, hand the browser to the owner and re-probe afterward,
-4. never log or persist provider passwords in the browser/session layer.
-
-That fits the existing browser-session lifecycle and lets the console present a generic dual choice for any browser-bound connector that also supports static-secret capture.
diff --git a/docs/research/heb-site-knowledge-2026-07-14.md b/docs/research/heb-site-knowledge-2026-07-14.md
deleted file mode 100644
index 83d33a80e..000000000
--- a/docs/research/heb-site-knowledge-2026-07-14.md
+++ /dev/null
@@ -1,81 +0,0 @@
-# HEB (heb.com) site knowledge — mined from vana-com/data-connectors
-
-Date: 2026-07-14. Source: `vana-com/data-connectors/connectors/heb` (README.md, heb-playwright.json,
-heb-playwright.js ~857 lines, schemas/*.json), read in full via GitHub API. That connector's
-*architecture* is rejected (see design note in `design-notes/heb-connector-manifest-design-2026-07-14.md`);
-this file preserves only its hard-won facts about H-E-B's website.
-
-## Authentication and session
-
-- Form-based login, no OAuth. The vana connector never fills credentials; it opens
- `https://www.heb.com/my-account/your-orders` headed and has the user sign in manually, polling
- every 2s until logged-in markers appear.
-- Logged-OUT signals: `input[type="password"]` present, `form[action*="sign-in"|"login"]`, or URL
- containing `/challenge`, `/checkpoint`, `/sign-in`, `/login`.
-- Logged-IN signals: `button[aria-label*="account" i]`, `a[href*="/my-account"]`,
- `button[aria-label="My account"]`.
-- Session persists via the browser profile's cookies across runs; no explicit token extraction.
-- No MFA/OTP flow was coded; unknown whether H-E-B accounts ever require one.
-
-## Bot protection: Imperva Incapsula (confirmed)
-
-- The `_Incapsula_Resource` marker appears in the HTML of **every** heb.com page — presence alone
- is NOT a block signal (the vana author explicitly fixed this false-positive).
-- A real block/challenge renders as an **empty shell**: no `h3`, no breadcrumb nav, no
- `[data-testid]` elements, `document.body.children.length <= 2`, and at least one `iframe`
- (the challenge document replaces all content as a single iframe).
-- Secondary markers checked: `iframe[src*="captcha"]`, `[id*="captcha"]`, `[class*="captcha"]`,
- URLs `geo.captcha-delivery.com` / `/challenge` / `/blocked`, titles matching
- `captcha|verify|access.denied|are.you.human|security.check`.
-- Recovery pattern that worked: show headed browser, let the user solve, return to headless,
- sleep 5–8s jittered, retry; after 3+ consecutive blocks pause 30s first.
-- Politeness that worked: 1500–2500 ms fixed waits after every navigation (pages are
- client-hydrated; DOM is not ready on `load`), 400–500 ms between order pages, jittered
- 1.5–3 s between product-page fetches (the surface most likely to trigger Incapsula).
-
-## Data surfaces (all pure DOM scraping — no usable JSON/GraphQL API was found)
-
-### Profile — `GET https://www.heb.com/my-account/profile`
-Label→value scraping (`` with exact text `Name`/`Email`/`Mobile number`, read
-`nextElementSibling`). Delivery-address cards under `main > div > div`, detected by
-`/[A-Z]{2}\s+\d{5}/`; primary flag = card text contains "Primary".
-
-### Order list — `GET https://www.heb.com/my-account/your-orders?page=N`
-- Order links: `a[href*="/my-account/order-history/HEB"]`. **Order IDs are prefixed `HEB`**
- (last path segment of the href).
-- Per-card free-text regex parsing:
- - date `/([A-Z][a-z]+ \d+, \d{4})/` — long-form US dates ("July 14, 2026")
- - total + count `/\$(\d+\.\d+),?\s*(\d+)\s*items?/i`
- - status `/Status:\s*([^\n]+)/i`
- - fulfillment `/(?:Delivery to|Curbside at)\s+([^\n]+)/i` — **the curbside-vs-delivery
- signal is a free-text prefix**, followed by an address/store string.
-- Pagination: `nav[aria-label*="Pagination"] a[href*="page="]`, max page from `page=(\d+)`.
- Global reverse-chronological pagination — NOT year-partitioned like Amazon.
-- **Scope limit: only curbside/delivery orders appear in account history. In-store purchases
- are structurally unreachable** (H-E-B's app has receipt-photo upload, but no web surface).
-
-### Order detail — `GET https://www.heb.com/my-account/order-history/{orderId}`
-- Line items: `a[href*="/product-detail"]` anchors, dedup by href;
- `productId = href.split('/').pop()`.
-- Per-item, from the closest `
` text: quantity `/Quantity:\s*([^\n.]+)/` (free text —
- may be non-numeric for weighted items; unverified), price `/Price:\s*\$?([\d.]+)/`
- (**unverified whether this is unit price or line total**).
-- Item image URL is derived, not scraped: `https://images.heb.com/is/image/HEBGrocery/prd-small/{productId zero-padded to 9 digits}.jpg`
- (full size: `https://images.heb.com/is/image/HEBGrocery/{id}-1`). Reverse-engineered CDN
- convention, not documented by H-E-B.
-- The detail page does NOT expose UPC, department, or nutrition — those only exist on
- product-detail pages (see below).
-
-### Product page — `GET https://www.heb.com/product-detail/{slug}/{productId}` (NOT user data)
-Kept for reference only; a per-user connector should not crawl these (catalog data, and the
-highest-Incapsula-risk surface). UPC/GTIN is available via JSON-LD (`gtin12`/`gtin13`/`gtin`/
-`gtin8` in `script[type="application/ld+json"]`), meta tags, or visible-text fallback.
-Nutrition Facts panel is an ``+`` structure (0-calorie items omit the calorie row);
-ingredients/allergens under `` headers; category from breadcrumb nav.
-
-## Explicitly excluded from the pdpp connector (bloat in the vana design)
-
-Roughly half the vana connector is USDA FoodData Central enrichment: product-name cleaning,
-`api.nal.usda.gov/fdc/v1/foods/search` lookups by UPC then fuzzy name match, USDA nutrient-id
-mapping, provenance/confidence tagging — plus the per-unique-product page crawl that feeds it.
-This is non-user-specific catalog enrichment layered on the order crawl. Deliberately excluded.
diff --git a/docs/research/owner-console-operator-prior-art-gaps-2026-07-09.md b/docs/research/owner-console-operator-prior-art-gaps-2026-07-09.md
deleted file mode 100644
index 804f669fe..000000000
--- a/docs/research/owner-console-operator-prior-art-gaps-2026-07-09.md
+++ /dev/null
@@ -1,418 +0,0 @@
-# Owner Console Prior-Art: Operator-Persona Gaps
-
-Date: 2026-07-09
-Status: Gap-fill research (not a re-derivation of the 2026-06-18 corpus)
-Scope: Fills four archetype gaps in the owner-console SLVP prior-art corpus that the
-Stripe/Datadog/Plaid-heavy pass under-covered: (A) personal/home-server operator
-consoles, (B) long-running sync progress + schedule pause/resume, (C) reconnect vs
-credential-update routing, (D) status semantics that avoid "misleading green."
-
-Cross-links into the existing corpus (read these first, do not re-derive):
-
-- `docs/research/owner-console-slvp-prior-art-index-2026-06-18.md` — master index.
-- `docs/research/owner-console-recovery-and-liveness-prior-art-2026-06-18.md` — Sentry/Temporal
- recovery-surface archetype (one cause, one closing action, progress, terminal reconciliation).
- This doc's §B/§C extend that pattern to *routing* (which affordance fires) and to
- *long-running* operations, which the recovery doc treats as already-scoped single actions.
-- `docs/research/owner-console-source-inventory-and-detail-prior-art-2026-06-18.md` — health/
- freshness/coverage/schedule per source. This doc's §D supplies the status-hierarchy theory
- that inventory page was implicitly assuming but never sourced.
-- `docs/research/connector-credential-session-repair-prior-art-2026-07-01.md` — PDPP's own
- credential/session repair design. This doc's §C is prior art that should have informed it;
- read together.
-- `docs/research/owner-console-add-data-connector-setup-prior-art-2026-06-18.md` — setup/catalog
- (Stripe/Plaid/GitHub). This doc's §A is the missing "steady state after setup" companion —
- setup doc covers onboarding, this covers ongoing operation.
-
-None of the six home-server/operator products in §A, the sync-progress semantics in §B, the
-reconnect-routing patterns in §C, or the status-hierarchy anti-pattern in §D appear anywhere in
-the existing corpus. This is genuinely new ground.
-
----
-
-## A. Personal/home-server operator consoles
-
-The closest persona match to a PDPP owner (self-hosted, single operator, wants "is my system OK"
-without reading logs) is under-represented in a Stripe/Datadog-anchored corpus. Five products
-checked; two (Home Assistant, Tailscale) have deep, well-documented answers; three (Start9,
-Umbrel, Synology) are thinner in public docs but still transferable.
-
-### Home Assistant: Repairs + System Health (strongest match)
-
-- Sources: `home-assistant.io/integrations/repairs/`, `home-assistant.io/integrations/system_health/`,
- `developers.home-assistant.io/docs/core/integration/system_health/`, HA community thread on
- cascading RAG dashboards (`community.home-assistant.io/t/has-anyone-made-a-system-health-dashboard-with-cascading-red-amber-green-statuses/1011065`). Accessed 2026-07-09.
-- Pattern: HA splits "is my system OK" into **two separate, differently-shaped surfaces** rather
- than one health page:
- 1. **Repairs** (Settings > System > Repairs) — a queue of *actionable* issues Home Assistant
- itself detected that need owner intervention (deprecated integration, config error, device
- needs manual action, a reauth/reconfigure flow pending). Each item either offers a direct
- in-dashboard fix or explicit instructions. The sidebar shows an unread-style count badge on
- "Settings," exactly like the update-count badge — repairs are visually equated with pending
- updates, both "things that want you," not just "things that are broken."
- 2. **System Health** — a developer-extensible reporting API where each integration contributes
- a dict of facts (server reachable, request quota remaining, version, connected endpoint).
- Critically, entries can be coroutines: "the frontend will display a waiting indicator and
- will automatically update once the coroutine finishes and provides a result." This means the
- UI has a first-class *loading* state for "we're checking right now," distinct from both
- "healthy" and "unhealthy" — it does not force a premature verdict while the check is in
- flight.
- - The community thread shows real owners independently converging on wanting a **cascading
- Red/Amber/Green rollup** across sub-statuses (HA core, individual devices, backup status,
- update status) — i.e., operators want an aggregate that is provably derived from real child
- states, not a single hand-set flag. This is grassroots evidence for hierarchical status
- (see §D) mattering even at hobbyist scale, not just SaaS-vendor scale.
- - HA does NOT conflate "integration is configured" with "integration is verified working" —
- system_health's reachability checks exist specifically to prove the second, separately from
- the first. This is the same distinction PDPP's source model needs (configured connector ≠
- connector that authenticated ≠ connector with fresh data — see §D).
-
-**Transferable contract for PDPP:**
-- Split "needs your attention" (owner-actionable queue, badge-counted like updates) from "system
- facts" (health snapshot, includes async/loading states) as two distinct surfaces, not one
- merged health page — mirrors PDPP's existing Source vs Recovery split but validates it against
- an independent product.
-- Every "system fact" that requires a live check (can we reach the provider, is the token still
- valid) needs an explicit **checking-in-progress** UI state, not just healthy/unhealthy — resist
- going straight from "unknown" to a guessed verdict.
-- A repairs-style queue should read like a to-do list (one line, one action) — not a diagnostic
- table.
-
-### Tailscale admin console: machine status, key expiry, health checks
-
-- Sources: `tailscale.com/docs/features/access-control/device-management/how-to/filter`,
- Tailscale changelog entries on health-check warnings (IP forwarding, DNS, firewall marking),
- `home-assistant.io/integrations/tailscale/` (documents the exposed entity model). Accessed
- 2026-07-09.
-- Pattern: Tailscale separates three *independent, differently-timed* facts about a device that a
- naive "online/offline" toggle would collapse:
- 1. **Last seen** — a timestamp-based liveness signal (there is deliberately no boolean "online"
- sensor; the HA integration docs note explicitly that consumers must derive online/offline
- from the last-seen timestamp themselves, because point-in-time connectivity is too noisy to
- assert as a single fact).
- 2. **Key expiry** — a wholly separate lifecycle clock (device auth key), with its own filter
- (`expired`, `expiry disabled`) independent of whether the device is currently connected. A
- device can be "seen 2 minutes ago" and simultaneously "key expires in 3 days" — two clocks,
- two owner actions, never merged into one flag.
- 3. **Health checks** (2025-2026 rollout) — configuration-correctness warnings (misconfigured
- subnet router, DNS conflicts, reverse-path-filter drops) that are neither about liveness nor
- about key expiry — they are "this device is reachable and its credentials are fine, but
- something about how it's configured will bite you." Tailscale's own changelog treats these
- as a third, additive layer, not a modifier of the connectivity status.
- - A live GitHub feature request (`tailscale/tailscale#18188`) shows users explicitly asking for
- a fourth layer — service-level health checks so a *node* being "Online" doesn't imply the
- *service running on it* is actually serving traffic — i.e., users want the same
- configured/reachable/serving distinction HA draws, and complain when a product doesn't offer
- it.
-
-**Transferable contract for PDPP:**
-- Do not compress "connector is configured," "credential still valid (has an expiry clock of its
- own)," and "data is actually fresh" into one boolean. Model them as separate clocks/facts the
- way Tailscale models last-seen vs key-expiry vs health-check — this is very close to PDPP's
- actual source-health shape (source configured / credential live / run recently succeeded / data
- fresh) and gives independent primary-source backing for keeping those four facts visually and
- semantically distinct rather than rolling them into a single status pill.
-- "Last seen" (a timestamp) is a more honest liveness signal than a boolean online/offline for
- anything with async, poll-based checking — PDPP's source liveness should prefer "last verified
- working: 4 minutes ago" phrasing over a binary green dot wherever the underlying check is
- periodic rather than a persistent connection.
-
-### Start9 StartOS: service dashboard states + Health Checks + Dependencies
-
-- Sources: `docs.start9.com` Managing Services and Dashboard Overview pages, `start9.com/faq`.
- Accessed 2026-07-09. (Public docs are thinner than HA/Tailscale; treat as directional, not
- exhaustively verified against the live 2026 UI.)
-- Pattern: a small, explicit state machine for each installed service — **Needs Config / Starting
- / Running / Stopping / Stopped** — plus two additive layers:
- 1. **Health Checks**, packager-authored, whose job is explicitly described as conveying "what is
- happening with their service, as well as possible actions they may want to take" — i.e.,
- health checks are required to carry a *next action*, not just a verdict.
- 2. **Dependencies**, which tells the owner when a service's own health is gated on a *different*
- service being correctly configured — an explicit dependency-graph disclosure rather than a
- opaque "degraded" on the dependent service alone.
- - The always-visible connectivity indicator (bottom-left corner: connected/not-connected to the
- server itself) is kept separate from any individual service's status — the transport-layer
- fact and the application-layer fact are never merged.
-
-**Transferable contract for PDPP:**
-- Every health-check-style status PDPP surfaces should be required to carry a next action, not
- just a color — StartOS makes this a packaging *requirement*, not a nice-to-have.
- Recovery-and-liveness doc (2026-06-18) already argues one-cause/one-action; StartOS is
- independent corroboration that this should be enforced as an authoring rule, not left to
- discretion per surface.
- - Where one source's health is downstream of another PDPP subsystem (e.g., a stream depends on
- a source's credential, or an aggregate depends on multiple upstream connectors), disclose the
- dependency explicitly rather than showing an unexplained "degraded."
-
-### Umbrel and Synology DSM (thin public evidence — noted, not load-bearing)
-
-- Umbrel: public docs/community threads mostly surface known bugs (status desync between actual
- container state and dashboard "Not Running" label after manual intervention) rather than a
- documented design philosophy. The one clear signal: Umbrel's own app-store ecosystem includes a
- third-party container-management app (Arcane) explicitly positioned as making container
- management "feel more like a well-crafted development environment than an administrative
- dashboard" — i.e., even within the Umbrel ecosystem, the built-in app-status surface is
- perceived as not enough, and richer tools get built on top. Weak signal, not a pattern to copy.
-- Synology DSM: not independently re-searched in depth this pass (time-boxed); the existing
- HA-community thread references a "Synology System Health widget" being pulled into HA
- dashboards, suggesting DSM's own health page is granular enough to be worth exporting facts
- from, but no primary DSM source was fetched. Flag as an open follow-up if a future pass wants
- it; do not cite DSM specifics from this document.
-
----
-
-## B. Long-running sync progress + schedule pause/resume
-
-### Airbyte Cloud connection status page
-
-- Source: `docs.airbyte.com/cloud/managing-airbyte-cloud/review-connection-status`,
- `docs.airbyte.com/platform/using-airbyte/core-concepts/sync-schedules`, Airbyte engineering
- blog "How Airbyte 1.0 Monitors Sync Progress and Solves OOM Failures." Accessed 2026-07-09.
-- Pattern: each connection has a persistent Status tab showing current status, next-scheduled-sync
- time, and historic sync trend — plus, added specifically to fix a UX gap, **10-second-polled
- real-time progress**: for a live sync, each stream shows how much data has been extracted and
- loaded, how long it's been actively syncing, and how long since data was last loaded. This
- answers three separate owner questions a single spinner cannot: "is it actually moving," "how
- long has this taken," and "has it stalled." Stalled-vs-slow is the actual failure mode a bare
- progress bar can't distinguish, and Airbyte's design explicitly targets it.
- - Rate-limit handling is a first-class UI state, not folded into "syncing": when the source
- itself is throttling, the UI shows an estimated time until the API is available again, with a
- countdown when known. This is a *third* state beyond running/stalled: "we are waiting on an
- external limit, not broken, and here's when it should resume."
- - Two different "Queued" meanings are explicitly kept apart at different granularity: a
- stream-level Queued (waiting within an already-running sync) vs a connection-level Queued
- (the whole job waiting for worker capacity) — collapsing these into one word would mislead the
- owner about what to expect next.
- - Only one sync per connection runs at a time; a newer scheduled run pending against an older
- queued one **replaces** it, so the owner only ever waits on the freshest request, not a stale
- queue.
-
-### Fivetran: paused semantics + historical resync progress
-
-- Source: `fivetran.com/docs/getting-started/fivetran-dashboard/connectors/status`,
- `fivetran.com/docs/connectors/troubleshooting/trigger-historical-re-syncs`, REST API resync
- reference. Accessed 2026-07-09.
-- Pattern: Fivetran's `sync_state` field has four values — `scheduled`, `syncing`, `paused`,
- `rescheduled` (rescheduled = "waiting until more API calls are available in the source service,"
- i.e., a named state for exactly the throttled case, distinct from paused-by-owner). Notably,
- **Fivetran does not expose a single percent-complete number for historical resyncs** — instead
- it shows "date fetched up to" vs "total days of history available" plus a running extracted-row
- count. This is a deliberate choice: a percentage implies a denominator the system may not
- actually know in advance (total volume can be unknown or change mid-sync), so Fivetran prefers
- an honest, partial, but truthful progress signal (date-cursor position) over a smooth-looking
- but potentially-fabricated percentage.
- - Pausing has explicit, load-bearing semantics for in-flight requests: "if the connection is
- paused, the historical sync flag is set but the sync doesn't start until the connection is
- unpaused" — a resync request against a paused connector is accepted and queued, not silently
- dropped or errored, and this is documented behavior an owner can rely on.
- - Paused + in-flight interaction is explicit: if a sync is already running when a pause request
- comes in, Fivetran tries to let it finish; only if that fails does the pause request get
- rejected (409). The owner-visible contract is "pausing never corrupts an in-flight sync."
-
-### Plaid update mode + Bank of America 2026 forced-migration case study
-
-- Source: `plaid.com/docs/link/update-mode/`, `plaid.com/docs/errors/item/`. Accessed 2026-07-09.
-- Pattern (most relevant to §C but included here for the "claims vs reality" angle): Plaid
- explicitly ties long-running/scheduled disruption to a *proactive webhook*
- (`PENDING_DISCONNECT`) fired **before** the item breaks, with a stated grace window ("one week
- after the webhook fires... if the item hasn't gone through update mode, it will be disconnected
- and enter ITEM_LOGIN_REQUIRED"). This is a pattern for "a scheduled sync-breaking change is
- coming" that gives the owner a deadline and a single fix action ahead of failure, rather than
- waiting for the break to notify.
-
-**Transferable contract for PDPP (§B):**
-- A running sync/backfill needs at minimum three owner-visible facts, not one spinner: (1) is data
- currently moving (throughput/last-progress timestamp), (2) how far through is it (progress
- cursor — prefer an honest partial signal like "processed through 2024-03" over a fabricated
- percentage when total volume is uncertain), (3) is it blocked on something external (rate limit,
- with an ETA if known) vs actually stalled.
-- Distinguish **connection-level** queued/paused from **stream/table-level** queued/paused in
- copy — do not use the same word for both if the owner-facing implication differs.
-- "Paused" needs a documented contract for what happens to in-flight and queued work (does a
- resync request survive a pause? does an in-flight run get allowed to finish?) — PDPP's run/sync
- model should state this explicitly rather than leave it implicit, following Fivetran's
- documented pause-interaction rules.
-- Where a scheduled disruption is foreseeable (token/session about to lapse, a provider-side
- migration), prefer proactively surfacing a deadline+single action (Plaid's `PENDING_DISCONNECT`
- pattern) over waiting for the failure state.
-
----
-
-## C. Reconnect vs credential-update UX
-
-This is the routing question: how do systems decide which single affordance to show an owner when
-something in a connection has gone wrong, given multiple different underlying causes (session
-expired, password changed, bank blocking, new consent required)?
-
-### Plaid: severity-graded update mode, one entry point for many causes
-
-- Source: `plaid.com/docs/link/update-mode/`, `plaid.com/docs/errors/item/`. Accessed 2026-07-09.
-- Pattern: Plaid routes *all* of the following into the **same single UI affordance** (re-launch
- Link in update mode): expired login (`ITEM_LOGIN_REQUIRED`), pending forced disconnect
- (`PENDING_DISCONNECT`/`PENDING_EXPIRATION` webhooks), missing permissions the user didn't
- originally grant, and even a wholesale backend migration (the 2026 Bank of America API cutover).
- The owner never has to diagnose *why* — they get one button, and Plaid internally decides how
- much re-auth to demand:
- - For most institutions, update mode intentionally **minimizes the ask**: "if the Item entered
- an error state because the user's OTP token expired, the user may be prompted to provide
- another OTP token, but not to fully re-login." The severity of the underlying cause
- determines how much friction the single flow imposes, not which flow is shown.
- - Cross-app propagation is explicit: if the same bank login is used in multiple apps and gets
- repaired in one, Plaid fires `LOGIN_REPAIRED` to the others so they can auto-dismiss their own
- "needs attention" state rather than asking the owner to redundantly repair the same
- credential twice.
-
-### GoCardless/Nordigen: EUA vs Requisition — the "refresh doesn't help" trap
-
-- Source: `developer.gocardless.com/bank-account-data/statuses/`, service-terms PDF, and multiple
- independent developer bug reports (Firefly III, Actual Budget, opencollective) converging on the
- same failure. Accessed 2026-07-09.
-- Pattern (a negative example worth citing): GoCardless separates a **Requisition** (the
- connection object) from an **End User Agreement** (a time-boxed consent grant, 90-180 days by
- region). Multiple independent open-source integrators hit the same bug: re-linking the account
- (making a new Requisition) silently reused the *old, already-expired* Agreement object, so the
- error persisted even after what looked like a full reconnect. The fix requires recreating the
- Agreement itself, not just the Requisition — but nothing in the UI/API surface tells the
- integrator (or, transitively, the owner) that two different objects need refreshing. The raw
- error is described by the developers themselves as "currently an obscure 401 error" surfaced
- with no actionable copy.
-- This is the anti-pattern PDPP's own connector-credential-session-repair doc (2026-07-01) is
- presumably designed to avoid, but it's worth citing directly: **a "reconnect" affordance that
- refreshes the wrong underlying object (session, not consent-grant; token, not the account link
- itself) will look successful and then fail again**, and the owner has no way to know the retry
- didn't touch the actual broken thing.
-
-### Monarch Money / Copilot Money: consumer bank-reconnect journeys
-
-- Source: Monarch Money support content (via search aggregation), Copilot Money Help Center
- articles (`help.copilot.money`). Accessed 2026-07-09.
-- Pattern: both apps converge independently on the same three design choices:
- 1. A **persistent, named section** for broken connections ("Connections Needing Attention" /
- accounts sorted to the top of Settings > Connections) rather than a transient toast — the
- owner can return to it later without having caught the original notification.
- 2. A **single lightweight action** ("Reverify" / "Sign in") that re-runs just the credential
- step, explicitly preserving all historical data — both apps state this guarantee directly in
- support copy, because the obvious owner fear is "will reconnecting wipe my history."
- 3. An explicit **verification-after-action** contract: Monarch tells the owner exactly how to
- confirm the fix worked ("give it a few minutes... if the connection is healthy, the last sync
- time advances and new transactions appear; if nothing changes, the link still needs
- attention") — i.e., the product tells the owner what "did it actually work" looks like,
- rather than declaring success the moment the credential step completes.
- - Both explicitly warn against delete-and-recreate as a workaround, because it silently
- discards history — Copilot's help center states this as a direct recommendation against a
- common but destructive user instinct.
- - A named edge case both surface: if the underlying institution *username* changed (not just
- password), the lightweight reverify path cannot work and a genuinely new connection is
- required — this is disclosed as a boundary condition, not left to fail silently.
-
-**Transferable contract for PDPP (§C):**
-- Collapse "session expired," "password changed," "provider requires new consent," and "provider
- migrated backend" into **one owner-facing repair affordance**, with the system — not the owner —
- determining how much re-auth friction is actually required (Plaid's model). Do not make the
- owner pick between "reconnect" and "update credentials" as separate menu items if the underlying
- fix is the same flow with variable friction.
- - Before shipping any "reconnect"/"repair" action, verify it actually refreshes *every* object
- that can independently expire (session token AND consent grant AND stored credential) —
- GoCardless's Requisition/EUA split is a documented, still-recurring failure mode where a
- reconnect that only refreshes one object looks successful and silently isn't.
- - State explicitly, in-product, what "the fix worked" looks like (e.g., "last successful sync
- advanced, new records appearing") so the owner isn't left guessing whether a reverify actually
- took — this is the same "terminal reconciliation" idea from the recovery-and-liveness corpus
- doc, but applied specifically to the repair-verification moment, not just the fix action.
- - Explicitly guarantee (and say so) that reconnect/repair never discards history, and name the
- one real boundary case (identity-level credential change forces a genuinely new connection)
- rather than silently failing when it's hit.
- - When the same underlying credential/session serves multiple PDPP surfaces, propagate a repair
- across them (Plaid's `LOGIN_REPAIRED` cross-app pattern) rather than asking the owner to fix
- the same thing more than once if PDPP's own architecture ever has more than one consumer of a
- shared credential.
-
----
-
-## D. Status semantics: avoiding "misleading green"
-
-### GitHub's own postmortem: the canonical primary-source case study
-
-- Source: GitHub's October 21 post-incident analysis (`github.blog/news-insights/company-news/oct21-post-incident-analysis/`)
- and the GitHub engineering blog on the redesigned status site
- (`github.blog/engineering/infrastructure/introducing-the-new-github-status-site/`). Accessed
- 2026-07-09.
-- GitHub's own words on the failure: during the October 21 incident, "many portions of GitHub were
- available throughout the incident" but the only available signals were "green, yellow, and red"
- — a single aggregate traffic light that could not represent a partial outage honestly. Their
- stated fix was structural, not cosmetic: split the single status into independently-tracked
- **components** (Git Operations, API Requests, Issues, Pull Requests, Actions, Pages,
- Codespaces), and explicitly **decouple component status from incident lifecycle** — "a
- component's degraded performance could be representative of a wider incident, but updating its
- status alone doesn't allow tracking mitigation steps." Component status is a snapshot fact;
- incident narrative (Identified → Investigating → Monitoring → Resolved) is a separate,
- time-ordered thread. Conflating the two was part of the original failure.
-
-### Atlassian Statuspage: the cascade-rule model
-
-- Source: `support.atlassian.com/statuspage/docs/top-level-status-and-incident-impact-calculations/`,
- `help.statuspage.io/knowledge_base/topics/using-components`. Accessed 2026-07-09.
-- Pattern: the industry-standard four-state vocabulary (Operational / Degraded Performance /
- Partial Outage / Major Outage) is aggregated to a top-level status via an explicit **priority
- cascade** ("these rules are applied in an if/else structure, so lower-priority rules only apply
- if higher-priority conditions aren't met") — e.g., any component at Major Outage forces the
- top-level to "Partial System Outage" regardless of how many other components are green. This
- means the aggregate is a **provable function of the children**, not an independently-set flag —
- exactly the property the HA community thread (§A) was asking for by hand.
-- Granularity guidance is explicit and numeric: 5-15 components is the cited workable range;
- beyond that, "cognitive load" from an N×M state matrix becomes the actual failure mode, not lack
- of detail. The advice is to group by **what a user depends on**, not by internal
- service-decomposition ("Website" as a parent with "Homepage/Product Pages/Checkout" children),
- and explicitly to choose components based on past incident/impact history, not org chart.
-- A named best practice directly opposes minimalism-as-default: "status pages build trust by
- showing more, not less — this runs counter to the instinct most teams have, which is to
- minimize what's visible so the page looks clean and reassuring." This is a direct citation
- against the temptation to hide sub-states to keep a dashboard looking calm.
-
-### Home Assistant / Tailscale (cross-reference from §A)
-
-Already covered above — repeated here only to name the specific transferable principle: both
-products keep **configured**, **verified-reachable-right-now**, and **freshness/last-seen** (or,
-for HA, "async check still pending") as separate facts rather than folding them into one status
-enum. This is the same shape as the GitHub/Statuspage lesson, but at the level of a single
-resource's lifecycle rather than a whole system's component tree.
-
-**Transferable contract for PDPP (§D):**
-- Any owner-facing aggregate status (a source's overall health, a dashboard's "attention needed"
- count) must be a **derived, provable function of real child states** — never an independently
- settable flag that can drift from the facts underneath it. This is directly checkable: if the
- top-level status and the child statuses can disagree, the aggregation is wrong.
-- Keep **status** (a point-in-time fact about a component) and **incident/repair narrative** (a
- time-ordered story with its own lifecycle) as two different data models, per GitHub's stated
- lesson — do not let "the last thing we said about this" double as "the current state."
-- Cap owner-facing status granularity to what maps to real owner decisions (roughly GitHub's 7
- components, Statuspage's 5-15 guideline) — grouped by what the owner depends on (a source, a
- stream) not by internal subsystem boundaries, and prefer showing a true partial/degraded state
- over collapsing it to green for calm.
-- Never merge "configured" with "verified working" with "data is fresh" into a single pill — keep
- them as separate, independently-truthful facts (HA/Tailscale pattern), and give each an honest
- loading/pending state rather than forcing an early green/red guess while a check is still async.
-
----
-
-## Summary of new transferable contracts (all sections)
-
-1. Split "needs your attention" (actionable queue) from "system facts" (health snapshot with
- explicit loading states) as two surfaces, not one merged page.
-2. Never collapse configured / credential-valid / data-fresh into one status boolean — model as
- independent clocks (HA, Tailscale).
-3. Every health-check-style status must carry a next action, not just a color (StartOS).
-4. Long-running syncs need three signals — moving/stalled, honest progress cursor, external-block
- with ETA — not a single spinner or a possibly-fabricated percentage (Airbyte, Fivetran).
-5. Document what "paused" means for in-flight and queued work; don't leave it implicit (Fivetran).
-6. Collapse all reconnect causes into one owner affordance with system-determined friction, not an
- owner-facing menu of causes (Plaid update mode).
-7. Verify a "reconnect" action refreshes every object that can independently expire — a partial
- refresh that looks successful is a documented, recurring failure mode (GoCardless EUA trap).
-8. State explicitly what "the fix worked" looks like, and guarantee (in-product) that repair never
- discards history (Monarch/Copilot).
-9. Any aggregate status must be a provable function of real child states, never independently
- settable — and status (snapshot) must stay a separate model from incident narrative (timeline)
- (GitHub's own postmortem).
-10. Prefer showing true partial/degraded detail over hiding it for a calmer-looking page — this is
- stated industry best practice, not merely a plausible inference.
diff --git a/docs/research/product-identity-enrichment-boundary-2026-07-15.md b/docs/research/product-identity-enrichment-boundary-2026-07-15.md
deleted file mode 100644
index 87e5bc33f..000000000
--- a/docs/research/product-identity-enrichment-boundary-2026-07-15.md
+++ /dev/null
@@ -1,155 +0,0 @@
----
-title: "Product identity on purchase line items: collection fact versus catalog enrichment"
-date: 2026-07-15
-topic: connectors
-tags: [heb, product-identity, gtin, gs1, catalog-enrichment]
-status: decided-for-heb-now
----
-
-# Product identity enrichment boundary — 2026-07-15
-
-## Question and conclusion
-
-What is the SLVP-grade contract when a retailer exposes its own catalog ID and might
-also expose UPC/EAN/GTIN in page state or product metadata?
-
-**Recommendation for H-E-B now: retain `order_items.product_id` as the nullable,
-provider-scoped H-E-B catalog reference and do not add a GTIN field or a generic
-identifier object.** The current collection path obtains the ID from the purchased
-line's product-detail link; the repository's H-E-B research says GTIN appears only on
-the separately fetched product page, which is deliberately outside the owner-data
-collector. A product-page lookup is catalog enrichment, not a fact observed on the
-purchase line, and must not silently turn `product_id` into a claimed global identity.
-
-This is a contract recommendation, not a schema or code change.
-
-## Local evidence
-
-- The H-E-B manifest and collector emit `product_id` from the last path component of
- the order-detail product link. It is nullable and also participates in the
- order-scoped line-item key. It has no declared global-identifier semantics.
-- The H-E-B site research says product-detail pages, rather than order-detail pages,
- may expose `gtin8`, `gtin12`, `gtin13`, or `gtin` in JSON-LD/metadata; that same
- research excludes product-page crawling as non-owner catalog enrichment and a
- high-risk surface.
-- The supplied worktree does not contain `docs/north-star.md` or
- `ai/research/INDEX.md`; this report therefore could not read or update those named
- corpus files. The H-E-B manifest design, H-E-B research, and current connector were
- read.
-
-## Identity contract
-
-| Thing | Contract | Do not infer |
-| --- | --- | --- |
-| `order_items.id` | An order-scoped purchase-line identity. It identifies this collected line, not a catalog item. | That equal names or equal retailer IDs across historical orders mean the same physical item. |
-| `product_id` | The exact H-E-B provider/catalog identifier observed in the order-detail link. Preserve it as text, including leading zeroes; null when the source link does not yield one. | UPC, EAN, GTIN, SKU, brand identity, package size, or an H-E-B product *variant* taxonomy not evidenced by the source. |
-| GTIN (including UPC/EAN renderings) | A GS1 trade-item identifier. Accept only a source assertion with an explicit GTIN/UPC/EAN field or an unambiguous GS1 Digital Link/JSON-LD property, after digit-length and check-digit validation. | That a numeric provider ID is a GTIN, or that name/price/image matching establishes one. |
-| Variant / packaging / fulfilment | A GTIN identifies a trade item at its assigned packaging level; a package grouping can have its own GTIN. GTIN + CPV is a distinct GS1 variant granularity. The purchased/fulfilled line can also be a substitution or variable-measure amount. | That one GTIN identifies a product family, every pack, every catalog variation, or the exact physical instance delivered. |
-| Inferred match | A separately produced, fallible assertion, never a replacement for an observed identifier. | That a high-confidence match is source truth or may be used as the order line's stable key. |
-
-GS1 describes GTIN as identifying trade items (products/services priced, ordered, or
-invoiced). Its Digital Link syntax represents GTIN in a 14-digit form (padding GTIN-8,
-GTIN-12, and GTIN-13 with leading filler zeroes), and distinguishes GTIN plus a
-Consumer Product Variant qualifier. Schema.org likewise publishes distinct
-`gtin8`/`gtin12`/`gtin13`/`gtin14` properties and describes GTIN-12 as the UPC form and
-GTIN-13 as the EAN/UCC-13 form. These are representations of the GTIN family, not
-evidence that a retailer-local numeric ID is one.
-
-## Provenance, confidence, and failure behavior
-
-1. **Observed provider ID:** provenance is the H-E-B order-detail link and extraction
- rule; confidence is not a score—it is source-observed. If missing or malformed,
- emit `product_id: null`; retain the order line with its order-scoped fallback key.
-2. **Observed GTIN:** only a direct, named source value can populate a future GTIN
- field. Preserve the raw source value in capture evidence where the runtime supports
- it; validate the check digit and normalize a usable GTIN to the 14-digit GS1 form
- for comparison. Invalid, ambiguous, or conflicting source values are **null**, not
- repaired or chosen by heuristic. The collection run should expose a diagnostic,
- rather than emit an invented identity.
-3. **Metadata/catalog lookup:** record the source URL/type, retrieval time, extractor
- version, and the provider product ID used to look it up. Its confidence is
- `source_asserted_catalog`, not purchase-line-observed. A lookup failure or absent
- GTIN leaves the purchase record unchanged.
-4. **Matched/inferred identity:** carry match method/version, candidates or decisive
- evidence, and numeric confidence in the enrichment result. It is nullable and is
- never written back into `product_id`, a GTIN observed field, `order_items.id`, or
- aggregates that claim observed distinct products.
-
-The distinction is deliberately observable: `null` means “this collection contract
-has no valid assertion,” not “unknown GTIN that a consumer may safely guess from the
-name.” A checksum only establishes syntactic plausibility; it does not prove that the
-value belongs to the purchased line.
-
-## Placement decision
-
-### H-E-B now — direct line-item field only
-
-`product_id` belongs directly on `order_items`: it is collected with the purchase
-line, is useful for same-provider deduplication and detail navigation, and the flat
-manifest already makes it queryable. Keep the existing name for migration
-compatibility, but describe it explicitly as an H-E-B provider product ID in future
-manifest prose. Do not rename it to `gtin`, `upc`, `catalog_id`, or
-`canonical_product_id`.
-
-If a future **already-collected purchase response** contains an explicit GTIN, the
-smallest compatible addition is a nullable scalar `gtin` on `order_items`, defined as
-the validated, normalized 14-digit value. Add it only with a field description that
-states the direct source surface and validation rule. The source-specific provenance is
-then stable connector behavior, rather than per-record speculative metadata. Do not
-claim a GTIN merely because a product URL is available.
-
-### Later catalog work — separate enrichment layer/stream
-
-If H-E-B intentionally fetches product pages or embedded catalog state, make that an
-optional, separately declared catalog-identity enrichment stream/layer keyed by
-`order_item_id` plus the observed `provider_product_id`. It should contain the asserted
-GTIN (if any), assertion provenance, retrieval time, extractor/version, validation
-status, and—only for matching—method and confidence. This preserves the purchase fact,
-lets the catalog refresh on its own lifecycle, and makes a crawl's coverage/failures
-honest without downgrading order-item coverage.
-
-For a flat manifest-declared stream, a separate flat enrichment record is preferable to
-an array such as `identifiers: [{ type, value, provenance, confidence }]`: the latter
-creates nested selection/query semantics that current manifest affordances do not
-declare, mixes direct observation with enrichment, and over-designs a one-connector,
-one-identifier case. A reusable identifiers structure should be considered only after
-multiple connectors demonstrate heterogeneous, source-backed identifiers needing the
-same query and provenance behavior.
-
-## Rejected alternatives
-
-1. **Treat H-E-B `product_id` as a UPC/GTIN.** Rejected: it is only a URL-derived
- provider identifier in the current collector; numeric shape is not identifier
- semantics.
-2. **Put a guessed/matched GTIN directly on `order_items`.** Rejected: it conflates
- purchase evidence with a revocable enrichment decision and hides confidence and
- provenance.
-3. **Crawl product pages during the order collector and call the result a line-item
- field.** Rejected: the H-E-B design expressly excludes that non-owner catalog
- surface, and its failures/rate risk should not become invisible purchase-data
- failures.
-4. **Create a universal product ontology now.** Rejected: there is no demonstrated
- cross-connector contract for nested identifiers, variants, packages, and matching.
- The provider ID plus a future narrow GTIN/enrichment seam is lossless and testable.
-5. **Make GTIN the line-item primary key.** Rejected: one GTIN may recur across orders,
- pack/variant granularity can differ, and lines without a valid GTIN must still exist.
- Square similarly keeps an order-local line UID separate from a catalog variation
- reference; Shopify associates a purchased line with a product variant rather than
- collapsing the two identities.
-
-## Sources
-
-All external sources accessed 2026-07-15.
-
-- GS1, [Global Trade Item Number (GTIN)](https://www.gs1.org/standards/id-keys/gtin) — GTIN identifies trade items priced, ordered, or invoiced.
-- GS1, [Digital Link URI Syntax, release 1.4.0](https://ref.gs1.org/standards/digital-link/uri-syntax/1.4.0/) — 14-digit GTIN representation, primary key and CPV qualifier semantics.
-- Schema.org, [Product](https://schema.org/Product) — `gtin8`, `gtin12`, `gtin13`, and `gtin14` field meanings (the rendered reference is also available at [finance.schema.org](https://finance.schema.org/Product)).
-- Shopify, [Admin GraphQL `LineItem`](https://shopify.dev/docs/api/admin-graphql/latest/objects/LineItem) and [ProductVariant](https://shopify.dev/docs/api/admin-graphql/latest/objects/Productvariant) — order line, product variant, SKU, and barcode remain distinct concepts.
-- Square, [Orders API `OrderLineItem`](https://developer.squareup.com/reference/square/enums/OrderLineItem) — order-local `uid`, catalog variation reference, version, and measured quantity are distinct fields.
-
-## Confidence
-
-**High** on the standards distinction and the recommended H-E-B-now boundary. **Medium**
-on the exact future scalar shape because no live H-E-B capture proving an embedded
-purchase-page GTIN was provided; that path should be live-verified before any schema
-change.
diff --git a/docs/research/remote-surface-mobile-trusted-input-2026-07-17.md b/docs/research/remote-surface-mobile-trusted-input-2026-07-17.md
deleted file mode 100644
index a3b97c03d..000000000
--- a/docs/research/remote-surface-mobile-trusted-input-2026-07-17.md
+++ /dev/null
@@ -1,29 +0,0 @@
-# Remote Surface mobile trusted-input research
-
-Access date: 2026-07-17
-
-Scope: durable record of the platform research used for the bounded Remote Surface
-mobile fix. This note adds no implementation scope.
-
-## Sources and conclusions
-
-| Source | Exact URL | Relevant conclusion |
-| --- | --- | --- |
-| “Transient activation - Glossary | MDN” | https://developer.mozilla.org/en-US/docs/Glossary/Transient_activation | Transient activation comes from a meaningful user interaction, is time-limited/consumable, and is available to the event handler for the originating pointer or touch gesture. A delayed transport callback cannot be treated as that activation. |
-| “User activation - Security | MDN” | https://developer.mozilla.org/en-US/docs/Web/Security/Defenses/User_activation | Browser capabilities can be gated by user activation; code should invoke the gated operation from the trusted interaction path rather than an arbitrary asynchronous callback. |
-| “HTMLElement: focus() method - Web APIs | MDN” | https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/focus | Source fact: `focus()` moves focus to an element. Project inference: the proxy call belongs in the trusted local affordance handler, while the remote event only reports confirmed focus state; this timing requirement is supported by the engine evidence below, not by this MDN page alone. |
-| “VisualViewport - Web APIs | MDN” | https://developer.mozilla.org/en-US/docs/Web/API/VisualViewport | Mobile layout and visual viewports can differ; the visual viewport can change when an on-screen keyboard appears. Layout fitting must therefore preserve the existing viewport negotiation and coordinate mapping while constraining the visible surface. |
-| “`` HTML attribute value - HTML | MDN” | https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/meta/name/viewport | Source fact: viewport metadata affects mobile layout sizing. Project inference: keep the existing page/remote viewport contract and address landscape overflow through local stream containment. |
-| W3C, “CSS Viewport Module Level 1” | https://www.w3.org/TR/css-viewport-1/ | Source fact: the specification defines viewport sizing and the initial containing block. Project inference: the stream dialog can use bounded `100%` sizing without changing remote dimensions; the specification does not prove that project-specific remedy. |
-| W3C, “VirtualKeyboard API” | https://www.w3.org/TR/virtual-keyboard/ | Source fact: virtual-keyboard behavior and visible geometry can be distinct from layout viewport behavior. Project inference: preserve the existing keyboard-overlay and safe-area policy and do not infer editability locally. |
-| WebKit Bugzilla, “195884 – Autofocus on text input does not show keyboard” | https://bugs.webkit.org/show_bug.cgi?id=195884 | Engine evidence: WebKit documents that programmatic focus outside touch/user-gesture handling can focus an input without displaying the software keyboard, with behavior varying by platform/version. This is evidence for the project’s observed compatibility constraint, not a universal web-platform guarantee. |
-| Chromium Blink source, `third_party/blink/renderer/core/dom/element.cc` | https://chromium.googlesource.com/chromium/src/%2B/f983be2fa84b1d8aeb5e7714d0ce82db5711c1f8/third_party/blink/renderer/core/dom/element.cc | Engine evidence: Blink’s focus path distinguishes script focus, checks transient activation for restricted script focus, and invokes virtual-keyboard display on focused elements in an activation context. This supports keeping the proxy focus call in the local gesture stack while treating emulation as proxy-focus evidence only. |
-
-## Design consequences
-
-- The transport-confirmed remote `keyboard_focus` event remains the only editable-state authority; no remote editable element is guessed.
-- If confirmation already exists when a trusted coarse-pointer touch is released, the existing one-tap fast path synchronously calls the package-owned `focusTextInput()` primitive.
-- If touch wins the race, the asynchronous event only exposes the existing corner keyboard control as an accessible, non-modal `Tap to type` affordance. Its trusted local tap synchronously focuses the proxy and then clears the affordance; SSE never opens the keyboard.
-- The VisualViewport/layout-viewport distinction informed `100%`/`max-width: 100%` dialog sizing and stream-surface containment, while leaving remote viewport negotiation, coordinates, rotation settling, and safe-area behavior unchanged.
-
-The AGENTS-referenced `ai/research/INDEX.md` was absent in this worktree; this file is the requested durable corpus artifact for the consulted primary sources.
diff --git a/docs/research/slack-stars-usergroups-reminders-readstate-api-reachability-2026-07-10.md b/docs/research/slack-stars-usergroups-reminders-readstate-api-reachability-2026-07-10.md
deleted file mode 100644
index 5536f4497..000000000
--- a/docs/research/slack-stars-usergroups-reminders-readstate-api-reachability-2026-07-10.md
+++ /dev/null
@@ -1,51 +0,0 @@
-# Slack stars/user_groups/reminders/dm_read_states — API reachability audit
-
-Status: decided, folded into OpenSpec change
-`complete-slack-bundled-connector-coverage`.
-
-Scope: read-only source inspection of `rusq/slackdump` and its `rusq/slack`
-API-client dependency, plus Slack's public API reference docs. No Slack API
-calls made, no credentials used, no personal record payloads read.
-
-## Finding
-
-The four Slack connector streams declared `coverage_policy: deferred` /
-`availability.state: unsupported_in_mode` (`stars`, `user_groups`,
-`reminders`, `dm_read_states`) are reachable with the credential the
-connector already captures (`SLACK_TOKEN` xoxc session token +
-`SLACK_COOKIE` `d` cookie). The "unsupported" claim was accurate about
-slackdump's CLI surface (`slackdump list` only supports `users`/`channels`;
-no CLI path calls `stars.list`, `usergroups.list`, `reminders.list`, or
-reads `conversations.info` read-state fields) but not about source
-availability: slackdump's own Go dependency `rusq/slack` (a
-`slack-go/slack` fork) implements `ListStars`/`GetStarred`,
-`GetUserGroups`, `ListReminders`, and `GetConversationInfo`
-(`LastRead`/`UnreadCount`/`UnreadCountDisplay` fields), all authenticated via
-the same `xoxc` token + `d` cookie pair slackdump itself uses
-(`auth.NewValueAuth`, `auth/value.go`).
-
-All four underlying Slack methods are documented, live, non-deprecated:
-
-| Method | Tier | Source |
-|---|---|---|
-| `stars.list` | Tier 3 (50+ req/min) | |
-| `usergroups.list` | Tier 2 (20+ req/min) | |
-| `reminders.list` | Tier 2 (20+ req/min) | |
-| `conversations.info` | Tier 3 (50+ req/min) | |
-
-Tier table: . The 2025-05
-tightening ()
-targets `conversations.history`/`conversations.replies`, not these four
-methods.
-
-## Full evidence trail
-
-Commands, greps, and source excerpts:
-`openspec/changes/complete-slack-bundled-connector-coverage/design-notes/slackdump-and-slack-api-capability-audit.md`.
-
-## Implication
-
-The Slack connector's manifest declarations for these four streams change
-from an accepted-absence (`deferred`) to genuine collection (`collect`,
-implemented via a small direct Slack Web API call path reusing the existing
-session credential). See the OpenSpec change for implementation.
diff --git a/docs/research/source-backed-fulfillment-prior-art-2026-07-09.md b/docs/research/source-backed-fulfillment-prior-art-2026-07-09.md
deleted file mode 100644
index eee135bd1..000000000
--- a/docs/research/source-backed-fulfillment-prior-art-2026-07-09.md
+++ /dev/null
@@ -1,263 +0,0 @@
-# Prior art: source-backed fulfillment for grant-scoped personal data reads
-
-Status: adopted (design-owner reviewed; load-bearing web citations spot-verified 2026-07-09)
-Date: 2026-07-09
-Related: `design-notes/passthrough-resource-server-mode-2026-06-04.md`, `openspec/changes/define-source-backed-fulfillment/`, `design-notes/prior-art/mcp-data-surface-prior-art-2026-06-09.md`, `docs/research/collection-prior-art-deep-dive.md`
-
-Scope: survey of federated-query, API-virtualization, and delegated-authorization systems relevant to PDPP's "source-backed fulfillment" mode (satisfying grant-scoped reads by querying live upstream sources at request time, rather than serving from locally collected/stored records).
-Access date for all citations: 2026-07-09.
-
-## Relationship to existing PDPP research corpus
-
-This artifact extends, and does not restart, three existing corpus entries:
-
-- `design-notes/prior-art/mcp-data-surface-prior-art-2026-06-09.md` already evaluated Steampipe's zero-ETL SQL-over-APIs model against PDPP's typed-tool read surface and adopted Airbyte's `source_defined_cursor` capability-advertisement and opaque-state disciplines. Section 2 below deepens the Steampipe analysis (KeyColumns, two-tier caching) for the fulfillment question specifically.
-- `docs/research/collection-prior-art-deep-dive.md` settled the collection-delivery-mode question (bounded-run pull stays primary; thin sibling profiles only under a three-criteria test). This artifact addresses the complementary read-side question that work bracketed: when the RS answers a query, where may the records come from?
-- `docs/research/acquisition-coverage-profile-prior-art-2026-06-13.md` covers owner-artifact/irregular acquisition (Immich/Timelinize/Plaid coverage-UX). The Plaid citations there are reused, not refetched; Section 6 adds the aggregator cache-vs-pass-through architecture debate.
-
----
-
-## 1. Federated query engines & foreign data wrappers
-
-### How it works
-
-**Trino** decides pushdown per-operation through a connector capability API on `ConnectorMetadata`. When a query plan contains a `ScanFilterProject` node, the optimizer calls `applyFilter`, passing a `Constraint` (summarized as a `TupleDomain` — a column→domain map of allowed value-sets/ranges). The connector returns either `Optional.empty()` (cannot or already did apply it) or a `ConstraintApplicationResult` with a new table handle representing the constraint applied at the source. The same request/response pattern repeats for `applyLimit`, `applyTopN`, `applyProjection`, `applyAggregation`, `applyJoin`, and `applyTableFunction` — each pushdown type is its own capability negotiation, not a single yes/no flag. Critically, the spec requires connectors to return "no-op" (`Optional.empty()`) rather than silently dropping a constraint, "since doing otherwise can cause the optimizer to loop indefinitely" — an honesty requirement baked into the protocol itself, not just documentation. ([Connectors — Trino 481 Documentation](https://trino.io/docs/current/develop/connectors.html), [Pushdown — Trino 481 Documentation](https://trino.io/docs/current/optimizer/pushdown.html), accessed 2026-07-09)
-
-When pushdown is impossible, Trino's fallback is silent-but-correct: the connector fetches unconstrained data and Trino's engine applies the filter/sort/limit itself. This is *always correct* but can be catastrophically slow — a cited GitHub issue on the Prometheus connector's lack of label pushdown describes requests that "time out because of retrieving large amounts of data" when a filterable label isn't pushed down. For sort/Top-N specifically, Trino's docs explicitly frame this as a connector-author *choice*: ignore the pushdown and let the engine sort all rows, or throw a `TrinoException` with `INVALID_ORDER_BY` and an actionable message if fetching everything would be prohibitively expensive. That is, Trino gives connector authors an explicit "refuse loudly" escape hatch instead of forcing silent full-scan-and-filter. ([Pushdown — Trino 481 Documentation](https://trino.io/docs/current/optimizer/pushdown.html); [Prometheus connector pushdown issue #8742](https://github.com/trinodb/trino/issues/8742), accessed 2026-07-09)
-
-**PostgreSQL `postgres_fdw`** takes a conservative-by-default posture: only WHERE clauses using built-in operators/functions are shipped to the remote server, specifically "to reduce the risk of misexecution" — an explicit correctness-over-performance default that can be loosened per-server via the `extensions` allowlist option. Time-dependent predicates (`WHERE created_at >= now()`) are never pushed down because the planner can't guarantee remote/local clock agreement — a direct analogue to freshness-disclosure problems in live personal-data fulfillment. Sort/limit pushdown was added later (PG12+) and only fires when the underlying scan is already pushable. Operators can inspect exactly what was shipped via `EXPLAIN VERBOSE`'s "Remote SQL" line — a debuggability affordance PDPP's grant-scoped fulfillment would want an analogue of. ([PostgreSQL 18 Docs: postgres_fdw](https://www.postgresql.org/docs/current/postgres-fdw.html); [freeCodeCamp: FDW pushdown](https://www.freecodecamp.org/news/fdw-pushdown/), accessed 2026-07-09)
-
-**DuckDB**'s Postgres/SQLite scanner extensions implement projection pushdown unconditionally (only fetch needed columns) and filter pushdown by translating DuckDB's internal `TableFilterSet` into a source-native `WHERE` clause appended to a `COPY` command (Postgres) — filter pushdown was historically opt-in (`pg_experimental_filter_pushdown`) rather than default, reflecting that the DuckDB team treated it as a performance optimization with correctness risk, not a free win. SQLite's in-process nature sidesteps the network-protocol overhead that motivated much of the Postgres scanner's pushdown engineering. ([DuckDB: Postgres Scanner blog](https://duckdb.org/2022/09/30/postgres-scanner); [DuckDB Postgres extension docs](https://duckdb.org/docs/current/core_extensions/postgres/overview), accessed 2026-07-09)
-
-### What transfers
-
-- **The capability-API pattern itself**: a per-operation-type negotiation (`applyFilter`/`applyLimit`/`applySort`/…) that returns "applied" vs "not applied — engine must compensate" is the cleanest primitive found across all research for honest capability disclosure. PDPP's source-backed fulfillment layer could adopt near-identical semantics: each source adapter declares, per query shape, whether it pushed the constraint down or the personal server must filter after fetch.
-- **The "refuse loudly" escape hatch for sort/Top-N** is directly relevant: PDPP grants often request sort/cursor stability that a source (e.g., Slack search) cannot honor at all. An explicit typed refusal (visible to the calling agent) is safer than silent full-fetch-and-sort.
-- **Conservative-by-default pushdown** (postgres_fdw's built-in-only default) models how PDPP should treat filter expressions the personal server can't prove are safe to ship upstream — default to fetch-and-filter-locally, opt into pushdown per predicate class.
-
-### What fails to transfer
-
-- Trino/FDW pushdown decisions optimize for *performance* under a trusted, single-tenant analyst; there's no disclosure-boundary concept — over-fetching from the source is a cost problem, not a privacy problem. In PDPP, fetch-and-filter is a *disclosure event*: pulling more from Gmail than the grant discloses to the client momentarily exists in personal-server memory even if never returned. FDW/Trino have no analogue to "the upstream fetch itself must be minimized independent of what's returned," because there's no adversarial party downstream of the query engine.
-- These systems assume a stable, introspectable schema and a query language (SQL) rich enough to express arbitrary predicates against it. PDPP's grant surface (stream/field projection, filters, sort, search, cursors) must map onto heterogeneous source query languages (Gmail `q=`, Slack search operators) that are *not* relational and don't support arbitrary boolean predicate composition — FDW/Trino's "intersect TupleDomains" model assumes a predicate algebra that doesn't exist upstream.
-- No single-owner concept: connectors serve a DBA-administered fleet of sources on behalf of an organization; there's no "the resource owner also uses the raw source directly and expects freshness/consistency with their own actions elsewhere" problem, which is central to personal data (e.g., a user searches Gmail's own UI and expects PDPP reads to reflect the same state).
-
----
-
-## 2. Steampipe (API-as-tables) vs. CloudQuery (sync-then-query)
-
-### How it works
-
-**Steampipe** models each API resource as a Postgres foreign table via a custom FDW, backed by a plugin SDK with two hydrate-function archetypes: `List` (streams multiple rows, used for enumeration) and `Get` (fetches one row by key). Every table declares `KeyColumns` — the qualifiers (`where` clauses) that map to upstream API parameters, each marked `Required` or optional, with an accepted-operator set. Google's own example, cited in the SDK docs: `GithubListUsers` cannot enumerate "ALL Github users," so Steampipe *requires* `where login = ...` on that table — the API's necessity to scope becomes a required key column, and querying without it is a hard error at plan time, not a slow full scan. This is a strong instance of *honest capability disclosure enforced by the query planner itself*, rather than left to documentation. ([Writing Plugins — Steampipe](https://steampipe.io/docs/develop/writing-plugins); [Using Key Column Qualifiers — Steampipe](https://steampipe.io/docs/guides/key-columns), accessed 2026-07-09)
-
-Steampipe layers two independent caches: a short-TTL (5 min default) query-result cache that is *quals-aware* — "determines whether a cache request is a subset of an existing cached item, taking the quals into account" — plus a separate `Memoize`-based hydrate-function memoization with custom cache-key functions (e.g., AWS plugin keys region-list caching by service ID). This two-tier design (result cache + hydrate cache, each independently keyed) is a mature answer to "caching is a spectrum, not binary." ([Writing Plugins — Steampipe](https://steampipe.io/docs/develop/writing-plugins); [Release 0.3.0 blog](https://steampipe.io/blog/release-0-3-0), accessed 2026-07-09)
-
-**CloudQuery** takes the opposite architectural stance: scheduled ELT syncs pull from source APIs into a persistent warehouse (Postgres, BigQuery, Snowflake, S3, ClickHouse); all queries run against the local copy. This trades live freshness for query speed independent of source-API latency/rate-limits, and — uniquely relevant to PDPP — *enables historical/point-in-time analysis* that live query cannot: "a point-in-time snapshot you can query, compare, and share." ([CloudQuery Blog: Steampipe vs CloudQuery](https://www.cloudquery.io/blog/steampipe-vs-cloudquery), accessed 2026-07-09)
-
-The two projects' own comparison converges on a clean rule of thumb: "Steampipe excels at real-time queries without infrastructure setup" for ad hoc investigation; CloudQuery fits "ongoing governance… loading data into a warehouse for historical analysis." ([CloudQuery Blog: Steampipe vs CloudQuery](https://www.cloudquery.io/blog/steampipe-vs-cloudquery), accessed 2026-07-09)
-
-### What transfers
-
-- This pair *is* the exact tradeoff PDPP is navigating, almost componentized: PDPP's current architecture (collect-then-serve) is the CloudQuery model; source-backed fulfillment is the Steampipe model. Steampipe's `Required` KeyColumns concept maps directly onto "which filters/sort a PDPP stream can satisfy via pushdown vs. must reject or degrade" — this is close to a ready-made vocabulary for PDPP's own capability-declaration schema.
-- The two-tier cache (quals-aware result cache + keyed hydrate memoization) is a strong concrete design for PDPP's "caching is a spectrum" problem — a short TTL on raw upstream responses plus separate memoization of expensive derived hydrate work (e.g., thread reconstruction) maps well onto PDPP's stream/field-projection model.
-- Steampipe's hard failure on missing required quals (rather than silently doing a slow full enumeration) is a strong precedent for what PDPP should do when a grant's filter/sort request has no upstream equivalent: fail typed and explicit, don't silently degrade to full-fetch.
-
-### What fails to transfer
-
-- Steampipe's caching model assumes a semi-trusted operator running ad hoc investigative queries (security/DevOps); it explicitly does not reason about a downstream client's grant boundary — the cache exists purely for perf, and a "cache hit" just means "don't call the API again," not "don't disclose beyond what was requested." PDPP must additionally ask: does serving a cached response satisfy the *current* grant, and was the *original* fetch itself grant-scoped or did it use a broader credential than the request being served now?
-- Neither Steampipe nor CloudQuery deals with per-request authorization scoping — the plugin authenticates once with a static credential (an AWS role, a GitHub PAT) and every query uses the same privilege. PDPP's core distinguishing constraint — one owner, many different grants of differing scope hitting the same source with the same strong credential — has no analogue in either system; both assume the querying principal (an operator) *is* the authorization boundary.
-- CloudQuery's historical/point-in-time capability is something source-backed fulfillment *gives up* relative to PDPP's current collect-then-serve mode; this is worth flagging as a real regression, not just a tradeoff to wave away — change/since cursors over live sources need a different mechanism than "diff two warehouse snapshots."
-
----
-
-## 3. GraphQL federation: Apollo Federation & Apollo Connectors
-
-### How it works
-
-Apollo Federation composes a single graph from independently owned subgraphs, using `@key` to declare entity identity, `@requires`/`@provides` to declare cross-subgraph field dependencies, and `@shareable`/`@external` to declare which subgraph is authoritative for which field. `@requires(fields: "model")` means "the gateway must resolve `model` (possibly from another subgraph) before this subgraph can resolve the requiring field" — a declarative dependency graph the query planner topologically sorts at request time. Composition is strict: a field marked `@shareable` in one subgraph must be marked `@shareable` or `@external` in *every* subgraph that defines it, or composition fails at build time (not query time) — capability mismatches are caught before deploy, not discovered as runtime errors. ([Resolve another subgraph's fields — Apollo Docs](https://www.apollographql.com/docs/graphos/schema-design/federated-schemas/entities/resolve-another-subgraphs-fields); [Apollo Federation Subgraph Spec](https://www.apollographql.com/docs/graphos/schema-design/federated-schemas/reference/subgraph-spec), accessed 2026-07-09)
-
-**Apollo Connectors** (GA alongside GraphOS Router 2.0) let a REST API become a GraphQL subgraph declaratively via `@connect`/`@source` directives and a "selection mapping" language (no imperative resolver code). Crucially, the mapping is *closed-world and exhaustive*: "every field defined in the schema must be mapped at least once in a Connector... you can never map an entire object and expect fields to be implicitly mapped." This forces explicit acknowledgment of what a REST endpoint *can* and *cannot* supply into the graph shape — an implicit capability-disclosure mechanism enforced by the schema-composition step, similar in spirit to Steampipe's required KeyColumns. ([Mapping Language Overview — Apollo Docs](https://www.apollographql.com/docs/graphos/connectors/mapping); [Why use Apollo Connectors? — Apollo Docs](https://www.apollographql.com/docs/graphos/connectors/why-connectors), accessed 2026-07-09)
-
-Federation's most telling limitation for this research is what it explicitly refuses to solve: there is no federation-level concept of arbitrary filter/sort/search pushed across subgraph boundaries. A GraphQL query's `where`/`orderBy`-equivalent arguments are just regular field arguments resolved *within* one subgraph's own resolver logic — federation composes the *shape* of the graph, not cross-subgraph query semantics. A documented, unresolved bug (`@requires` field also `@shareable` across subgraphs) shows that even simple field-dependency resolution has known correctness gaps in the query planner, years into the project's life. ([`@requires` field always resolved from same subgraph — GitHub Issue #2801](https://github.com/apollographql/federation/issues/2801), accessed 2026-07-09)
-
-### What transfers
-
-- Build-time (schema-composition-time) capability verification, rather than runtime discovery, is a strong idea PDPP doesn't currently have an analogue for: if a PDPP source's connector manifest declared filter/sort/search capability per stream, a "compose" step could statically catch grant-schema mismatches before a client ever issues a request — rather than the client hitting a runtime 501-equivalent.
-- Apollo Connectors' exhaustive/closed-world field-mapping requirement is a good discipline to borrow: every field a PDPP stream schema advertises must be explicitly backed by either a pushdown-capable upstream parameter or a documented fetch-and-filter compensation — no implicit "it'll probably work."
-
-### What fails to transfer
-
-- Federation's entity/field-ownership model assumes each field has exactly one (or a small, explicit, composition-verified set of) authoritative resolver(s) known at schema-build time across a *fixed* set of subgraphs owned by *different engineering teams* of the same organization. PDPP's source set is dynamic per grant/connection (a user connects Gmail today, Slack tomorrow) and per-owner — there's no shared build-time composition step across all personal servers; capability disclosure has to happen at connection/grant time, dynamically, not at a central schema-compile step.
-- Federation explicitly does not attempt cross-subgraph filter/sort/search pushdown — this is precisely the hardest part of PDPP's design problem, and federation's answer is "don't do it; keep it inside one subgraph's resolver." That's not available to PDPP because the "subgraph" (the source) doesn't offer arbitrary filter semantics to begin with — the problem federation sidesteps is exactly the one PDPP must solve.
-
----
-
-## 4. Source-native search/query APIs as fulfillment targets
-
-### Gmail API
-
-`messages.list`/`threads.list` accept a `q` parameter supporting "most of the same advanced search syntax as the Gmail web interface" (`from:`, `after:`/`before:`, `is:unread`, `rfc822msgid:`, etc.), combinable with `labelIds[]`. Notably the API's `q` semantics are **not** identical to the web UI — e.g., Gmail's UI performs account-alias expansion on `from:` that the API does not, a documented drift between "what the source natively supports" in two different surfaces of the *same* provider. There is **no arbitrary sort** — `messages.list` returns results in a fixed (reverse-chronological-ish, undocumented-order) sequence; there is no `orderBy` parameter. ([Search and filter messages — Gmail API](https://developers.google.com/workspace/gmail/api/guides/filtering); [List Gmail messages](https://developers.google.com/workspace/gmail/api/guides/list-messages); [users.messages.list reference](https://developers.google.com/workspace/gmail/api/reference/rest/v1/users.messages/list), accessed 2026-07-09)
-
-Change/since semantics are handled by a *separate* endpoint, `users.history.list`, keyed on `startHistoryId` (not a timestamp), returning `messagesAdded`/`messagesDeleted`/`labelAdded`/`labelRemoved` since that checkpoint, itself paginated via `nextPageToken`. History IDs are monotonically increasing but non-contiguous and have a validity window — the primary contract only guarantees they are "typically valid for at least a week" (longer practical windows are reported in secondary sources and are not load-bearing) — and an expired `startHistoryId` returns 404/410 and *requires falling back to a full resync*. This "cursor can silently expire, forcing full resync" pattern is a first-class concern for PDPP's `since`-cursor design. History calls are cheaper than list calls (2 quota units vs. 5), an explicit incentive structure toward using the delta path. ([Method: users.history.list](https://developers.google.com/workspace/gmail/api/reference/rest/v1/users.history/list); [Gmail API pagination and sync — Nylas](https://developer.nylas.com/docs/cookbook/email/gmail-api-pagination-sync/), accessed 2026-07-09)
-
-### Slack API
-
-`conversations.history`/`conversations.replies` use cursor pagination (`response_metadata.next_cursor`) plus an orthogonal time-range mechanism (`oldest`/`latest` timestamps, `has_more` flag) — two pagination axes that can be combined. As of **May 29, 2025**, Slack imposed dramatically stricter rate limits (15 messages/request, 1 request/minute) on `conversations.history`/`conversations.replies` for commercially distributed apps *not* approved for the Slack Marketplace; per the changelog, "existing installations of apps distributed outside the Marketplace (excluding internal customer-built apps) will not be impacted," and internal/first-party apps retain the older Tier-3-class limits (1,000 msgs/request, 50+ req/min). Slack's stated rationale (verified 2026-07-09 against the changelog) is explicitly anti-bulk-exfiltration: "To help keep workspace data secure and prevent bulk data exfiltration by unvetted applications, these methods will have a new rate limit of 15 messages per request at one request per minute," noting these methods "have the potential to exfiltrate large amounts of sensitive conversational data" — a live instance of a source provider throttling *precisely* the sync-then-query pattern PDPP's current architecture uses, and implicitly favoring live/on-demand query patterns instead. ([Rate limit changes for non-Marketplace apps — Slack](https://docs.slack.dev/changelog/2025/05/29/rate-limit-changes-for-non-marketplace-apps/); [conversations.history — Slack Developer Docs](https://docs.slack.dev/reference/methods/conversations.history/), accessed 2026-07-09)
-
-Slack's rate-limit FAQ also describes a **Real-Time Search API** program (limited availability at research time; timeline claims not independently verified) framed as enabling real-time querying of Slack data without storing it — i.e., Slack itself is building toward encouraging the source-backed-fulfillment pattern over bulk sync, for privacy/risk reasons. `search.messages` has no documented stable-ordering guarantee and coarse filter operators relative to `conversations.history`'s precise channel/thread addressing. ([Rate limit changes FAQ — Slack](https://api.slack.com/changelog/2025-05-terms-rate-limit-update-and-faq), accessed 2026-07-09)
-
-### Microsoft Graph
-
-Delta queries (`/users/delta`, `/groups/delta`, `/messages/delta`, etc.) support incremental sync per-resource-type, but `$filter` in delta mode is severely restricted — for users/groups/devices, "the only supported `$filter` expression is... by object ID," confirmed by a real developer report of an `endswith(mail,...)` filter returning an "Unsupported Query" error. There is no cross-resource-type delta query — each resource type requires its own separate endpoint and its own delta token; there's no unified "everything that changed" feed. This is a sharp instance of *capability heterogeneity even within one provider's own API family* — Outlook messages/calendar delta support differs from Entra ID user/group delta support differs from OneDrive drive-item delta support, all under the same "Microsoft Graph delta query" umbrella term. ([Use delta query to track changes — Microsoft Graph](https://learn.microsoft.com/en-us/graph/delta-query-overview); [Does user delta query support filter query parameters? — Microsoft Q&A](https://learn.microsoft.com/en-us/answers/questions/1090594/does-user-delta-query-support-filter-query-paramet), accessed 2026-07-09)
-
-### What transfers
-
-- These three APIs are close to a worked example of exactly the heterogeneity PDPP must disclose: Gmail has rich search but no sort and a fragile change-cursor; Slack has cursor+time-range pagination but a coarse/unordered search and a hostile, shifting rate-limit regime that specifically penalizes bulk historical fetch; Graph has per-resource delta but almost no filter expressiveness within it. A PDPP capability-disclosure schema needs at minimum: {searchable: bool + operator set}, {sortable: bool + which fields}, {changeCursor: bool + expiry/resync behavior}, {rateLimitClass}. All three exist in the wild as *independent* axes that do not correlate provider-to-provider.
-- Slack's 2025 rate-limit policy change is a *constrained* portability-market signal, not a general rule: it applies to newly created/new-install commercially distributed non-Marketplace apps, while internal customer-built apps explicitly retain higher limits (and session-token paths are outside the app rate-limit regime entirely). Within that scope it shows one major source provider discouraging bulk historical fetch by unvetted apps for stated data-protection reasons — directionally friendly to narrowly-scoped on-demand query, but it should not be cited as "sources broadly penalize sync-and-store."
-- Gmail's cursor-expiry-forces-full-resync pattern, and history.list's cheaper quota cost, are concrete precedents for PDPP's `since`-cursor design: cursors must have documented validity windows and a well-defined fallback (typed error → client must request full sync) rather than silently returning wrong results.
-
-### What fails to transfer
-
-- None of these three APIs are grant-scoped in the RAR sense — a Gmail/Slack/Graph OAuth scope is coarse (e.g., `gmail.readonly`) and does not express PDPP's field-projection/filter/pagination-shape constraints at all; the *source* has no concept of "this specific request must only return fields X/Y and rows matching Z because the grant says so." All grant enforcement is necessarily done by the PDPP personal server *after* or *around* the source call, never *by* the source — meaning fetch-and-filter's over-fetch/disclosure problem is essentially unavoidable for any provider whose native filters don't precisely match the grant's filters, for every one of these three APIs, all of the time.
-- Rate limits here are provider-level and often per-app/per-workspace, not per-end-user-grant; a PDPP server serving many grants against the same connected Gmail account must ration a single shared rate-limit budget across arbitrarily many concurrent grant-scoped requests, a multiplexing problem none of these APIs' own docs address (they assume one app = one rate-limit consumer, not many downstream grants sharing one upstream credential).
-
----
-
-## 5. Zero-ETL / federated analytics
-
-### How it works
-
-**AWS Athena Federated Query** runs source-specific Lambda "Data Source Connectors" (paired metadata + record-reading functions) fanned out in parallel per query; results reflect current source state at query time. AWS's own connector SDK acknowledges partial pushdown as the norm: "the DynamoDB connector can push down simple predicates and key conditions... but complex expressions might not push down, causing more data to be fetched than necessary." AWS's own recommended mitigation for repeated-access patterns is explicit hybridization: "materialize federated data into S3 for faster repeated access... use the live federated query only when you need real-time data" — i.e., AWS's own guidance is that federated (live) query and cached/materialized query are not competing architectures but complementary policy choices per access pattern, not per system. Large per-query responses that exceed Lambda's 6MB limit are transparently spilled to S3 and encrypted — a transient, per-query mechanism that doesn't affect freshness semantics. As of April 2026, AWS has begun offering "managed connectors" for common sources (Postgres, MySQL, Snowflake, DynamoDB) that remove the need to deploy/maintain the Lambda yourself, while preserving the same live-query freshness model. ([Query any data source with Athena's federated query — AWS Big Data Blog](https://aws.amazon.com/blogs/big-data/query-any-data-source-with-amazon-athenas-new-federated-query/); [Use Athena Federated Query — AWS docs](https://docs.aws.amazon.com/athena/latest/ug/federated-queries.html); [Amazon Athena simplifies federated queries with managed connectors — AWS What's New](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-athena/), accessed 2026-07-09)
-
-**BigQuery** has two genuinely distinct federation mechanisms often conflated: `EXTERNAL_QUERY()` federated queries (push a query, in the source's own SQL dialect, to Cloud SQL/AlloyDB/Spanner, get a temp-table result back) vs. **external tables** (query Bigtable/Spanner/Drive/Salesforce Data Cloud data as if native, no dialect translation needed) vs. **BigQuery Omni** (a genuinely separate multi-cloud product querying S3/Azure Blob via a Google-managed Dremel engine running inside AWS/Azure). Federated-query pushdown is documented as narrow and honest about its limits: "only applied to federated queries of the form `SELECT * FROM T`, with only column pruning and filter pushdowns supported; compute, join, limit, order by, and aggregation pushdowns aren't supported" — and there's a sharp, easy-to-miss correctness gap: `EXTERNAL_QUERY()` "won't honor the ordering of the external query result, even if your external query includes ORDER BY" — i.e., a sort *silently* dropped by the pushdown/wrapping layer, exactly the kind of silent capability failure PDPP must avoid. ([Introduction to federated queries — BigQuery docs](https://docs.cloud.google.com/bigquery/docs/federated-queries-intro); [Federated query functions — BigQuery docs](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/federated_query_functions), accessed 2026-07-09)
-
-**Snowflake** external/Iceberg tables use polling-based metadata refresh (default 30s interval, configurable via `REFRESH_INTERVAL_SECONDS`) rather than a push/webhook model for externally-managed Iceberg catalogs, with explicit freshness introspection via `SYSTEM$AUTO_REFRESH_STATUS` and `ICEBERG_TABLE_SNAPSHOT_REFRESH_HISTORY` — freshness is a first-class, queryable property of the table, not an undocumented implementation detail. Ownership transfer silently disables auto-refresh, a sharp footgun precedent (a metadata/permission change silently degrading a freshness guarantee). ([Automatically refresh Apache Iceberg tables — Snowflake docs](https://docs.snowflake.com/en/user-guide/tables-iceberg-auto-refresh); [ALTER ICEBERG TABLE REFRESH — Snowflake docs](https://docs.snowflake.com/en/sql-reference/sql/alter-iceberg-table-refresh), accessed 2026-07-09)
-
-**Salesforce Data Cloud "Zero Copy"** cleanly names the two modes PDPP is choosing between: *Query Federation* ("sends a query to the remote data source, where the external system computes the query, applies its own governance, and returns a result set" — i.e., pushdown-style live query) vs. *File Federation* (direct Iceberg/Parquet file access via REST catalog, bypassing the remote system's query engine entirely). Salesforce's own cost disclosure is a striking, quantified answer to "is live query worth it": federated querying costs "70 credits per million records vs... 2,000 credits per million records for batch data pipelines" — a 28x cost difference favoring live query at their observed usage pattern, with the caveat that this is Salesforce's specific credit-pricing model, not a universal truth. ([Data Cloud – Zero Copy Connectivity — Salesforce](https://www.salesforce.com/data/connectivity/zero-copy/); [Zero Copy Revolutionizes Data Cloud — Salesforce Engineering Blog](https://engineering.salesforce.com/zero-copy-revolutionizes-data-cloud-real-time-analysis-without-data-duplication-or-migration/), accessed 2026-07-09)
-
-### What transfers
-
-- The "materialize for repeated access, live-query only when freshness matters" hybrid stance (AWS's own explicit guidance) is probably the correct default policy framing for PDPP too: source-backed fulfillment and locally-collected fulfillment are not an either/or architectural choice per source, but a per-*request* or per-*field* policy decision, potentially informed by grant metadata (does this grant need freshness guarantees?) and access pattern (repeated identical query vs. one-off).
-- BigQuery's silently-dropped `ORDER BY` on federated queries is a concrete cautionary tale: PDPP must never let a sort/filter clause silently vanish between "client asked for it" and "source couldn't do it" — every request must return an explicit, structured disclosure of what was and wasn't honored (echoing Trino's typed refusal pattern).
-- Snowflake's queryable freshness-status functions (`SYSTEM$AUTO_REFRESH_STATUS`, snapshot refresh history) are a good precedent for a PDPP grant-scoped read response including an explicit, machine-readable freshness/staleness disclosure alongside the data.
-
-### What fails to transfer
-
-- All four systems (Athena, BigQuery, Snowflake, Salesforce) operate under a single organizational security/governance boundary controlling both query origin and query destination — "the external system applies its own governance" (Salesforce's own description of Query Federation) assumes that governance is a *second, independent, trusted* layer, not the *only* enforcement layer as it is for PDPP (the personal server IS the sole grant-enforcement point; the source's own access controls are typically all-or-nothing per credential, not grant-shaped).
-- None of these systems have a "one human, coarse credential, many differently-scoped downstream consumers" fan-out problem — they're built for "one analytics team, many source systems," which is the *inverse* shape of PDPP's "one source credential, many grant-scoped agent/client consumers."
-
----
-
-## 6. Personal-data authorization & delegated-access systems
-
-### Solid pods
-
-Solid's core protocol is LDP (Linked Data Platform) — resources are RDF/non-RDF documents fetched whole via standard HTTP GET, discoverable via `Link` headers. SPARQL query support is explicitly **optional**, layered on top, not part of the core protocol — and the community's own stated reason for wanting it is a direct echo of PDPP's motivating problem: "with LDP you cannot retrieve just part of a document... if users have made thousands of comments, it doesn't make sense to retrieve them all client-side to then filter." I.e., Solid's baseline model is "fetch-and-filter is mandatory because there's no query capability at all," and SPARQL was added specifically to give large pods a pushdown escape hatch — but the community itself flags server-side SPARQL as a live *security* concern ("numerous problems... stemming mainly from the very power and expressiveness of SPARQL") requiring "more lightweight interfaces" as the real first move — precisely the pushdown-vs-attack-surface tension PDPP would face if it ever let clients express arbitrary predicates against a source. Solid's PATCH mechanism reuses SPARQL Update (DELETE/INSERT statements) scoped to a single resource-as-endpoint, deliberately avoiding the graph-naming complexity of a server-wide SPARQL endpoint. ([Introduction to the Solid Data Protocol — Linode Docs](https://www.linode.com/docs/guides/introduction-to-the-solid-data-protocol/); [SOLID Server SPARQL Endpoint — Solid Forum](https://forum.solidproject.org/t/solid-server-sparql-endpoint/2920/9); [Level of SPARQL Update support — Issue #125](https://github.com/solid/specification/issues/125), accessed 2026-07-09)
-
-### UMA 2.0 and GNAP
-
-UMA 2.0 (Kantara Initiative rec., built as an OAuth 2.0 extension) is the closest standards-track analogue to PDPP's grant model: the **resource owner**, not an admin, configures fine-grained authorization policy at the AS, and the **resource server** (source) is the one that "defines the boundaries of resources and the scopes available to each resource... by virtue of being the publisher of the API being protected." The runtime flow is a three-way negotiation: client requests → RS returns 403/permission-ticket → client exchanges ticket for a scoped RPT (Requesting Party Token) at the AS → client retries with RPT. UMA explicitly names data aggregation as a use case in its own canonical example ("financial information aggregation company Decide Account" as one of several parties Alice, the resource owner, can grant differently-scoped access to over her bank accounts) — this is architecturally the closest published standard to "personal server enforces per-grant reads against a resource server it doesn't operate." The companion **Federated Authorization** extension explicitly supports "multiple resource servers operating in different domains" reporting to one AS — relevant if PDPP's personal server plays an AS-like role across several independently-operated upstream source APIs. ([User-Managed Access (UMA) 2.0 Grant — Kantara](https://docs.kantarainitiative.org/uma/wg/rec-oauth-uma-grant-2.0.html); [Federated Authorization for UMA 2.0 — Kantara](https://docs.kantarainitiative.org/uma/wg/rec-oauth-uma-federated-authz-2.0.html), accessed 2026-07-09)
-
-GNAP (RFC 9635) is a more general redesign of the delegation layer — clients need not be pre-registered, the grant request is a rich negotiable structure, and a continuation API lets the negotiation evolve across interactions, decoupling delegation from the browser-redirect assumption OAuth 2.0 hardcodes. Its main relevance to PDPP is architectural inspiration (dynamic, richer negotiation of what is being granted) rather than a query-capability model — GNAP, like OAuth/UMA, defines *how permission is granted*, not *how a resource server discloses what kinds of reads it can satisfy*; the companion RFC 9767 covers resource-server-facing concerns (introspection, resource registration) and likewise defines no query-capability disclosure. ([RFC 9635: Grant Negotiation and Authorization Protocol (GNAP) — IETF](https://www.rfc-editor.org/rfc/rfc9635); [RFC 9767: GNAP — Resource Server Connections — IETF](https://www.rfc-editor.org/rfc/rfc9767), accessed 2026-07-09)
-
-### Plaid / FDX: cache-and-store vs. pass-through
-
-This is a genuinely contested, currently-shifting industry design question, not settled prior art. Plaid's own consumer disclosures confirm it **does** store account-holder info, transactions, and balances on its servers so that later retrieval does not re-query the bank — it is explicitly *not* a pure live pass-through, though bank *login credentials* specifically are tokenized and not retained in plaintext post-connection. FDX (the nonprofit standard-setting body Plaid/MX and others implement) defines the *data-sharing protocol* (OAuth-based, consumer-permissioned) but is agnostic to whether an aggregator caches or passes through — that's an implementation choice each aggregator makes. There is a live, named industry critique of the cache-and-store default: "legacy aggregators achieve [ease-of-integration] by silently caching customers' general ledgers, bank feeds, and PII on their own servers — creating a massive, unnecessary attack surface," with an explicit architectural vocabulary contrast: "cache-and-serve architectures (useful for offline analytics or when upstream APIs have brutal rate limits) versus pass-through architectures... for financial data specifically, security calculus strongly favors pass-through." This is effectively the financial-data industry independently re-deriving PDPP's own source-backed-fulfillment thesis, for the same reason (reducing the blast radius of a stored copy). ([What is FDX? — Stripe](https://stripe.com/resources/more/what-is-the-financial-data-exchange-fdx-here-is-what-you-should-know); [Secure Unified APIs for Financial Data — Truto Blog](https://truto.one/blog/the-vendor-neutral-guide-to-secure-unified-apis-for-financial-data/); [What data does Plaid access — Plaid Help Center](https://support-my.plaid.com/hc/en-us/articles/4410324477847-What-data-does-Plaid-access-from-my-financial-institution), accessed 2026-07-09)
-
-### What transfers
-
-- UMA 2.0's resource-server-defines-scopes-per-resource model, plus its explicit three-party (owner/requesting-party/resource-server) vocabulary, is the strongest available standards-track precedent for PDPP's grant-enforcement architecture — worth citing directly if PDPP ever needs to justify its model against "why not just use an existing standard."
-- The pass-through-vs-cache-and-store framing from the FDX/fintech-aggregator debate is directly reusable language: PDPP's source-backed fulfillment is the security-motivated "pass-through" architecture the fintech industry is independently converging toward for the same reasons (reduce attack surface of a stored copy of someone else's data).
-- Solid's own admission that server-side SPARQL (i.e., letting a client push an arbitrary predicate into a personal data store) is a live security concern requiring "more lightweight interfaces" is a direct caution against over-generalizing PDPP's filter language to arbitrary source-native query pass-through.
-
-### What fails to transfer
-
-- Solid pods are *self-hosted by the resource owner*, so there's no separate "source" to fetch from at all — the pod *is* the source, collapsing exactly the distinction (collector vs. fulfiller) that PDPP's source-backed-fulfillment design is trying to introduce. Solid's LDP-only-whole-document-fetch limitation is a cautionary tale about what happens *without* a query capability layer at all, not a model for building one.
-- UMA/GNAP standardize the *grant/negotiation* layer thoroughly but say essentially nothing about *query capability disclosure* — neither protocol specifies how a resource server should tell a client "I can filter by X but not sort by Y." That gap is exactly the space PDPP (and Steampipe/OData, per below) are filling; UMA/GNAP are necessary-but-not-sufficient prior art.
-- The Plaid/FDX debate is about aggregator *architecture choice*, not a technical *protocol* — there's no citable capability-negotiation API in the FDX spec analogous to Trino's `applyFilter` or OData's Capabilities vocabulary; the "pass-through" side of that industry debate currently amounts to a security *posture*, not a documented interface contract PDPP could adopt wholesale.
-
----
-
-## 7. OData Capabilities vocabulary
-
-### How it works
-
-OData's `Org.OData.Capabilities.V1` vocabulary, maintained by the OASIS OData Technical Committee, exists for exactly PDPP's stated problem: "there are some capabilities that a service may choose to support or not support and in varying degrees. `$filter` and `$orderby` are such good examples... this vocabulary aims to define terms to specify support or no support for such capabilities." It is a machine-readable, per-entity-set CSDL annotation schema, not free-text documentation. `FilterRestrictionsType` declares `Filterable` (bool), `RequiredProperties` (must appear in every `$filter`), and `NonFilterableProperties` (blocklist) — directly modeling Steampipe's `KeyColumns`-required/optional split, but as a *standardized, declarative, machine-parseable* vocabulary rather than a plugin-SDK-specific Go struct. `SortRestrictionsType` mirrors this with `Sortable`, `AscendingOnlyProperties`, `DescendingOnlyProperties`, and `UnsortableProperties`. Defaults matter here too: some capabilities (e.g., `$top`/`$skip`) are "strongly recommended... assumed default even without an annotation," and the annotation exists mainly to *declare non-support* or explicitly confirm support — an inversion of "document everything," instead "assume support, annotate exceptions," which keeps the common case terse. ([Introducing a Capabilities Vocabulary — OData.org](https://www.odata.org/blog/introducing-a-capabilities-vocabulary/); [Org.OData.Capabilities.V1 — OASIS TC GitHub](https://github.com/oasis-tcs/odata-vocabularies/blob/main/vocabularies/Org.OData.Capabilities.V1.md), accessed 2026-07-09)
-
-This is, of the entire survey, **the closest existing prior art to a machine-readable, per-collection query-capability disclosure format** — the exact shape ("this stream supports filtering by fields {A, B} with operators {eq, gt}, sorting by {C} ascending-only, search unsupported") that PDPP's own stream schema needs, expressed in a standards-body-governed vocabulary rather than an ad hoc convention.
-
-### What transfers
-
-- The vocabulary's structure — per-collection (PDPP: per-stream), per-operation-type (filter/sort/search/top-skip/expand each get their own `*Restrictions` term), required-vs-optional-vs-forbidden property lists — is close to directly adoptable as the shape of PDPP's own capability-disclosure schema in `schema(stream)` responses, whether or not PDPP literally reuses OData's CSDL/XML serialization.
-- The "assume support, annotate the exceptions" default-posture design decision is worth adopting for terseness in PDPP's own schema responses, provided the *default* assumption is safe (i.e., default should probably be "not supported" for PDPP given the disclosure stakes are higher than OData's typical enterprise-service context — this is a place where PDPP's defaults should diverge from OData's).
-- OData's clean separation of `RequiredProperties` (must be present) from `NonFilterableProperties` (must be absent) from the general filterable set gives PDPP good vocabulary for expressing Gmail/Slack/Graph-style "you must supply X to even list" constraints (echoing Steampipe's required KeyColumns) inside the same schema shape used for ordinary optional filters.
-
-### What fails to transfer
-
-- OData's Capabilities vocabulary describes *static, source-declared* capability (set once by the service author, rarely changing) — it has no notion of *dynamic, per-connection* capability (the same "stream" — e.g., Gmail messages — might have different effective capability depending on which OAuth scope was granted, or which Workspace-vs-consumer Gmail edition is connected). PDPP's capability disclosure likely needs to be connection-instance-scoped, not just stream-type-scoped, which OData's service-level annotation model doesn't anticipate.
-- OData assumes the *service itself* controls and can attest to its own capabilities (a company publishing an OData API knows exactly what its own backend supports). PDPP's personal server is a *third party* wrapping a source it doesn't control (Gmail) — its capability declarations for "Gmail as a PDPP stream" are themselves a reverse-engineered/documented-by-Google artifact, not a first-party attestation, meaning PDPP capability manifests need a versioning/drift story that OData's model doesn't need (Google can silently change what `q=` supports; PDPP's declared capabilities can go stale).
-
----
-
-## Synthesis
-
-### (a) Recurring invariants across all systems surveyed
-
-1. **Capability disclosure must precede — and be separable from — query execution.** Every mature system in this survey (Trino's `applyFilter` capability API, Steampipe's `KeyColumns`, OData's Capabilities vocabulary, Apollo Connectors' exhaustive field mapping, Athena's per-connector predicate support) treats "what can this source do" as a first-class artifact queryable/declarable independently of any single request — never inferred implicitly by trial-and-error at request time. PDPP's `schema(stream)` mechanism is well-positioned to carry this, but needs the *operation-type-granular* structure (filter vs. sort vs. search vs. cursor, each independently declared) that Trino/OData/Steampipe all converge on, rather than a single coarse "supports queries" flag.
-
-2. **"Pushdown-or-fail" vs. "pushdown-or-compensate" is a real, consequential design fork, and different systems choose differently by default.** Trino/FDW/DuckDB default to compensate (silently fetch more, filter locally) with an opt-in "throw instead" escape hatch for expensive cases (Top-N). Steampipe defaults to fail loudly (missing required qual = hard error, not slow full scan). BigQuery's federated-query `ORDER BY` silently vanishing is the cautionary failure mode: a *compensate* default that fails to actually compensate and instead silently drops the requirement. **For PDPP, the disclosure stakes of silent fetch-and-filter (an over-fetch is a privacy event, not just a perf cost) argue for defaulting toward Steampipe's "fail loudly, typed, visible-to-caller" posture** rather than Trino's "silently do it locally" posture — with a compensate mode available only where the personal server can *prove* the over-fetch stays within acceptable disclosure bounds (e.g., filtering already-fetched-for-a-broader-reason data server-side, never issuing a broader upstream request than the grant needs).
-
-3. **Identity mapping is always the unglamorous hard part.** Every system needs a stable way to say "this result row IS that entity" across repeated queries/pages, and it's solved differently everywhere: Trino/FDW rely on the source's own primary key; Gmail uses immutable message IDs plus a separate historyId *cursor* concept (identity ≠ change-tracking token); GraphQL Federation's `@key` directive is exactly this same problem restated for a graph; UMA's resource server "publishes" resource identity to the AS. **PDPP's "stable record identity without local storage" problem has no shortcut in any of this prior art — every source must supply its own stable ID, and PDPP's cursor/pagination design must not conflate "identity of a record" with "position in a result stream" (Gmail gets this right by keeping message IDs and historyId separate; BigQuery's dropped-ORDER BY shows what happens when a system conflates "the query looks stable" with "the result actually is stable").**
-
-4. **Caching is a spectrum, not a binary, and the spectrum's dimensions are: TTL, keying granularity (quals-aware or not), and staleness disclosure.** Steampipe's two-tier cache (quals-aware result cache + independently-keyed hydrate memoization), Snowflake's queryable freshness/refresh-status functions, and AWS's own explicit "materialize for repeated access, live-query for freshness" guidance all point the same direction: **the honest design is not "cache or don't," it's "disclose the staleness bound of every response, and let the freshness requirement (which may vary per grant, per field, or per request) pick a point on the spectrum."** PDPP's freshness/consistency disclosure problem should be modeled as a response *metadata* field (a staleness bound, analogous to Snowflake's refresh-status), not a binary architectural choice made once per source at connector-design time.
-
-5. **No surveyed system has PDPP's exact ownership topology: one credential, one owner, many independently-scoped downstream consumers, each entitled to different capability views of the same source.** Every federated-query/ELT system assumes an operator-analyst relationship to the data (Trino, Steampipe, CloudQuery, Athena, BigQuery, Salesforce). Every delegated-auth standard (UMA, GNAP) defines *how* a grant is negotiated but not how a resource server should *describe its own query capability* to the grantee. This is the actual gap PDPP is filling — not a reinvention, but a genuine synthesis of two previously-separate prior-art lineages (query-capability-disclosure systems + delegated-authorization systems) that have not, per this research, been combined by any surveyed system.
-
-### (b) Naming survey
-
-What these systems call "satisfy a query against a live upstream source instead of local storage":
-
-| Term | Used by | Connotation |
-|---|---|---|
-| **Federation** / **federated query** | Trino/Presto, Athena, BigQuery, Data Cloud "Query Federation" | Emphasizes *many sources, one query surface*; industry-standard, well-understood, but doesn't foreground the "no local copy" aspect (a federated query engine *can* also query cached/materialized local tables). |
-| **Foreign data wrapper (FDW)** / **external table** | PostgreSQL, Snowflake, BigQuery, DuckDB scanners | Emphasizes *schema virtualization* of a remote thing as a local-looking table; strong precedent for "the source's native shape is hidden behind PDPP's stream schema," but "external table" wrongly implies a persistent catalog entry rather than an ephemeral per-request fetch. |
-| **Zero-copy** | Salesforce Data Cloud, Snowflake/Databricks sharing | Emphasizes the *absence of a stored duplicate* — closest in spirit to PDPP's actual privacy motivation (no local copy = smaller disclosure surface), but it's become a Salesforce/Snowflake marketing term specific to warehouse-to-warehouse Iceberg/Delta sharing, not API-backed live query; would likely read as borrowed jargon outside that context. |
-| **Live query** | Steampipe (implicitly, via contrast with CloudQuery), Slack's "Real-Time Search API" | Plain, accurate, low-jargon — but generic enough to not clearly signal grant-enforcement is layered on top. |
-| **Virtualization** / **data virtualization** | Generic industry term (Denodo and similar vendors, not directly surveyed here but implied by the FDW/Trino lineage) | Accurately captures "queries against a virtual, non-materialized view of remote data," but is heavily associated with enterprise-BI middleware vendors — carries connotations unrelated to personal-data/consent.|
-| **Pass-through** | Fintech/FDX aggregator debate (Plaid-critique framing), and — separately, with an *opposite, negative* connotation — MCP/OAuth security literature | **Collision, confirmed**: MCP's authorization spec explicitly names "token passthrough" as a forbidden anti-pattern — "an MCP server accepts tokens from an MCP client without validating that the tokens were properly issued to the MCP server and passes them through to the downstream API," creating confused-deputy risk ([Security Best Practices — Model Context Protocol](https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices), accessed 2026-07-09). PDPP's design is the *opposite* of that anti-pattern (the personal server holds its own strong credential and enforces the grant itself — it never forwards a client's token upstream), but reusing "pass-through" as PDPP's feature name risks being misread as *doing* the forbidden thing, in exactly the audience (MCP-literate engineers/agents) most likely to notice and object. **Recommend avoiding "pass-through" in the feature name for this reason alone**, even though it's linguistically the most literal description of the mechanism. |
-| **Delegated query** | Not a standard term found in this research, but a natural extension of UMA/GNAP's "delegated access" vocabulary | Would tie PDPP's naming to the UMA/GNAP lineage explicitly, foregrounding that this is authorization-aware querying, not just a performance/architecture choice — but "delegated query" isn't an established term anywhere surveyed, so it would need to be introduced/defined rather than recognized. |
-
-### (c) Candidate names for the PDPP feature
-
-1. **"Source-backed fulfillment"** (the working name already in use) — *Pros*: already adopted internally, accurately neutral, avoids the "pass-through" collision entirely, "fulfillment" ties cleanly to PDPP's existing "grant-scoped read" vocabulary (a read request is *fulfilled* either from local storage or from the source). *Cons*: "backed" is slightly vague about mechanism (backed *by* vs. backed *from*); doesn't by itself signal "live" to a newcomer.
-2. **"Live fulfillment"** / **"live-source fulfillment"** — *Pros*: shortest, plainest-English option; directly contrasts with the existing collect-then-serve mode without needing a new noun; echoes Steampipe/Slack's own "live query" framing, which is well-understood industry shorthand. *Cons*: "live" slightly overclaims real-time-ness in cases where PDPP still applies a short cache TTL (per the caching-is-a-spectrum invariant above) — could mislead callers into assuming zero staleness.
-3. **"Grant-scoped federation"** / **"federated fulfillment"** — *Pros*: "federation" is the most industry-recognized umbrella term (Trino, Athena, BigQuery, GraphQL all use it), lowering the explanation burden for engineers already familiar with federated query engines; "grant-scoped" prefix makes explicit what's different about PDPP's version versus generic federation (the enforcement layer). *Cons*: "federation" carries a connotation of *many sources combined into one query* (a join across sources), which is not what PDPP is doing (each grant-scoped read still targets one source/stream at a time) — could overpromise cross-source query capability PDPP doesn't (yet) offer.
-4. **"On-demand fulfillment"** — *Pros*: cleanly contrasts with "collected/stored fulfillment," avoids "live"'s freshness overclaim, avoids "federation"'s multi-source connotation, avoids "pass-through"'s MCP collision; "on-demand" is generic-enough English to not require industry jargon literacy to understand. *Cons*: weakest tie to existing prior-art vocabulary — doesn't signal to an engineer who already knows Trino/Steampipe/FDX that this is the same well-studied tradeoff, losing some "we're standing on established ground" credibility.
-5. **"Direct fulfillment"** / **"direct-source fulfillment"** — *Pros*: short, emphasizes the absence of an intermediate stored copy (echoing the "zero-copy" motivation without importing that term's warehouse-sharing baggage), reads naturally alongside "collected fulfillment" as the paired term for the existing mode. *Cons*: "direct" is used loosely elsewhere in API contexts (e.g., "Direct API" in the FDX ecosystem, per the Plaid/MX research above, to mean bank-direct vs. aggregator-mediated) — a minor but real term collision risk within the exact adjacent domain (financial-data aggregation) PDPP's design most resembles.
-
-**Recommendation for the design owner to weigh**: given the confirmed MCP "token passthrough" collision, avoid "pass-through" outright. Of the above, "source-backed fulfillment" (current) and "on-demand fulfillment" both cleanly sidestep every naming collision found in this research; "federated fulfillment" is the most industry-legible but risks overpromising cross-source joins. If a single-word qualifier is wanted for casual reference (e.g., in schema output: `"fulfillment": "live"` vs `"fulfillment": "collected"`), "live" is short and matches Steampipe/Slack precedent, with the caveat above about not overclaiming zero-staleness — pairing it with an explicit staleness-bound field in the response (per synthesis point 4) would resolve that ambiguity structurally rather than terminologically.
-
----
-
-## Sources index (all accessed 2026-07-09)
-
-- Trino: [Connectors](https://trino.io/docs/current/develop/connectors.html), [Pushdown](https://trino.io/docs/current/optimizer/pushdown.html), [ConnectorMetadata.java](https://github.com/trinodb/trino/blob/master/core/trino-spi/src/main/java/io/trino/spi/connector/ConnectorMetadata.java), [Prometheus pushdown issue #8742](https://github.com/trinodb/trino/issues/8742)
-- postgres_fdw: [PG18 docs](https://www.postgresql.org/docs/current/postgres-fdw.html), [freeCodeCamp pushdown explainer](https://www.freecodecamp.org/news/fdw-pushdown/)
-- DuckDB: [Postgres scanner blog](https://duckdb.org/2022/09/30/postgres-scanner), [Postgres extension docs](https://duckdb.org/docs/current/core_extensions/postgres/overview)
-- Steampipe: [Writing Plugins](https://steampipe.io/docs/develop/writing-plugins), [Key Column Qualifiers](https://steampipe.io/docs/guides/key-columns), [Release 0.3.0 blog](https://steampipe.io/blog/release-0-3-0)
-- CloudQuery: [Steampipe vs CloudQuery](https://www.cloudquery.io/blog/steampipe-vs-cloudquery)
-- Apollo Federation: [Resolve another subgraph's fields](https://www.apollographql.com/docs/graphos/schema-design/federated-schemas/entities/resolve-another-subgraphs-fields), [Subgraph Spec](https://www.apollographql.com/docs/graphos/schema-design/federated-schemas/reference/subgraph-spec), [`@requires` bug #2801](https://github.com/apollographql/federation/issues/2801)
-- Apollo Connectors: [Mapping Language Overview](https://www.apollographql.com/docs/graphos/connectors/mapping), [Why use Connectors?](https://www.apollographql.com/docs/graphos/connectors/why-connectors)
-- Gmail API: [Search and filter messages](https://developers.google.com/workspace/gmail/api/guides/filtering), [List messages](https://developers.google.com/workspace/gmail/api/guides/list-messages), [users.messages.list](https://developers.google.com/workspace/gmail/api/reference/rest/v1/users.messages/list), [users.history.list](https://developers.google.com/workspace/gmail/api/reference/rest/v1/users.history/list), [Nylas pagination/sync guide](https://developer.nylas.com/docs/cookbook/email/gmail-api-pagination-sync/)
-- Slack API: [Rate limit changes for non-Marketplace apps](https://docs.slack.dev/changelog/2025/05/29/rate-limit-changes-for-non-marketplace-apps/), [Rate limit FAQ](https://api.slack.com/changelog/2025-05-terms-rate-limit-update-and-faq), [conversations.history](https://docs.slack.dev/reference/methods/conversations.history/)
-- Microsoft Graph: [Delta query overview](https://learn.microsoft.com/en-us/graph/delta-query-overview), [Filter support Q&A](https://learn.microsoft.com/en-us/answers/questions/1090594/does-user-delta-query-support-filter-query-paramet)
-- Athena: [Federated query blog](https://aws.amazon.com/blogs/big-data/query-any-data-source-with-amazon-athenas-new-federated-query/), [Use federated query docs](https://docs.aws.amazon.com/athena/latest/ug/federated-queries.html), [Managed connectors 2026](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-athena/)
-- BigQuery: [Federated queries intro](https://docs.cloud.google.com/bigquery/docs/federated-queries-intro), [Federated query functions](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/federated_query_functions)
-- Snowflake: [Auto-refresh Iceberg tables](https://docs.snowflake.com/en/user-guide/tables-iceberg-auto-refresh), [ALTER ICEBERG TABLE REFRESH](https://docs.snowflake.com/en/sql-reference/sql/alter-iceberg-table-refresh)
-- Salesforce Data Cloud: [Zero Copy Connectivity](https://www.salesforce.com/data/connectivity/zero-copy/), [Zero Copy engineering blog](https://engineering.salesforce.com/zero-copy-revolutionizes-data-cloud-real-time-analysis-without-data-duplication-or-migration/)
-- Solid: [Intro to Solid Data Protocol](https://www.linode.com/docs/guides/introduction-to-the-solid-data-protocol/), [SPARQL endpoint forum thread](https://forum.solidproject.org/t/solid-server-sparql-endpoint/2920/9), [SPARQL Update support issue #125](https://github.com/solid/specification/issues/125)
-- UMA 2.0: [UMA 2.0 Grant spec — Kantara](https://docs.kantarainitiative.org/uma/wg/rec-oauth-uma-grant-2.0.html), [Federated Authorization for UMA 2.0](https://docs.kantarainitiative.org/uma/wg/rec-oauth-uma-federated-authz-2.0.html)
-- GNAP: [RFC 9635 — GNAP](https://www.rfc-editor.org/rfc/rfc9635), [RFC 9767 — GNAP Resource Server Connections](https://www.rfc-editor.org/rfc/rfc9767)
-- Plaid/FDX: [What is FDX? — Stripe](https://stripe.com/resources/more/what-is-the-financial-data-exchange-fdx-here-is-what-you-should-know), [Secure Unified APIs for Financial Data — Truto](https://truto.one/blog/the-vendor-neutral-guide-to-secure-unified-apis-for-financial-data/), [What data does Plaid access — Plaid Help Center](https://support-my.plaid.com/hc/en-us/articles/4410324477847-What-data-does-Plaid-access-from-my-financial-institution)
-- OData Capabilities vocabulary: [Introducing a Capabilities Vocabulary](https://www.odata.org/blog/introducing-a-capabilities-vocabulary/), [Org.OData.Capabilities.V1 — OASIS TC](https://github.com/oasis-tcs/odata-vocabularies/blob/main/vocabularies/Org.OData.Capabilities.V1.md)
-- MCP security: [Security Best Practices — Model Context Protocol](https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices)
-- MCP tool discovery: [MCP Tool Discovery — Obot AI](https://obot.ai/resources/learning-center/mcp-tool-discovery/)