[docs] Plan gateway triggers: research, proposal, and WP/WL/WS breakdown

AGENTS.md

@@ -40,11 +40,69 @@ If so, use the `but` CLI instead of raw `git branch`/`git commit`:
 - `but pr new` needs interactive forge auth; use `but push <branch>` then
   `gh pr create --head <branch> --base <parent-or-main>` instead. For stacked PRs,
   set `--base` to the parent branch so each PR shows only its own diff.
+- **`but push` prints NOTHING on success.** It is not a confirmation — always verify
+  the push landed by comparing SHAs:
+  `git ls-remote --heads origin <branch>` vs `git rev-parse <branch>`. They must match.
 - To update an already-committed file, `but absorb <path>` amends it into the right
   commit; force-push with `but push <branch> -f`.
-- To commit to a specific branch in a stack, stage the files to it first
-  (`but rub <path> <branch>`), then `but commit <branch> --only`. `but commit`
-  alone sweeps ALL uncommitted changes into that branch.
+
+### Committing to specific lanes in a stack (the part that bites)
+
+Changes are assigned to the **stack**, not to an individual branch. `but rub <file>
+<branch>` and `but commit <branch> --only` both operate on the stack's *assigned-changes*
+set — `--only` commits **whatever is currently assigned** to the named branch, regardless
+of which branch name you used when staging. So:
+
+- **Never pre-stage multiple lanes' files and then commit them one lane at a time.** The
+  first `but commit --only` sweeps the entire assigned set into that one branch (the others
+  end up empty or scrambled). Instead, work **one lane at a time**: assign exactly that
+  lane's files → `but commit <branch> --only` → **verify** → then assign the next lane's
+  files. Keep the assigned set equal to exactly one lane's files at each commit.
+- **Verify every commit immediately:** `git show --stat --name-only <branch>`. If a file
+  from another lane leaked in, stop and fix before continuing.
+- **`but rub` by path goes stale after any mutation.** Every `but` mutation kicks a
+  background sync that invalidates the path index, so the *next* path-based
+  `but rub <path> ...` often fails with "Source '<path>' not found". Use the stable
+  **cliId** instead (the 2-4 char code in `but status` / `but status --json`):
+  `but rub <cliId> <target>`. cliIds survive across the sync; paths don't.
+- **Splitting one file across two stacked lanes** (e.g. `routers.py` where the lower lane
+  owns half the edit and the upper lane the other half): you cannot split mixed hunks
+  reliably. Instead use sequential working-tree states — make the file the lower lane's
+  version, commit it to the lower lane; then edit the file to add the upper lane's delta
+  and `but rub <fileCliId> <upperCommitCliId>` to amend that delta into the upper commit.
+- The **branch ref can diverge from the workspace-applied commit** mid-session (after
+  absorb/amend/rebase). The **working tree is the source of truth**; `but push` pushes the
+  applied state. Don't panic if `git diff <branch> -- <file>` shows a delta while
+  `git status` is clean — verify against `git show "<branch>:<file>"` and re-push.
+
+### Stacks are linear; a fan-out is expressed through PR bases, not graph shape
+
+A GitButler **stack** is a linear series. `but branch new <name> --anchor <parent>` does NOT
+create a sibling of `<parent>` — it **inserts the new branch into the line** on top of it. So
+anchoring two branches on the same parent produces `parent → first → second`, not two children
+of `parent`. `but branch new <name>` with **no** anchor makes a separate parallel stack, but a
+parallel stack branches off the workspace base (main), so a branch that genuinely depends on an
+ancestor's commits can't live there with a clean diff.
+
+This matters when a design's dependency tree fans out (e.g. a web lane and an SDK lane that both
+depend on an API lane but not on each other). You cannot draw that fan-out in the git graph here.
+You don't need to. The clean per-PR diff is a **PR-base** property, not a graph-shape property:
+a stacked branch contains every commit below it, and GitHub shows only the delta against the base
+you set. So put everything in **one linear stack in dependency order** and set each PR's base to
+the branch directly below it. Order independent lanes however you like (sort by fewest conflicts);
+lanes that touch disjoint files (e.g. `web/**` vs `api/**`) can sit anywhere in the line.
+
+- Build the line with `but move <branch> <target-branch>` (stacks `<branch>` on top of `<target>`)
+  and `but move <branch> zz` (tears `<branch>` off into its own parallel stack). Use these to
+  reorder after the fact; take a `but oplog snapshot` first.
+- **Verify the line by diffing, not by eyeballing the tree.** For each branch, run
+  `git diff --name-only <base>..<branch>` where `<base>` is the branch below it. The file list
+  must be exactly that lane's files. If a lower lane's files appear, the order is wrong (a lane got
+  inserted into another's ancestry) — `but move` it out of the way and re-diff.
+- A branch torn off to its own parallel stack (base = main) gives a **wrong** diff against an
+  ancestor branch: `git diff <ancestor>..<torn-off>` reverses the ancestor's own changes (their
+  merge base is main). That's the tell that the branch needs to be stacked, not parallel.
+- Set PR bases to match: bottom lane `--base main`, every other lane `--base <branch-below-it>`.
 
 ### Hard-won gotchas (don't relearn these)
 

docs/designs/gateway-triggers/gap.md

@@ -0,0 +1,140 @@
+# Gateway Triggers — Gap
+
+The delta between **what exists today** and **what the proposal requires**. Every row is
+something that must be built, moved, or decided; the "Source" column names what it is
+patterned on (per `mimics.md`), and "Kind" classifies it:
+
+- **extract** — move shipped code into a shared home (the connection only).
+- **mimic** — replicate an existing pattern in new triggers-domain files.
+- **net-new** — no precedent; needs a design decision before code (per `mimics.md` §
+  Triggers vs Everything).
+- **decision** — an open question to lock before or during build (from proposal § Risks
+  and `mapping.md` § Open questions).
+
+Nothing here changes the outbound `webhooks` domain or the `/tools` HTTP contract — both
+are invariants (proposal § Success criteria).
+
+---
+
+## 1. What exists today (the baseline)
+
+| Capability | Where | Reusable as-is? |
+|---|---|---|
+| Composio **auth** (initiate/status/refresh/revoke) | `ComposioToolsAdapter` (`core/tools/providers/composio/adapter.py`) | Yes — **extract** the auth verbs to the shared connection adapter |
+| Connection persistence | `ToolConnectionDBE` / `tool_connections` (`dbs/postgres/tools/dbes.py:38`) | Yes — **rename** to `gateway_connections` (already domain-neutral) |
+| Connection CRUD + OAuth callback | `ToolsService` (`core/tools/service.py:138-383`), `/tools/connections/...` + `/callback` (`router.py:785`) | Yes — **extract** to shared service; `/tools/connections` contract frozen |
+| Action catalog (providers/integrations/actions) | `core/tools` catalog + `apis/fastapi/tools` | Pattern only — **mimic** for events |
+| Composio call surface (httpx `_get/_post/_delete`, slug mapping) | `ComposioToolsAdapter` | Pattern only — **mimic** for the triggers REST surface |
+| Two-table subscription/delivery model | `webhooks`: `webhook_subscriptions` + `webhook_deliveries` (`core/webhooks/`, `dbs/postgres/webhooks/`) | Pattern only — **mimic** (separate tables, no reuse) |
+| DBA mixins for a subscription/delivery domain | `dbs/postgres/webhooks/dbas.py` | Pattern only — **mimic** (tools has no `dbas.py`) |
+| Payload-mapping template + resolver | `payload_fields` + `resolve_payload_fields` (`core/webhooks/delivery.py:95`) → `resolve_json_selector` (`sdk/utils/resolvers.py:114`) | Resolver **reused** (promote + rename); template **mimicked** as `inputs_fields` |
+| Inbound, signature-verified provider webhook | billing `POST /billing/stripe/events/` (`ee/.../billing/router.py:106,240`) | Pattern only — **mimic** the ingress shape |
+| Workflow dispatch seam | `WorkflowsService.invoke_workflow` (`core/workflows/service.py:1698`) | Reused **as-is** — no new execution path |
+| `env.composio` (api_key/api_url/enabled) | `utils/env.py:507`; wiring `entrypoints/routers.py:578` | Reused; **add** `COMPOSIO_WEBHOOK_SECRET` |
+
+> Tools never persisted a per-use record and webhooks never had a provider connection;
+> **triggers is the first domain that needs both** a connection *and* a per-event standing
+> record — which is why the connection is extracted (shared) and the subscription/delivery
+> pair is mimicked (triggers-owned).
+
+---
+
+## 2. The gap, by domain
+
+### 2.1 Shared `connections` domain (extract — A2-2)
+
+The connection moves out of `/tools` into a routerless shared domain.
+
+| # | Item | Kind | Source / note |
+|---|---|---|---|
+| C1 | `gateway_connections` table — rename `tool_connections` (+ `uq_`/`ix_`), no data transform | extract | `dbes.py:38`; table already domain-neutral |
+| C2 | Migration authored **once in the shared `core_oss` chain** (runs in both editions), **not** the parked legacy `core` tree nor EE-only `core_ee` | extract | rename op only; `core` is frozen at `park00000000`; `gateway_connections` is shared schema. See `oss-ee-convergence/migration-chains-and-edition-switch.md` |
+| C3 | `core/gateway/connections/` — service + DAO + interface, **no router** | extract | from `ToolsService` connection code (`service.py:138-383`) |
+| C4 | `ConnectionsGatewayInterface` + Composio **auth** adapter (initiate/status/refresh/revoke) | extract | from `ComposioToolsAdapter` auth verbs |
+| C5 | Repoint tools' connection auth at the shared service; `/tools/connections` contract frozen | extract | ~4 code refs: `dbes.py`, `dao.py:72`, `router.py:160` |
+| C6 | `/tools/connections` and `/triggers/connections` both delegate to the one shared service over the same rows | mimic | no `/gateway/connections` route exists |
+| C7 | **Cross-domain revoke rule**: revoke-for-everyone + show usage; deleting a subscription must not revoke the connection | net-new / decision | no prior connection had two consumers (`mimics.md` §6) |
+
+### 2.2 `triggers` domain — events catalog + adapter (mimic Tools)
+
+| # | Item | Kind | Source / note |
+|---|---|---|---|
+| E1 | Domain skeleton `apis/fastapi/triggers/`, `core/triggers/`, `dbs/postgres/triggers/` | mimic | tools layout |
+| E2 | `ComposioTriggersAdapter` (own httpx client; `triggers_types`, `trigger_instances/...`) implementing `TriggersGatewayInterface` | mimic | `ComposioToolsAdapter` shape |
+| E3 | Events catalog: `/triggers/catalog/.../integrations/{i}/events/{event_key}` returning the event's `trigger_config` schema | mimic | tools action catalog (`action → event`) |
+| E4 | Wiring block in `entrypoints/routers.py` next to tools; adapter built only when `env.composio.enabled` | mimic | `routers.py:578` |
+| E5 | **Exact Composio v3 REST paths** for trigger types/instances | decision | verify vs live OpenAPI (SDK names stable) |
+
+### 2.3 `triggers` domain — subscriptions + deliveries (mimic Webhooks)
+
+| # | Item | Kind | Source / note |
+|---|---|---|---|
+| S1 | `subscriptions` table: project-scoped, FlagsDBA (enabled/valid), DataDBA with `ti_*`, `trigger_config`, `inputs_fields`, destination `references`/`selector`, workflow ref; **FK → `gateway_connections`** | mimic | `webhook_subscriptions` (`types.py:116`) |
+| S2 | `deliveries` table: one audit row per inbound event — resolved `inputs`, workflow `references`, `result`/`error`; migration defined once in `core_oss` | mimic | `webhook_deliveries` (`types.py:156`) |
+| S3 | DBA mixins for both tables | mimic | `dbs/postgres/webhooks/dbas.py` (tools has none) |
+| S4 | Subscription CRUD routes `/triggers/subscriptions/` · `/query` · `/{id}` · `/{id}/refresh` · `/{id}/revoke` + create/disable/delete the Composio `ti_*` via the adapter | mimic | `/webhooks/subscriptions/` + adapter calls |
+| S5 | Delivery read routes `/triggers/deliveries` · `/{id}` · `/query` | mimic | `/webhooks/deliveries` |
+
+### 2.4 `triggers` domain — ingress (mimic Billing)
+
+| # | Item | Kind | Source / note |
+|---|---|---|---|
+| I1 | `POST /triggers/composio/events/` — read raw body before parsing | mimic | billing `/stripe/events/` |
+| I2 | HMAC-SHA256 verify over `{id}.{ts}.{body}` with `COMPOSIO_WEBHOOK_SECRET`; 401 on bad sig; 200 no-op when secret unset | mimic | billing uses `stripe.Webhook.construct_event`; `research.md` § Webhook verification |
+| I3 | Recover `project_id` from `metadata.user_id`; route `metadata.trigger_id` → local subscription; 200-skip unknown/disabled | mimic | billing's payload-scoping; `research.md` §1 |
+| I4 | **Idempotency** dedup on `metadata.id` (store: column vs cache) | net-new / decision | billing leans on Stripe; we own it |
+| I5 | Optional `target`-style env fan-out guard (one Composio webhook URL → many deployments) | decision | cf. `env.stripe.webhook_target` |
+| I6 | **One-time project webhook-URL registration** with Composio (API vs dashboard, per-env) | net-new / decision | no precedent (`research.md` §4.2) |
+
+### 2.5 `triggers` domain — mapping + dispatch (mimic Webhooks resolver + net-new binding)
+
+| # | Item | Kind | Source / note |
+|---|---|---|---|
+| M1 | Promote `resolve_payload_fields` → `resolve_target_fields` into `agenta.sdk.utils.resolvers`; update the webhooks call site to the new name | mimic / extract | `mapping.md` §5/§6; lands at this point |
+| M2 | `inputs_fields` template stored on the subscription; resolves into `WorkflowServiceRequest.data.inputs` **only** | mimic | `mapping.md` §3, §4.2 |
+| M3 | `TRIGGER_EVENT_FIELDS` allowlist (event `data`/`type`/`timestamp`/curated `metadata`; never `ca_*`/secrets); context `{event, subscription, scope}` | mimic | `EVENT_CONTEXT_FIELDS` analogue |
+| M4 | Destination = workflow `references` (+ `selector`), the `/retrieve` shape; drop into `request.references` at dispatch | mimic | `mapping.md` §4.1; `invoke_workflow` threads it (`service.py:556-557`) |
+| M5 | **Trigger ↔ workflow binding** — store + resolve the workflow ref at dispatch | net-new | no domain binds a provider resource to a workflow |
+| M6 | **System-initiated `invoke_workflow`** — what identity (`user_id`) a no-human invocation runs as | net-new / decision | seam only ever called request-scoped (`mimics.md` §2) |
+| M7 | **Async dispatch** — ack-fast + enqueue vs inline (avoid webhook timeout → retry storm) | net-new / decision | proposal § Risks |
+| M8 | **Default mapping** (`"$"` vs stricter) and **schema validation** of `inputs_fields` against the bound workflow's input schema | decision | `mapping.md` §6 |
+| M9 | **Dispatch retry policy** for a failed invocation recorded in `deliveries` vs Composio redelivery | decision | `mapping.md` §6 |
+
+### 2.6 Frontend
+
+| # | Item | Kind | Source / note |
+|---|---|---|---|
+| F1 | "Triggers" surface on a connected integration: events browse, create subscription (pick event + bind workflow + mapping), list/disable/delete | mimic | tools UI (`web/.../gatewayTool`, `web/oss/.../settings/Tools`) |
+| F2 | FE expects **overlapping connection reads** across `/tools/connections` and `/triggers/connections` (same rows) | net-new | consequence of A2-2 |
+| F3 | Deliveries view (audit log) | mimic | could defer past v1 |
+
+---
+
+## 3. Cross-cutting decisions to lock (consolidated)
+
+These appear above tagged `decision`; collected here because they gate multiple work items
+and should be settled (some before code, some during).
+
+| Decision | Gates | Lean / default | Lock by |
+|---|---|---|---|
+| Exact Composio v3 REST paths (E5) | E2, E3, S4 | verify vs live OpenAPI | before adapter code |
+| Project webhook-URL registration (I6) | ingress end-to-end test | manual setup step documented if API-less | before ingress test |
+| Cross-domain revoke rule (C7) | C3–C6, F2 | revoke-for-everyone + show usage | before connection extract lands |
+| Idempotency store (I4) | I-lane, dispatch | column on `deliveries` (dedup on `metadata.id`) | with deliveries table |
+| Sync vs async dispatch (M7) | dispatch lane | async (ack-fast) | before dispatch code |
+| System-initiated `user_id` (M6) | dispatch lane | a project-system identity (resolve from project) | before dispatch code |
+| Default mapping + validation (M8) | subscription create | inputs-only default; validation = stretch | before subscription activate |
+| Dispatch retry policy (M9) | deliveries semantics | bounded retries, else rely on Composio | with dispatch |
+
+---
+
+## 4. Out of scope (restating non-goals so the gap isn't read as larger than it is)
+
+- No merge with / routing through the outbound `webhooks` domain.
+- No workflow-hooks involvement.
+- No downstream consumer beyond a single `invoke_workflow` per event (no eval/queue/re-emit).
+- No new workflow execution path.
+- No custom-OAuth ingress registration; managed-auth only.
+- No polling fallback we own (Composio normalizes to one webhook).
+- No SDK dependency (httpx direct, as tools).
+- No EE-only gating beyond what tools already carry.