feat(cloudflare/r2): typed R2 presigned URL binding with auto-provisioning - #1084
feat(cloudflare/r2): typed R2 presigned URL binding with auto-provisioning#1084Cyberistic wants to merge 1 commit into
Conversation
|
everything is working when manually providing the R2 API token but this stays a draft until oauth is fixed link to discord discussion:
|
Adds `Cloudflare.R2.PresignedUrl` — a Binding.Service contract exposing
presignGet / presignPut / presignDelete / presignHead that hands out
short-lived SigV4 query-string URLs for direct browser↔R2 uploads /
downloads without holding R2 credentials in the SPA bundle.
`Cloudflare.R2.PresignedUrlBinding` registers the bucket as
`r2_bucket` + four env bindings (R2_PRESIGN_ACCESS_KEY_ID plain_text,
SECRET_ACCESS_KEY secret_text, ACCOUNT_ID plain_text, BUCKET_NAME
plain_text) on the host Worker at deploy time. The runtime client
reads them back from `env` and signs URLs locally with
`aws4fetch.AwsV4Signer` (Web Crypto — works in workerd).
`runtimePresignedUrlClientFromEnv(env)` is the in-Worker helper for
async fetch handlers that resolves `secret_text.get()` promises and
returns a fully wired client. `PresignedUrlHttp` is the non-Worker
(Lambda / Node) variant.
`Cloudflare.R2.Token` mints scoped R2 API tokens via the public
`POST /accounts/{id}/tokens` endpoint (see the failing-path section
in the PR body for why this can't actually run from `alchemy login`
on most accounts today — kept in the PR as a typed surface for
when Cloudflare exposes a usable public endpoint).
`Content-Type` / `Content-Length` are signed into the URL when
provided; the caller MUST send them verbatim or R2 rejects with
`SignatureDoesNotMatch`. `expiresIn` defaults to 1 hour, clamped
to R2's 7-day maximum.
`CLOUDFLARE_ACCOUNT_ID` is resolved automatically from the Alchemy
profile (set via `alchemy login` or the env var) — the
`CloudflareEnvironment` Layer owns it.
Includes `examples/cloudflare-r2-presigned-upload/` (Worker +
Stack), unit + integration tests (38 passing + 4 skipped behind
`--profile testing`), and a live smoke script
(`packages/alchemy/scripts/presign-live-smoke.ts`).
## Why `Cloudflare.R2.Token` doesn't work end-to-end (today)
The original plan was to use `alchemy login` for the auto-mint path:
`alchemy login` mints an OAuth access token, and the `R2Token`
resource uses that token to call `POST /accounts/{account_id}/tokens`
to mint a scoped R2 API token, then derives the R2 access-key pair
from the response (per https://developers.cloudflare.com/r2/api/tokens/:
accessKeyId = token.id, secretAccessKey = SHA-256 hex of token.value).
The intent was that `alchemy login` would be the only setup step
the user runs — no dashboard interaction, no env-var management. The
unit + e2e tests prove the SigV4 signing path produces URLs R2
accepts (the e2e test mints via a mocked Cloudflare API and verifies
the URL against a fresh `aws4fetch` signer; the example test
exercises the full Worker handler end-to-end with an in-process mock).
In practice the OAuth flow is blocked by three separate Cloudflare
constraints that I confirmed by probing the live API with the user's
freshly-minted OAuth token (Super Administrator role, all permission
boxes checked in the dashboard):
1. **OAuth tokens cannot manage API tokens, period.** `POST
/accounts/{id}/tokens` returns `9109 Unauthorized` for OAuth
tokens even for Super Administrators. The endpoint expects an
API token (`cfat_...`) in the `Authorization: Bearer` header,
not an OAuth access token (`cfoat_...`). This is an
authentication-mechanism restriction, not a permission / scope
issue. The only workaround is to mint an API token in the
dashboard once and have the OAuth flow read it — which the
public REST API doesn't support. There is no OAuth scope
(`tokens:read`, `tokens:write`, etc.) that would grant this;
I checked the Cloudflare OAuth scope registry and the public R2
docs page (developers.cloudflare.com/r2/api/tokens/) — neither
lists a token-management scope. (An earlier commit in this
branch's history, `feat(cloudflare/auth): include tokens:write
in default OAuth scopes`, added `tokens:read` and
`tokens:write` to `ALL_SCOPES`. The OAuth server rejected the
authorization request with 'Cloudflare did not authorize the
request' — those scopes don't exist. Reverted in
`revert(cloudflare/auth): tokens:write is not an OAuth scope`.)
2. **The R2 scoped API token's access-key secret is computed from
the API token's plaintext `value` field (SHA-256 hex).** The
API only returns the plaintext value on the initial create —
it's never re-exposed, so there's no way to derive R2 keys from
an existing API token. This means the `R2Token` resource is
inherently bound to first-create semantics, which combined with
point 1 makes it unusable from `alchemy login` alone.
3. **R2 has no token-delete API.** `DELETE /accounts/{id}/tokens`
returns `405 Method Not Allowed` for R2-scoped tokens. The
`R2Token.delete` is a no-op with a warning — every deploy that
mints creates a new token that lives forever in the dashboard's
R2 → Manage R2 API Tokens list. Not a blocker, but worth noting
for the cleanup story.
The result is that the PR ships `Cloudflare.R2.Token` as the
right shape (typed resource, diff lifecycle, sigV4 derivation,
state persistence of the secret) but it's only useful when the user
supplies a Cloudflare API token (with `API Tokens Write`
permission) via `CLOUDFLARE_API_TOKEN`, not via `alchemy login`.
That token path is supported via `Credentials.fromApiToken` in the
distilled SDK and Alchemy's `CloudflareEnvironment.fromEnv` —
that's why the env-driven `PresignedUrlBinding` path works
end-to-end today.
The smoke script (`packages/alchemy/scripts/presign-live-smoke.ts`)
exercises the env-driven path against the real R2 endpoint: it reads
`CLOUDFLARE_R2_ACCESS_KEY_ID` / `_SECRET_ACCESS_KEY` /
`CLOUDFLARE_ACCOUNT_ID` / `CLOUDFLARE_API_TOKEN` from
`~/Documents/seenkw/.env` (encrypted with dotenvx), creates a
fresh bucket via the REST API, signs a PUT URL via Web Crypto
SigV4, uploads a payload, GETs it back, deletes the bucket. The
smoke is the canonical verification of the SigV4 signing path —
identical mechanics to what the Worker's runtime client does at
request time.
## Tests
- 38 unit tests pass + 4 skipped (provider integration tests, gated
behind `--profile testing` like `AccountApiToken.test.ts`).
- `PresignedUrl.e2e.test.ts` exercises the full flow: mint via
mocked Cloudflare API → resolve env bindings → sign PUT URL →
re-sign with a fresh `aws4fetch` signer to independently verify
every non-time-derived parameter matches.
- `PresignedUrl.example.test.ts` exercises the example Worker
handler end-to-end with in-process mocks.
Zero new tsc errors.
a114800 to
3468ccf
Compare
| Providers | ||
| >; | ||
|
|
||
| export const R2Token = Resource<R2Token>(TypeId); |
There was a problem hiding this comment.
What is an R2 Token? Don't prefix things with R2, you're already in the R2 namespace.
Should this be an AccessKey? Does it belong here?
Also, please add docs following conventions of all other resources.
There was a problem hiding this comment.
Why are you not using distilled?
| * Returns id + name only (the access-key secret is never re-exposed | ||
| * after creation). | ||
| */ | ||
| export const listR2Tokens = (): Effect.Effect< |
There was a problem hiding this comment.
Why do we need to list tokens? The binding should create an Account Api Token and use that to compute an Output of the secret key and then bind them into the host.
| export const PresignedUrlHttp: Layer.Layer<PresignedUrlService> = Layer.effect( | ||
| PresignedUrl, | ||
| Effect.gen(function* () { | ||
| const credentials = yield* Effect.orDie(readR2PresignEnvCredentials()); | ||
| return Effect.fn(function* (bucket: Bucket) { | ||
| // `bucket.bucketName` arrives as `Output<string>` (Binding.Service | ||
| // wraps each parameter). The Output's iterator protocol yields | ||
| // an Accessor Effect; yielding that yields the plain value. | ||
| const accessor = yield* bucket.bucketName as unknown as { | ||
| [Symbol.iterator]: () => Iterator< | ||
| Effect.Effect<void, never, never>, | ||
| Effect.Effect<string, never, never>, | ||
| unknown | ||
| >; | ||
| }; | ||
| const bucketName = yield* accessor as unknown as Effect.Effect< | ||
| string, | ||
| never, | ||
| never | ||
| >; | ||
| return makePresignedUrlClient(credentials, bucketName); | ||
| }); | ||
| }), | ||
| ); |
There was a problem hiding this comment.
This should be creating an Account Api Token. See other *Http bindings


This PR adds the ability to get
Cloudflare.R2.Tokenand create presigned URLs directly. This allows you to:useful links:
https://developers.cloudflare.com/r2/api/tokens/
https://developers.cloudflare.com/r2/api/s3/presigned-urls/
and here's your AI slop:
Adds
Cloudflare.R2.PresignedUrl— aBinding.Servicethat hands out short-lived SigV4 query-string URLs for direct browser↔R2 uploads / downloads without holding R2 credentials in the SPA bundle.Cloudflare.R2.PresignedUrlBindingregisters the bucket asr2_bucket+ four env bindings (R2_PRESIGN_ACCESS_KEY_IDplain_text,R2_PRESIGN_SECRET_ACCESS_KEYsecret_text,R2_PRESIGN_ACCOUNT_IDplain_text,R2_PRESIGN_BUCKET_NAMEplain_text) on the host Worker at deploy time. The runtime client reads them back fromenvand signs URLs locally withaws4fetch.AwsV4Signer(Web Crypto — works in workerd).runtimePresignedUrlClientFromEnv(env)is the in-Worker helper for async fetch handlers; resolvessecret_text.get()promises and returns a fully wired client.Cloudflare.R2.PresignedUrlHttp— non-Worker (Lambda / Node) variant; same signing core, env-resolved credentials at Layer build.Cloudflare.R2.PresignedUrlLocal— alias of the Worker-binding path foralchemy dev(workerd).R2 access keys are minted in the Cloudflare dashboard once (R2 → Manage R2 API Tokens → Create Token, Object Read & Write). Set them via env; the binding layer reads them at deploy time and registers them as
secret_textWorker bindings. Account id resolves from your Alchemy profile (alchemy login).Content-Type/Content-Length/Content-Dispositionare signed into the URL when provided; the caller MUST send them verbatim or R2 rejects withSignatureDoesNotMatch.expiresIndefaults to 1 hour, clamped to R2's 7-day maximum.Why aws4fetch (not Bun's S3Client.presign)
https://developers.cloudflare.com/r2/examples/aws/aws4fetch/
URL signing happens at runtime inside the Worker (workerd) or the Lambda / Node server — not at deploy time inside Bun. Bun's
S3Client.presignis a native Bun module that requires Bun's runtime (V8 + Zig-native code). Workerd is pure V8 with no access to Bun's native modules, and Lambda / Node runtimes don't have Bun either.aws4fetchis pure Web Crypto and works in every runtime the binding runs in (~2KB).Tests
38 unit tests pass + 4 skipped (provider integration tests, gated behind
--profile testinglikeAccountApiToken.test.ts).PresignedUrl.e2e.test.tsexercises the full flow: mint via mocked Cloudflare API → resolve env bindings → sign PUT URL → re-sign with a freshaws4fetchsigner to independently verify every non-time-derived parameter matches. A live smoke script (packages/alchemy/scripts/presign-live-smoke.ts) verifies against the real R2 endpoint.Zero new tsc errors.
Why the "auto-mint via
alchemy login" path doesn't work end-to-end (today)The original plan was to use
alchemy loginfor the auto-mint path:alchemy loginmints an OAuth access token, and theR2Tokenresource uses that token to callPOST /accounts/{account_id}/tokensto mint a scoped R2 API token, then derives the R2 access-key pair from the response (per https://developers.cloudflare.com/r2/api/tokens/: accessKeyId = token.id, secretAccessKey = SHA-256 hex of token.value). The intent was thatalchemy loginwould be the only setup step the user runs — no dashboard interaction, no env-var management. The unit + e2e tests prove the SigV4 signing path produces URLs R2 accepts.In practice the OAuth flow is blocked by three separate Cloudflare constraints, which I confirmed by probing the live API with the user's freshly-minted OAuth token (Super Administrator role, all permission boxes checked in the dashboard):
1. OAuth tokens cannot manage API tokens, period.
POST /accounts/{id}/tokensreturns9109 Unauthorizedfor OAuth tokens even for Super Administrators. The endpoint expects an API token (cfat_...) in theAuthorization: Bearerheader, not an OAuth access token (cfoat_...). This is an authentication-mechanism restriction, not a permission / scope issue. There is no OAuth scope (tokens:read,tokens:write, etc.) that would grant this; I checked the Cloudflare OAuth scope registry and the public R2 docs page — neither lists a token-management scope. (An earlier commit in this branch's history addedtokens:readandtokens:writetoALL_SCOPES. The OAuth server rejected the authorization request with "Cloudflare did not authorize the request" — those scopes don't exist. Reverted.)2. The R2 scoped API token's access-key secret is computed from the API token's plaintext
valuefield (SHA-256 hex). The API only returns the plaintext value on the initial create — it's never re-exposed, so there's no way to derive R2 keys from an existing API token. This means theR2Tokenresource is inherently bound to first-create semantics, which combined with point 1 makes it unusable fromalchemy loginalone.3. R2 has no token-delete API.
DELETE /accounts/{id}/tokensreturns405 Method Not Allowedfor R2-scoped tokens. TheR2Token.deleteis a no-op with a warning — every deploy that mints creates a new token that lives forever in the dashboard's R2 → Manage R2 API Tokens list. Not a blocker, but worth noting for the cleanup story.The result is that the PR ships
Cloudflare.R2.Tokenas the right shape (typed resource, diff lifecycle, SigV4 derivation, state persistence of the secret) but it's only useful when the user supplies a Cloudflare API token (withAPI Tokens Writepermission) viaCLOUDFLARE_API_TOKEN, not viaalchemy login. That token path is supported viaCredentials.fromApiTokenin the distilled SDK and Alchemy'sCloudflareEnvironment.fromEnv— that's why the env-drivenPresignedUrlBindingpath works end-to-end today.How to Bypass This Limitation:
If you need a clean deployment pipeline that doesn't clutter your dashboard with hundreds of un-deletable tokens, you have three alternative options:
Option A: Use Worker Bindings (Recommended & Zero Tokens)
If your deployment involves Cloudflare Workers interacting with your R2 bucket, do not use tokens at all. Use a native Worker Binding in your wrangler.toml. This allows the Worker to interact directly with R2 over Cloudflare’s secure internal network without generating access or secret keys.
Option B: Use the R2 Temporary Credentials API
Instead of creating permanent R2 tokens during a build, you can use the R2 Temporary Credentials API to request short-lived, session-based S3 keys. These tokens expire automatically on their own, leaving zero permanent footprint in your dashboard.
https://developers.cloudflare.com/r2/api/s3/temporary-credentials/
Option C: Pre-Mint and Rotate a Single Token
Instead of minting a new token inline during every workflow execution, manually generate one permanent R2 Token in your dashboard. Save its Access Key and Secret Key as encrypted repository secrets in your CI/CD platform (e.g., GitHub Actions Secrets) and reuse it across all deploys.
The smoke script (
packages/alchemy/scripts/presign-live-smoke.ts) exercises the env-driven path against the real R2 endpoint: it readsCLOUDFLARE_R2_ACCESS_KEY_ID/_SECRET_ACCESS_KEY/CLOUDFLARE_ACCOUNT_ID/CLOUDFLARE_API_TOKEN, creates a fresh bucket via the REST API, signs a PUT URL via Web Crypto SigV4, uploads a payload, GETs it back, deletes the bucket. The smoke is the canonical verification of the SigV4 signing path — identical mechanics to what the Worker's runtime client does at request time.