Skip to content

feat(cloudflare/r2): typed R2 presigned URL binding with auto-provisioning - #1084

Draft
Cyberistic wants to merge 1 commit into
alchemy-run:mainfrom
Cyberistic:cloudflare-presigned-url
Draft

feat(cloudflare/r2): typed R2 presigned URL binding with auto-provisioning#1084
Cyberistic wants to merge 1 commit into
alchemy-run:mainfrom
Cyberistic:cloudflare-presigned-url

Conversation

@Cyberistic

@Cyberistic Cyberistic commented Aug 4, 2026

Copy link
Copy Markdown

This PR adds the ability to get Cloudflare.R2.Token and create presigned URLs directly. This allows you to:

  1. Upload straight to buckets without going through Cloudflare workers or their limits
  2. Implement resumable protocols like https://tus.io/
  3. Get upload progress and time estimation for bigger files
    1. Interface with libs like https://better-upload.com/ which require R2 tokens

useful links:
https://developers.cloudflare.com/r2/api/tokens/
https://developers.cloudflare.com/r2/api/s3/presigned-urls/

and here's your AI slop:

Adds Cloudflare.R2.PresignedUrl — a Binding.Service that hands out short-lived SigV4 query-string URLs for direct browser↔R2 uploads / downloads without holding R2 credentials in the SPA bundle.

import * as Alchemy from "alchemy";
import * as Cloudflare from "alchemy/cloudflare";
import * as Effect from "effect/Effect";

const Media = Cloudflare.R2.Bucket("Media");

class PresignWorker extends Cloudflare.Worker<PresignWorker>()(
  "PresignWorker",
  { main: "./src/worker.ts", bindings: { MEDIA: Media }, url: true },
  Effect.gen(function* () {
    const presign = yield* Cloudflare.R2.PresignedUrl(Media);
    return {
      fetch: Effect.gen(function* () {
        const { key, contentType } = /* from request */;
        const { url } = yield* presign.presignPut(key, {
          contentType,
          expiresIn: 300,
        });
        return HttpServerResponse.json({ url });
      }),
    };
  }).pipe(Alchemy.provide(Cloudflare.R2.PresignedUrlBinding)),
) {}
  • Cloudflare.R2.PresignedUrlBinding registers the bucket as r2_bucket + four env bindings (R2_PRESIGN_ACCESS_KEY_ID plain_text, R2_PRESIGN_SECRET_ACCESS_KEY secret_text, R2_PRESIGN_ACCOUNT_ID plain_text, R2_PRESIGN_BUCKET_NAME plain_text) on the host Worker at deploy time. The runtime client reads them back from env and signs URLs locally with aws4fetch.AwsV4Signer (Web Crypto — works in workerd).
  • runtimePresignedUrlClientFromEnv(env) is the in-Worker helper for async fetch handlers; resolves secret_text.get() promises and returns a fully wired client.
  • Cloudflare.R2.PresignedUrlHttp — non-Worker (Lambda / Node) variant; same signing core, env-resolved credentials at Layer build.
  • Cloudflare.R2.PresignedUrlLocal — alias of the Worker-binding path for alchemy dev (workerd).

R2 access keys are minted in the Cloudflare dashboard once (R2 → Manage R2 API Tokens → Create Token, Object Read & Write). Set them via env; the binding layer reads them at deploy time and registers them as secret_text Worker bindings. Account id resolves from your Alchemy profile (alchemy login).

Content-Type / Content-Length / Content-Disposition are signed into the URL when provided; the caller MUST send them verbatim or R2 rejects with SignatureDoesNotMatch. expiresIn defaults to 1 hour, clamped to R2's 7-day maximum.

Why aws4fetch (not Bun's S3Client.presign)

https://developers.cloudflare.com/r2/examples/aws/aws4fetch/

URL signing happens at runtime inside the Worker (workerd) or the Lambda / Node server — not at deploy time inside Bun. Bun's S3Client.presign is a native Bun module that requires Bun's runtime (V8 + Zig-native code). Workerd is pure V8 with no access to Bun's native modules, and Lambda / Node runtimes don't have Bun either. aws4fetch is pure Web Crypto and works in every runtime the binding runs in (~2KB).

Tests

38 unit tests pass + 4 skipped (provider integration tests, gated behind --profile testing like AccountApiToken.test.ts). PresignedUrl.e2e.test.ts exercises the full flow: mint via mocked Cloudflare API → resolve env bindings → sign PUT URL → re-sign with a fresh aws4fetch signer to independently verify every non-time-derived parameter matches. A live smoke script (packages/alchemy/scripts/presign-live-smoke.ts) verifies against the real R2 endpoint.

Zero new tsc errors.


Why the "auto-mint via alchemy login" path doesn't work end-to-end (today)

The original plan was to use alchemy login for the auto-mint path: alchemy login mints an OAuth access token, and the R2Token resource uses that token to call POST /accounts/{account_id}/tokens to mint a scoped R2 API token, then derives the R2 access-key pair from the response (per https://developers.cloudflare.com/r2/api/tokens/: accessKeyId = token.id, secretAccessKey = SHA-256 hex of token.value). The intent was that alchemy login would be the only setup step the user runs — no dashboard interaction, no env-var management. The unit + e2e tests prove the SigV4 signing path produces URLs R2 accepts.

In practice the OAuth flow is blocked by three separate Cloudflare constraints, which I confirmed by probing the live API with the user's freshly-minted OAuth token (Super Administrator role, all permission boxes checked in the dashboard):

1. OAuth tokens cannot manage API tokens, period. POST /accounts/{id}/tokens returns 9109 Unauthorized for OAuth tokens even for Super Administrators. The endpoint expects an API token (cfat_...) in the Authorization: Bearer header, not an OAuth access token (cfoat_...). This is an authentication-mechanism restriction, not a permission / scope issue. There is no OAuth scope (tokens:read, tokens:write, etc.) that would grant this; I checked the Cloudflare OAuth scope registry and the public R2 docs page — neither lists a token-management scope. (An earlier commit in this branch's history added tokens:read and tokens:write to ALL_SCOPES. The OAuth server rejected the authorization request with "Cloudflare did not authorize the request" — those scopes don't exist. Reverted.)

2. The R2 scoped API token's access-key secret is computed from the API token's plaintext value field (SHA-256 hex). The API only returns the plaintext value on the initial create — it's never re-exposed, so there's no way to derive R2 keys from an existing API token. This means the R2Token resource is inherently bound to first-create semantics, which combined with point 1 makes it unusable from alchemy login alone.

3. R2 has no token-delete API. DELETE /accounts/{id}/tokens returns 405 Method Not Allowed for R2-scoped tokens. The R2Token.delete is a no-op with a warning — every deploy that mints creates a new token that lives forever in the dashboard's R2 → Manage R2 API Tokens list. Not a blocker, but worth noting for the cleanup story.

The result is that the PR ships Cloudflare.R2.Token as the right shape (typed resource, diff lifecycle, SigV4 derivation, state persistence of the secret) but it's only useful when the user supplies a Cloudflare API token (with API Tokens Write permission) via CLOUDFLARE_API_TOKEN, not via alchemy login. That token path is supported via Credentials.fromApiToken in the distilled SDK and Alchemy's CloudflareEnvironment.fromEnv — that's why the env-driven PresignedUrlBinding path works end-to-end today.

How to Bypass This Limitation:
If you need a clean deployment pipeline that doesn't clutter your dashboard with hundreds of un-deletable tokens, you have three alternative options:

Option A: Use Worker Bindings (Recommended & Zero Tokens)
If your deployment involves Cloudflare Workers interacting with your R2 bucket, do not use tokens at all. Use a native Worker Binding in your wrangler.toml. This allows the Worker to interact directly with R2 over Cloudflare’s secure internal network without generating access or secret keys.

Option B: Use the R2 Temporary Credentials API
Instead of creating permanent R2 tokens during a build, you can use the R2 Temporary Credentials API to request short-lived, session-based S3 keys. These tokens expire automatically on their own, leaving zero permanent footprint in your dashboard.
https://developers.cloudflare.com/r2/api/s3/temporary-credentials/

Option C: Pre-Mint and Rotate a Single Token
Instead of minting a new token inline during every workflow execution, manually generate one permanent R2 Token in your dashboard. Save its Access Key and Secret Key as encrypted repository secrets in your CI/CD platform (e.g., GitHub Actions Secrets) and reuse it across all deploys.

The smoke script (packages/alchemy/scripts/presign-live-smoke.ts) exercises the env-driven path against the real R2 endpoint: it reads CLOUDFLARE_R2_ACCESS_KEY_ID / _SECRET_ACCESS_KEY / CLOUDFLARE_ACCOUNT_ID / CLOUDFLARE_API_TOKEN, creates a fresh bucket via the REST API, signs a PUT URL via Web Crypto SigV4, uploads a payload, GETs it back, deletes the bucket. The smoke is the canonical verification of the SigV4 signing path — identical mechanics to what the Worker's runtime client does at request time.

@Cyberistic Cyberistic closed this Aug 4, 2026
@Cyberistic Cyberistic reopened this Aug 4, 2026
@Cyberistic

Cyberistic commented Aug 4, 2026

Copy link
Copy Markdown
Author

everything is working when manually providing the R2 API token but this stays a draft until oauth is fixed
feel free to throw fable at it, last slop paragraph in the pr describes the problem.

link to discord discussion:
https://discord.com/channels/1359694195782320389/1359694196830765059/1534219310292140173

image

Adds `Cloudflare.R2.PresignedUrl` — a Binding.Service contract exposing
presignGet / presignPut / presignDelete / presignHead that hands out
short-lived SigV4 query-string URLs for direct browser↔R2 uploads /
downloads without holding R2 credentials in the SPA bundle.

`Cloudflare.R2.PresignedUrlBinding` registers the bucket as
`r2_bucket` + four env bindings (R2_PRESIGN_ACCESS_KEY_ID plain_text,
SECRET_ACCESS_KEY secret_text, ACCOUNT_ID plain_text, BUCKET_NAME
plain_text) on the host Worker at deploy time. The runtime client
reads them back from `env` and signs URLs locally with
`aws4fetch.AwsV4Signer` (Web Crypto — works in workerd).

`runtimePresignedUrlClientFromEnv(env)` is the in-Worker helper for
async fetch handlers that resolves `secret_text.get()` promises and
returns a fully wired client. `PresignedUrlHttp` is the non-Worker
(Lambda / Node) variant.

`Cloudflare.R2.Token` mints scoped R2 API tokens via the public
`POST /accounts/{id}/tokens` endpoint (see the failing-path section
in the PR body for why this can't actually run from `alchemy login`
on most accounts today — kept in the PR as a typed surface for
when Cloudflare exposes a usable public endpoint).

`Content-Type` / `Content-Length` are signed into the URL when
provided; the caller MUST send them verbatim or R2 rejects with
`SignatureDoesNotMatch`. `expiresIn` defaults to 1 hour, clamped
to R2's 7-day maximum.

`CLOUDFLARE_ACCOUNT_ID` is resolved automatically from the Alchemy
profile (set via `alchemy login` or the env var) — the
`CloudflareEnvironment` Layer owns it.

Includes `examples/cloudflare-r2-presigned-upload/` (Worker +
Stack), unit + integration tests (38 passing + 4 skipped behind
`--profile testing`), and a live smoke script
(`packages/alchemy/scripts/presign-live-smoke.ts`).

## Why `Cloudflare.R2.Token` doesn't work end-to-end (today)

The original plan was to use `alchemy login` for the auto-mint path:
`alchemy login` mints an OAuth access token, and the `R2Token`
resource uses that token to call `POST /accounts/{account_id}/tokens`
to mint a scoped R2 API token, then derives the R2 access-key pair
from the response (per https://developers.cloudflare.com/r2/api/tokens/:
accessKeyId = token.id, secretAccessKey = SHA-256 hex of token.value).

The intent was that `alchemy login` would be the only setup step
the user runs — no dashboard interaction, no env-var management. The
unit + e2e tests prove the SigV4 signing path produces URLs R2
accepts (the e2e test mints via a mocked Cloudflare API and verifies
the URL against a fresh `aws4fetch` signer; the example test
exercises the full Worker handler end-to-end with an in-process mock).

In practice the OAuth flow is blocked by three separate Cloudflare
constraints that I confirmed by probing the live API with the user's
freshly-minted OAuth token (Super Administrator role, all permission
boxes checked in the dashboard):

1.  **OAuth tokens cannot manage API tokens, period.** `POST
    /accounts/{id}/tokens` returns `9109 Unauthorized` for OAuth
    tokens even for Super Administrators. The endpoint expects an
    API token (`cfat_...`) in the `Authorization: Bearer` header,
    not an OAuth access token (`cfoat_...`). This is an
    authentication-mechanism restriction, not a permission / scope
    issue. The only workaround is to mint an API token in the
    dashboard once and have the OAuth flow read it — which the
    public REST API doesn't support. There is no OAuth scope
    (`tokens:read`, `tokens:write`, etc.) that would grant this;
    I checked the Cloudflare OAuth scope registry and the public R2
    docs page (developers.cloudflare.com/r2/api/tokens/) — neither
    lists a token-management scope. (An earlier commit in this
    branch's history, `feat(cloudflare/auth): include tokens:write
    in default OAuth scopes`, added `tokens:read` and
    `tokens:write` to `ALL_SCOPES`. The OAuth server rejected the
    authorization request with 'Cloudflare did not authorize the
    request' — those scopes don't exist. Reverted in
    `revert(cloudflare/auth): tokens:write is not an OAuth scope`.)

2.  **The R2 scoped API token's access-key secret is computed from
    the API token's plaintext `value` field (SHA-256 hex).** The
    API only returns the plaintext value on the initial create —
    it's never re-exposed, so there's no way to derive R2 keys from
    an existing API token. This means the `R2Token` resource is
    inherently bound to first-create semantics, which combined with
    point 1 makes it unusable from `alchemy login` alone.

3.  **R2 has no token-delete API.** `DELETE /accounts/{id}/tokens`
    returns `405 Method Not Allowed` for R2-scoped tokens. The
    `R2Token.delete` is a no-op with a warning — every deploy that
    mints creates a new token that lives forever in the dashboard's
    R2 → Manage R2 API Tokens list. Not a blocker, but worth noting
    for the cleanup story.

The result is that the PR ships `Cloudflare.R2.Token` as the
right shape (typed resource, diff lifecycle, sigV4 derivation,
state persistence of the secret) but it's only useful when the user
supplies a Cloudflare API token (with `API Tokens Write`
permission) via `CLOUDFLARE_API_TOKEN`, not via `alchemy login`.
That token path is supported via `Credentials.fromApiToken` in the
distilled SDK and Alchemy's `CloudflareEnvironment.fromEnv` —
that's why the env-driven `PresignedUrlBinding` path works
end-to-end today.

The smoke script (`packages/alchemy/scripts/presign-live-smoke.ts`)
exercises the env-driven path against the real R2 endpoint: it reads
`CLOUDFLARE_R2_ACCESS_KEY_ID` / `_SECRET_ACCESS_KEY` /
`CLOUDFLARE_ACCOUNT_ID` / `CLOUDFLARE_API_TOKEN` from
`~/Documents/seenkw/.env` (encrypted with dotenvx), creates a
fresh bucket via the REST API, signs a PUT URL via Web Crypto
SigV4, uploads a payload, GETs it back, deletes the bucket. The
smoke is the canonical verification of the SigV4 signing path —
identical mechanics to what the Worker's runtime client does at
request time.

## Tests

- 38 unit tests pass + 4 skipped (provider integration tests, gated
  behind `--profile testing` like `AccountApiToken.test.ts`).
- `PresignedUrl.e2e.test.ts` exercises the full flow: mint via
  mocked Cloudflare API → resolve env bindings → sign PUT URL →
  re-sign with a fresh `aws4fetch` signer to independently verify
  every non-time-derived parameter matches.
- `PresignedUrl.example.test.ts` exercises the example Worker
  handler end-to-end with in-process mocks.

Zero new tsc errors.
@Cyberistic
Cyberistic force-pushed the cloudflare-presigned-url branch from a114800 to 3468ccf Compare August 4, 2026 15:40
Providers
>;

export const R2Token = Resource<R2Token>(TypeId);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is an R2 Token? Don't prefix things with R2, you're already in the R2 namespace.

Should this be an AccessKey? Does it belong here?

Also, please add docs following conventions of all other resources.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you not using distilled?

* Returns id + name only (the access-key secret is never re-exposed
* after creation).
*/
export const listR2Tokens = (): Effect.Effect<

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to list tokens? The binding should create an Account Api Token and use that to compute an Output of the secret key and then bind them into the host.

Comment on lines +20 to +43
export const PresignedUrlHttp: Layer.Layer<PresignedUrlService> = Layer.effect(
PresignedUrl,
Effect.gen(function* () {
const credentials = yield* Effect.orDie(readR2PresignEnvCredentials());
return Effect.fn(function* (bucket: Bucket) {
// `bucket.bucketName` arrives as `Output<string>` (Binding.Service
// wraps each parameter). The Output's iterator protocol yields
// an Accessor Effect; yielding that yields the plain value.
const accessor = yield* bucket.bucketName as unknown as {
[Symbol.iterator]: () => Iterator<
Effect.Effect<void, never, never>,
Effect.Effect<string, never, never>,
unknown
>;
};
const bucketName = yield* accessor as unknown as Effect.Effect<
string,
never,
never
>;
return makePresignedUrlClient(credentials, bucketName);
});
}),
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be creating an Account Api Token. See other *Http bindings

@Cyberistic

Cyberistic commented Aug 5, 2026

Copy link
Copy Markdown
Author

I'll elaborate more on the implementation then answer your questions. Creating signed URL's requires Access tokens which are only visible on bucket creation:
image

Two approaches are possible:

  1. User provides Cloudflare.R2.Token at run time
  2. At build time, persist tokens to state using a binding like PresignedUrlBinding

The latter is more useful to me personally and can be standardized, however I do feel like there are cases where a user would want to provide tokens manually (e.g. buckets not built/provisioned with alchemy)

So now to answer all your questions:

@sam-goodwin

What is an R2 Token? Don't prefix things with R2, you're already in the R2 namespace.
Should this be an AccessKey? Does it belong here?
Also, please add docs following conventions of all other resources.

R2 tokens/access keys are required to pre-sign urls. They are not the same as account api tokens. They are given at bucket creation only.
Ofc, docs will be added once implementation is decided. This is just a draft.

Why are you not using distilled?

Distilled has no bucket scoped create tokens.. It is not possible to create tokens after the bucket has already been created. The token is shown once and once only, no new tokens can be added. Does it belong in distilled? Should the whole thing be moved to distilled? ¯\(ツ)

Why do we need to list tokens? The binding should create an Account Api Token and use that to compute an Output of the secret key and then bind them into the host.

Account API tokens are not the same as R2 access tokens. Unless if I'm misunderstanding what you meant, this wouldn't work. Listing tokens goes back to approach (1) where a user has pre-stablished tokens they'd want to use which are not generated by alchemy. (Now that I think about it, it might make sense to also add a way to add account id field, to allow for cross-accounts token usage, in a scenario where an outside party [or different department at work] provides you with a bucket you can use?).

This should be creating an Account Api Token. See other *Http bindings

Again, I think this comes to confusion between account api tokens and bucket access tokens?

I'm happy to discuss this more, @ me anytime here or on discord.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants