Skip to content

AWS env auth: unset AWS_ACCOUNT_ID silently hangs every deploy — getAccountId's STS call re-enters the in-flight cached AWSEnvironment via the Endpoint service (incomplete fix of #704) #1106

Description

@mepuka

Amended 2026-08-05. The original report blamed the auth file lock (Auth/Lock.js) and an unbounded API retry. Both were wrong; verification traced the real mechanism, which is an Effect-level self-await, and found this to be residual of #704. Corrected in full below — apologies for the initial misdirection.

Summary

On 2.0.0-beta.67, any alchemy deploy of a stack that touches an AWS resource with env-based auth (method: "env") hangs forever — zero output, 0% CPU, no HTTP request ever dispatched — when AWS_ACCOUNT_ID is unset, even though key/secret/token/region are all present and valid. Setting AWS_ACCOUNT_ID fixes it completely.

Same symptom, trigger, and mechanism class as #704 (closed 2026-06-30). That fix is present in beta.67 but incomplete: it provided Region and Credentials directly in getAccountId and missed Endpoint.

Root cause

  1. resolveCredentials, method: "env" branch (src/AWS/AuthProvider.ts) falls back to getAccountId(...) when AWS_ACCOUNT_ID is absent.
  2. getAccountId calls STS.getCallerIdentity({}), providing Credentials and Region directly (Layer.succeed / Region.of) — exactly the deploy/plan hangs forever with env auth when AWS_ACCOUNT_ID is unset (layer dependency cycle) #704 fix, with its explanatory comment still in place.
  3. The distilled client (@distilled.cloud/aws, client/api.ts) additionally resolves the Endpoint service: Effect.serviceOption(Endpoint.Endpoint). Nothing shadows it, so it finds the ambient Endpoint.fromEnvironment (src/AWS/Endpoint.ts), whose value is Effect.map(<cached AWSEnvironment effect>, env => env.endpoint).
  4. That cached AWSEnvironment effect (src/AWS/Environment.ts) is the effect currently executingauth.read is its body. The fiber awaits its own Effect.cached deferred and suspends permanently, before the STS request is dispatched.

Consistent with the observed evidence: the stall is at 0% CPU (a suspended fiber, not a poll loop), no network activity, and instrumentation shows execution stopping at exactly the Endpoint service resolution.

For the record, since the first version of this report pointed there: the <profile>-AWS auth lock IS held across read (Auth/AuthProvider.ts LOCKED_METHODS), but it is never contended here — getAccountId has no path back into the AuthProviders registry — and the 120s lock timeout never fires. Both retry policies (makeDefault in distilled, awsRetryFactory in AWS/Providers.ts) are bounded and would not retry this anyway; the lock timeout would surface as a defect, which Effect.retry does not retry.

Repro

# stack with any AWS resource, method env, account id NOT exported:
export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_SESSION_TOKEN=... AWS_REGION=us-east-1
CI=1 alchemy deploy run.ts --stage test --yes   # hangs forever, silent

export AWS_ACCOUNT_ID=123456789012
CI=1 alchemy deploy run.ts --stage test --yes   # completes normally

A stack with AWS.providers() but no AWS resource completes fine — no credential read occurs — which makes the failure look resource-specific when it is not.

Fix

Provide Endpoint alongside Credentials and Region in getAccountId's layer, mirroring the existing Region deadlock comment:

Layer.succeed(Endpoint.Endpoint, Effect.succeed(undefined))

Verified locally: with that one line patched in, the previously-hanging run resolves the account id via STS and completes end to end. Worth auditing whether any other ambient fromEnvironment service is reachable from inside auth.read for the same reason.

As of today, packages/alchemy/src/AWS/AuthProvider.ts on main still has no Endpoint provision in getAccountId.

Environment: macOS arm64, node v24.2.0, alchemy 2.0.0-beta.67, @distilled.cloud/aws 0.30.3, method "env" under CI=1.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions