You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
AWS env auth: unset AWS_ACCOUNT_ID silently hangs every deploy — getAccountId's STS call re-enters the in-flight cached AWSEnvironment via the Endpoint service (incomplete fix of #704) #1106
Amended 2026-08-05. The original report blamed the auth file lock (Auth/Lock.js) and an unbounded API retry. Both were wrong; verification traced the real mechanism, which is an Effect-level self-await, and found this to be residual of #704. Corrected in full below — apologies for the initial misdirection.
Summary
On 2.0.0-beta.67, any alchemy deploy of a stack that touches an AWS resource with env-based auth (method: "env") hangs forever — zero output, 0% CPU, no HTTP request ever dispatched — when AWS_ACCOUNT_ID is unset, even though key/secret/token/region are all present and valid. Setting AWS_ACCOUNT_ID fixes it completely.
Same symptom, trigger, and mechanism class as #704 (closed 2026-06-30). That fix is present in beta.67 but incomplete: it provided Region and Credentials directly in getAccountId and missed Endpoint.
Root cause
resolveCredentials, method: "env" branch (src/AWS/AuthProvider.ts) falls back to getAccountId(...) when AWS_ACCOUNT_ID is absent.
The distilled client (@distilled.cloud/aws, client/api.ts) additionally resolves the Endpoint service: Effect.serviceOption(Endpoint.Endpoint). Nothing shadows it, so it finds the ambient Endpoint.fromEnvironment (src/AWS/Endpoint.ts), whose value is Effect.map(<cached AWSEnvironment effect>, env => env.endpoint).
That cached AWSEnvironment effect (src/AWS/Environment.ts) is the effect currently executing — auth.read is its body. The fiber awaits its own Effect.cached deferred and suspends permanently, before the STS request is dispatched.
Consistent with the observed evidence: the stall is at 0% CPU (a suspended fiber, not a poll loop), no network activity, and instrumentation shows execution stopping at exactly the Endpoint service resolution.
For the record, since the first version of this report pointed there: the <profile>-AWS auth lock IS held across read (Auth/AuthProvider.tsLOCKED_METHODS), but it is never contended here — getAccountId has no path back into the AuthProviders registry — and the 120s lock timeout never fires. Both retry policies (makeDefault in distilled, awsRetryFactory in AWS/Providers.ts) are bounded and would not retry this anyway; the lock timeout would surface as a defect, which Effect.retry does not retry.
Repro
# stack with any AWS resource, method env, account id NOT exported:export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_SESSION_TOKEN=... AWS_REGION=us-east-1
CI=1 alchemy deploy run.ts --stage test --yes # hangs forever, silentexport AWS_ACCOUNT_ID=123456789012
CI=1 alchemy deploy run.ts --stage test --yes # completes normally
A stack with AWS.providers() but no AWS resource completes fine — no credential read occurs — which makes the failure look resource-specific when it is not.
Fix
Provide Endpoint alongside Credentials and Region in getAccountId's layer, mirroring the existing Region deadlock comment:
Verified locally: with that one line patched in, the previously-hanging run resolves the account id via STS and completes end to end. Worth auditing whether any other ambient fromEnvironment service is reachable from inside auth.read for the same reason.
As of today, packages/alchemy/src/AWS/AuthProvider.ts on main still has no Endpoint provision in getAccountId.
Summary
On
2.0.0-beta.67, anyalchemy deployof a stack that touches an AWS resource with env-based auth (method: "env") hangs forever — zero output, 0% CPU, no HTTP request ever dispatched — whenAWS_ACCOUNT_IDis unset, even though key/secret/token/region are all present and valid. SettingAWS_ACCOUNT_IDfixes it completely.Same symptom, trigger, and mechanism class as #704 (closed 2026-06-30). That fix is present in beta.67 but incomplete: it provided
RegionandCredentialsdirectly ingetAccountIdand missedEndpoint.Root cause
resolveCredentials,method: "env"branch (src/AWS/AuthProvider.ts) falls back togetAccountId(...)whenAWS_ACCOUNT_IDis absent.getAccountIdcallsSTS.getCallerIdentity({}), providingCredentialsandRegiondirectly (Layer.succeed/Region.of) — exactly thedeploy/planhangs forever withenvauth whenAWS_ACCOUNT_IDis unset (layer dependency cycle) #704 fix, with its explanatory comment still in place.@distilled.cloud/aws,client/api.ts) additionally resolves theEndpointservice:Effect.serviceOption(Endpoint.Endpoint). Nothing shadows it, so it finds the ambientEndpoint.fromEnvironment(src/AWS/Endpoint.ts), whose value isEffect.map(<cached AWSEnvironment effect>, env => env.endpoint).AWSEnvironmenteffect (src/AWS/Environment.ts) is the effect currently executing —auth.readis its body. The fiber awaits its ownEffect.cacheddeferred and suspends permanently, before the STS request is dispatched.Consistent with the observed evidence: the stall is at 0% CPU (a suspended fiber, not a poll loop), no network activity, and instrumentation shows execution stopping at exactly the
Endpointservice resolution.For the record, since the first version of this report pointed there: the
<profile>-AWSauth lock IS held acrossread(Auth/AuthProvider.tsLOCKED_METHODS), but it is never contended here —getAccountIdhas no path back into theAuthProvidersregistry — and the 120s lock timeout never fires. Both retry policies (makeDefaultin distilled,awsRetryFactoryinAWS/Providers.ts) are bounded and would not retry this anyway; the lock timeout would surface as a defect, whichEffect.retrydoes not retry.Repro
A stack with
AWS.providers()but no AWS resource completes fine — no credential read occurs — which makes the failure look resource-specific when it is not.Fix
Provide
EndpointalongsideCredentialsandRegioningetAccountId's layer, mirroring the existing Region deadlock comment:Verified locally: with that one line patched in, the previously-hanging run resolves the account id via STS and completes end to end. Worth auditing whether any other ambient
fromEnvironmentservice is reachable from insideauth.readfor the same reason.As of today,
packages/alchemy/src/AWS/AuthProvider.tson main still has noEndpointprovision ingetAccountId.Environment: macOS arm64, node v24.2.0, alchemy 2.0.0-beta.67, @distilled.cloud/aws 0.30.3, method "env" under CI=1.