A resource whose plan action is noop never re-persists its downstream edges, so any dependent added to (or removed from) it in that deploy is invisible to later dependency ordering. Destroy then tears the resource down concurrently with something that still depends on it.
Repro
Surfaced by a live SES test (#976), but nothing about it is SES-specific.
// deploy 1
const tenant = yield* Tenant("Customer", {});
const first = yield* ConfigurationSet("ConfigA", {});
const second = yield* ConfigurationSet("ConfigB", {});
yield* TenantResourceAssociation("Link", {
tenantName: tenant.tenantName,
resourceArn: first.configurationSetArn, // Link -> ConfigA
});
// deploy 2 — same stack, association re-pointed. `resourceArn` is immutable,
// so Link is REPLACED. Both config sets are unchanged.
yield* TenantResourceAssociation("Link", {
tenantName: tenant.tenantName,
resourceArn: second.configurationSetArn, // Link -> ConfigB
});
// deploy 2 plan: [ConfigA] noop [ConfigB] noop [Link] replace
Destroy:
[Link] deleting ← concurrent
[ConfigB] deleting
[ConfigB] fail: BadRequestException: Cannot delete <arn> because it has
tenant associations. Remove all tenant associations and try again.
[Link] deleted
[ConfigA] deleting ← correctly waited for Link
[ConfigA] deleted
ConfigA — which Link no longer depends on — was ordered after Link. ConfigB — which it actually depends on — raced it.
Cause
Destroy ordering reads the downstream edges persisted on each resource (Plan.ts:539):
const oldDownstreamDependencies = Object.fromEntries(
oldResources.filter(Boolean).map((r) => [r.fqn, r.downstream]),
);
Create, update, and replace all write downstream: node.downstream when they commit (Apply.ts 537 / 561 / 598 / 648). The noop branch does not (Apply.ts:503):
if (node.action === "noop") {
if (node.state.resourceType !== node.resource.Type) {
yield* commit({ ...node.state, resourceType: node.resource.Type });
}
yield* signalReadyStable;
yield* storeAndSignal({
output: node.state.attr,
props: node.state.props,
bindings: node.state.bindings ?? [],
instanceId: node.state.instanceId,
});
return; // node.downstream is never persisted
}
So an unchanged resource keeps whatever edge list was current the last time it was itself created or updated. In the repro ConfigB still carries the empty list from deploy 1, and ConfigA still carries [Link].
The freshly planned value is already on the node — node.downstream is what every other branch writes — so the edges are available, just not committed.
Impact
Silent wherever the parent API tolerates deleting while children exist, and an intermittent destroy failure wherever it doesn't — security groups with attached ENIs, VPCs with subnets, buckets with objects, configuration sets with associations. It needs only a dependent to be added to or removed from an otherwise-unchanged resource, which makes "re-point a child at a different parent" the common trigger. The failure looks provider-specific, so it tends to get papered over in the provider or the test rather than traced here.
Related: ResourceState.ts:86 carries a TODO(sam): do I need to track the old downstream edges? on UpdatingResourceState.old — same area, different question.
Suggested fix
Persist node.downstream in the noop branch. A no-op means the resource needs no API call; it does not mean its graph position is unchanged.
A resource whose plan action is
noopnever re-persists itsdownstreamedges, so any dependent added to (or removed from) it in that deploy is invisible to later dependency ordering. Destroy then tears the resource down concurrently with something that still depends on it.Repro
Surfaced by a live SES test (#976), but nothing about it is SES-specific.
Destroy:
ConfigA— whichLinkno longer depends on — was ordered afterLink.ConfigB— which it actually depends on — raced it.Cause
Destroy ordering reads the
downstreamedges persisted on each resource (Plan.ts:539):Create, update, and replace all write
downstream: node.downstreamwhen they commit (Apply.ts537 / 561 / 598 / 648). Thenoopbranch does not (Apply.ts:503):So an unchanged resource keeps whatever edge list was current the last time it was itself created or updated. In the repro
ConfigBstill carries the empty list from deploy 1, andConfigAstill carries[Link].The freshly planned value is already on the node —
node.downstreamis what every other branch writes — so the edges are available, just not committed.Impact
Silent wherever the parent API tolerates deleting while children exist, and an intermittent destroy failure wherever it doesn't — security groups with attached ENIs, VPCs with subnets, buckets with objects, configuration sets with associations. It needs only a dependent to be added to or removed from an otherwise-unchanged resource, which makes "re-point a child at a different parent" the common trigger. The failure looks provider-specific, so it tends to get papered over in the provider or the test rather than traced here.
Related:
ResourceState.ts:86carries aTODO(sam): do I need to track the old downstream edges?onUpdatingResourceState.old— same area, different question.Suggested fix
Persist
node.downstreamin thenoopbranch. A no-op means the resource needs no API call; it does not mean its graph position is unchanged.