Skip to content

Maintenance: Add a scheduled sweeper for leaked e2e infrastructure #5519

Description

@dreamorosi

Summary

Add a scheduled GitHub Actions workflow that removes leaked e2e infrastructure from the test account.

This is a follow-up to #5465, which introduces run-scoped shared Lambda Managed Instances (LMI) capacity provider stacks named LmiShared-<runId>-<arch>. These stacks include EC2 fleets with a minimum capacity of 12 vCPUs and no scale-to-zero, a VPC, and a CloudWatch Logs interface endpoint. Unlike existing serverless e2e leftovers, leaked LMI infrastructure incurs hourly charges.

This issue tracks only the scheduled sweeper. Complementary mitigations, including job timeout-minutes and a destroy retry/second pass, are being handled during the review of #5465.

Why is this needed?

A cancelled or timed-out LMI test cell can leave its per-suite function stack running, with the function attached to the shared capacity provider by ARN. Because this relationship crosses stacks and is invisible to CloudFormation, the if: always() teardown job can encounter DELETE_FAILED when deleting the provider stack. The current destroy script does not retry, and no later process revisits leaked resources in the account.

Other leak paths include:

  • The teardown job being blocked by runner availability.
  • Environment protection rules preventing teardown from starting.
  • Runs using DISABLE_TEARDOWN.
  • LmiShared-local-* stacks created from developer machines.

A recurring cleanup mechanism is needed as a final safety net to limit cost and prevent abandoned e2e resources from accumulating.

This is a follow-up to #5465 and is related to #5092 and #5518.

Which area does this relate to?

  • Automation
  • Tests

Solution

Add a cron-based GitHub Actions workflow that runs nightly or every few hours and can also be invoked manually. It should assume the existing E2E OIDC role and require no standing AWS infrastructure of its own.

In eu-west-1, the workflow should:

  1. List CloudFormation stacks that are older than an age threshold of approximately three hours and either:
    • Match LmiShared-*, including LmiShared-local-*; or
    • Have the tag Service: Powertools-for-AWS-e2e-tests, which all e2e stacks receive through TestStack.
  2. Delete resources in dependency order:
    • Delete function stacks first so their capacity-provider associations are detached.
    • Delete the corresponding shared capacity provider stacks afterward.
  3. Revisit and retry stacks in DELETE_FAILED.
  4. If deletion still fails after two attempts, file an issue or otherwise notify maintainers with the affected stack names and failure details.

The sweeper implementation can live in packages/testing and reuse the toolkit-lib machinery introduced by #5465.

Acknowledgment

Future readers

Please react with 👍 and your use case to help us understand customer demand.

Metadata

Metadata

Assignees

No one assigned

    Labels

    automationThis item relates to automationgithub_actionsPull requests that update GitHub Actions codeinternalPRs that introduce changes in governance, tech debt and chores (linting setup, baseline, etc.)testsPRs that add or change tests

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions