Skip to content

PERF-18 · ec2 makes ~12 sequential API calls per region even when the region is empty #94

Description

@kiwifellows

Part of #63. Found by the #64 baseline.

Problem

ec2 is 35% of total scan time — 259s of a 735s scan, at 15.24s per region against 2.1–3.4s for every other service:

service jobs mean total
ec2 17 15.24s 259s
rds 17 3.41s 58s
lambda 17 2.12s 36s

The slowest jobs include regions with nothing in them — ec2/sa-east-1 33s, ec2/eu-north-1 18s.

Cause

rules/tasks/ec2/tasks.json declares two tasks, and the second fans out hard:

describeInstances -> describeSecurityGroups, describeSubnets
describeVpcs      -> describeSubnets, describeInternetGateways, describeNatGateways,
                     describeRouteTables, describeNetworkAcls, describeVpcPeeringConnections,
                     describeVpcEndpoints, describeSecurityGroups, describeFlowLogs

Every AWS region has a default VPC. So describeVpcs always returns at least one item, and the engine then issues 9 sequential per-VPC calls — in all 17 regions, regardless of whether the account uses that region. That is ~12 API calls per region floor, versus 1 for lambda or kms.

Options

  1. Do nothing. Concurrency (PERF-6 · Run more than one queue worker #69) divides this by the worker count like everything else. 259s/5 ≈ 52s.
  2. Batch the VPC actions. Most of those nine are describe* calls that accept filters — several could take Filters: [{vpc-id, [ids]}] once per region instead of once per VPC. Fewer calls, same data.
  3. Skip regions with only a default and empty VPC. Cheaper than the Tagging API prune (PERF-16 · Revisit region-pruning defaults #79) and safer, since "default VPC with no instances, no NAT, no endpoints" is a positive signal rather than an absence of tags. Needs care — an empty-looking VPC can still hold a misconfigured security group or NACL, which is exactly what some rules look for.

Option 2 is the one that keeps coverage identical. Option 3 changes what gets looked at and needs the same scrutiny the Tagging API prune got.

Open questions

  • Are all nine per-VPC actions actually consumed by rules? scan:validate-rules reports 54 collected methods against 74 rules across 3 rulesets — worth checking whether any of these nine collect data no rule reads.

Priority: P1 — biggest single service cost, but #69 dilutes it, so this is not on the critical path

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceScan speed, throughput, resource useroadmap:nextQueued behind the current milestonesize/SA day or two

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions