Skip to content

PERF-17 · Batch the SQS dispatch loop #80

Description

@kiwifellows

Part of #63.

Problem

ProcessAuditScanJob::dispatchRegionScansForService() dispatches region jobs one at a time in a loop. On the SQS path that is ~153 sequential SendMessage round trips.

Change

Use SendMessageBatch (10 per call) for the dispatch loop.

Constraints

Largely irrelevant in practice — every install runs scans on the database connection (docker/installer/scripts/install-messaging.sh:213-214), where dispatch is a local INSERT. Only worth doing if someone actually runs scans over SQS.

Also note the per-region try/catch in the current loop is deliberate (a failed dispatch is excluded from the expected total rather than aborting the loop); batching must preserve that per-region failure accounting.

Priority: P3

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceScan speed, throughput, resource useroadmap:laterReal, but waiting on a triggersize/XSUnder a daywaiting-on-triggerDeliberately not started until a condition fires

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions