serverless-phone-extractor

An asynchronous, serverless API on AWS that extracts and deduplicates valid German phone numbers from uploaded text files, built end-to-end with Python and the AWS CDK (infrastructure as code).

Upload a file, get a task ID, and retrieve the parsed results later. Processing happens off the request path through an event-driven S3 → SQS → Lambda pipeline, so the API stays fast and the work scales independently.

The problem this service solves is described in PROBLEM.md.

What a valid number looks like

A number is kept when it:

• starts with +49 or 0049, and
• has exactly 11 digits after that prefix.

Whitespace in the input is ignored. Results are normalized to +49XXXXXXXXXXX and deduplicated per task. Examples of valid input: +4915201365263, 004915201365263.

Architecture

Client
  -> API Gateway
  -> API Lambda            (creates task, returns presigned S3 upload URL)
  -> DynamoDB Tasks
  -> Client uploads file directly to S3
  -> S3 object-created event
  -> SQS                   (buffers + enables retries via a dead-letter queue)
  -> Worker Lambda         (parses, deduplicates, writes results)
  -> DynamoDB TaskResults + Tasks status update

Infrastructure is defined with the AWS CDK in Python, which synthesizes to CloudFormation — a single source of truth for every resource above.

Component	Responsibility
API Lambda	Creates tasks, writes the initial Tasks row, returns a presigned S3 upload URL, serves reads, and deletes tasks.
S3	Stores uploaded files; clients upload directly via a presigned URL.
SQS	Buffers S3 object-created events; a dead-letter queue captures repeatedly failing messages.
Worker Lambda	Loads the uploaded file, runs the shared parser, writes TaskResults, and updates task state.
Tasks table	Task metadata: s3_key, status, timestamps, result count, error state.
TaskResults table	One row per normalized number, keyed by (task_id, phone_number) — natural per-task dedup.

API

Method	Path	Description
POST	/tasks	Create a task; returns a task_id and a presigned S3 upload URL.
GET	/tasks	List all tasks.
GET	/tasks/{task_id}	Fetch task status and results (sorted by number).
DELETE	/tasks/{task_id}	Delete a task and all of its results.

POST /tasks creates a task and returns a presigned upload URL:

{
  "filename": "phone_numbers_3.txt",
  "content_type": "text/plain"
}

{
  "task_id": "72f0aef7-389d-4f45-a49d-3c0df6dc30f1",
  "status": "UPLOAD_PENDING",
  "upload": {
    "method": "PUT",
    "url": "https://...",
    "headers": { "Content-Type": "text/plain" }
  }
}

After the client PUTs the file to the presigned URL, S3 triggers asynchronous processing. GET /tasks/{task_id} then returns task metadata plus the extracted numbers once status is DONE.

Repository layout

domain/   reusable parsing logic (pure, no AWS dependencies)
cloud/    Lambda runtime code (API handler, worker, storage repository)
infra/    AWS CDK infrastructure
tests/    unit tests + live AWS integration tests
samples/  sample input files

The parser in domain/ has no AWS imports, so the core logic is unit-tested in isolation and reused by the worker Lambda unchanged.

Getting started

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

One .venv is used for unit tests, CDK commands, and AWS integration tests.

Testing

Local testing is unit-test only and needs no AWS account:

make test

This covers number normalization and validation, duplicate removal, API request parsing, the delete flow, and worker-side processing.

After deployment, run the live integration suite against the deployed stack:

make test-aws

It runs every samples/phone_numbers_*.txt file through the real API and verifies task creation, direct S3 upload via the presigned URL, asynchronous processing through S3/SQS/Lambda, result retrieval, listing, and deletion.

Deploying

make bootstrap   # one-time CDK bootstrap for the account/region
make deploy      # deploy the stack
make test-aws    # run integration tests against the live stack
make destroy     # tear everything down

Configure the target account via the standard AWS environment variables or AWS_PROFILE / AWS_REGION (see the Makefile).

Commands

make venv         # create local Python venv
make install      # install dependencies into .venv
make test         # run local unit tests
make test-aws     # run deployed AWS integration tests
make bootstrap    # bootstrap CDK in AWS
make synth        # synthesize the CDK stack
make deploy       # deploy the CDK stack
make destroy      # destroy the CDK stack
make diagram      # regenerate the architecture diagram PNG
make clean        # remove generated local build artifacts

Design notes and tradeoffs

• Files upload directly to S3 via presigned URLs rather than through the API, keeping large payloads off the API Lambda and request path.
• Failed worker messages are retried via SQS and land in a dead-letter queue after repeated failures, so a single bad file never blocks the pipeline.
• The worker currently reads the full S3 object into memory — fine for the sample files (~1.1 MB each); very large files would call for streaming.
• GET /tasks/{task_id} returns results inline, so very large result sets would benefit from pagination.
• Results are returned in sorted phone-number order, not original file order.

License

Released under the MIT License.