Skip to content

TASK: IGDB ETL – Dead Letter Handling Strategy (placeholder table or log) #32

Description

@AndersonGACFilho

Summary

Implement a basic dead-letter handling strategy for records that fail to be processed after multiple retries.

Details

  • For now, this can be as simple as logging the failed record to a specific log level (e.g., ERROR) with all its details, or inserting it into a dedicated ingestion_dlq table in the database.

Acceptance Criteria

  • Records that fail consistently are not dropped silently.
  • Failed records are either logged in detail or saved to a DLQ table.
  • Tests updated/added (unit/integration)
  • No linter / build errors

Dependencies

  • IGDB ETL – Retry & Backoff Strategy Implementation

Out of Scope

  • Automated reprocessing of the dead-letter queue.

Test Plan

  • Write a test that simulates a record that always fails validation and verify that it ends up in the designated dead-letter location.

Checklist

  • Labels added (task, component/igdb-etl)
  • CHANGELOG updated if user-visible
  • No secrets committed

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions