Data Engg project
This project implements an end-to-end data engineering pipeline using a medallion architecture (Bronze, Silver, Gold).
⚙️ Tech Stack Azure Data Lake Storage (ADLS Gen2) Azure Data Factory (ADF) Azure Databricks (Apache Spark) GitHub Actions (CI/CD) 🏗️ Architecture File ingestion (CSV) API ingestion (REST) Data Lake landing zone Bronze → Silver → Gold transformations Orchestration via ADF CI/CD via GitHub Actions