Pipeline ETL automatizado de municípios brasileiros — Python + Apache Airflow + AWS S3 + PostgreSQL + Docker
-
Updated
May 17, 2026 - Python
Pipeline ETL automatizado de municípios brasileiros — Python + Apache Airflow + AWS S3 + PostgreSQL + Docker
End-to-end predictive maintenance pipeline for a cleaning-robot fleet: synthetic data generation → Bronze → Silver → Gold → Gradient Boosting.
Production-grade, serverless AWS data pipeline simulating large-scale autonomous-vehicle telemetry-processing at fleet scale. This repository demonstrates end-to-end ingestion, Distributed Stream Triage, Columnar Storage Optimization with Apache Parquet, and Data Lakehouse Partitioning — with future Terraform Infrastructure as Code (IaC) modules.
A LINQ query layer for ParquetSharp that can read entire Parquet datasets — not just single files. Supports Delta Lake tables, Hive partitions, predicate pushdown, and efficient column projection for high-performance .NET data processing.
Add a description, image, and links to the hive-partitioning topic page so that developers can more easily learn about it.
To associate your repository with the hive-partitioning topic, visit your repo's landing page and select "manage topics."