Skip to content
View renatoaragon's full-sized avatar

Highlights

  • Pro

Block or report renatoaragon

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
renatoaragon/README.md

Hi, I'm Renato AragΓ³n πŸ‘‹

Senior Data Engineer Β· Software Builder Β· Founder @ AragΓ³n Tecnologia
Cloud data platforms on AWS by day. Shipping AI-driven SaaS products on the side.

Portfolio LinkedIn AragΓ³n Tecnologia


πŸš€ About me

  • πŸ› οΈ Senior Data Engineer building large-scale data pipelines and platforms on AWS.
  • 🧱 Founder & engineer at AragΓ³n Tecnologia, where I take products from idea to production: architecture, development, deployment and operations.
  • πŸ€– I integrate AI where it genuinely improves a product, not as a gimmick.
  • 🌍 Based in Porto, Portugal. Working in international, English-speaking teams.
  • πŸ’¬ Ask me about data engineering, AWS, PySpark, infrastructure as code, and building SaaS solo.

🧰 Tech stack

Data Engineering

Python PySpark Apache Kafka Debezium Databricks Apache Airflow dbt Apache Iceberg DuckDB Pandas SQL Apache Hadoop Apache Hive MapReduce

Architecture & Patterns

Data Lakehouse Medallion Architecture Dimensional Modeling Data Warehousing ETL / ELT Streaming CDC Event-Driven Data Quality IaC CI/CD Blue-Green Deploys

AWS & Cloud

AWS Amazon S3 AWS Glue Amazon Athena AWS Lambda Terraform Docker Linux

Software & Web

TypeScript Next.js React Node.js PostgreSQL


πŸ—ΊοΈ How I think about data engineering

Tools are the disposable part. The framework I reach for today will be replaced sooner than I would like, and that is fine. What lasts is the architecture: a design that is sustainable, scalable and durable rarely has to change, whatever tool implements it underneath. So I optimize for the shape, not the stack.

The projects below are laid out along the path data takes through a platform. Each is a decision rather than a demo, runs on synthetic data, is verified in CI, and carries a short Design principles note that explains the why. Read together, they are one opinion about how a data platform should be built.

      INGEST              PROCESS            ORCHESTRATE          STORE + MODEL           CONSUME
 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚ kafka-retail-  β”‚ β”‚ spark-retail-  β”‚ β”‚ airflow-retail-  β”‚ β”‚ delta-lakehouse- β”‚ β”‚ dbt-duckdb-      β”‚
 β”‚ streaming      β”‚ β”‚ etl            β”‚ β”‚ orchestration    β”‚ β”‚ mlflow           β”‚ β”‚ analytics        β”‚
 β”‚ stream + CDC   β”‚ β”‚ batch ETL      β”‚ β”‚ medallion DAG    β”‚ β”‚ Delta + MLflow   β”‚ β”‚ nl-to-sql        β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

 foundations    terraform-aws-datalake (IaC)  Β·  data-mapping-framework (canonical model)  Β·  kanban-streaming-metrics (flow metrics)
 in every repo  explicit schemas  Β·  data-quality gates  Β·  idempotent, replayable design  Β·  tests + CI

πŸ§ͺ Open-source projects

Small, focused data-engineering projects. Each one runs with synthetic data and is verified in CI.

Project What it demonstrates CI
kafka-retail-streaming Kafka (KRaft) + Spark Structured Streaming: event-time windows with watermarks, dead-letter queue, Iceberg sink with exactly-once semantics (ADR), CDC with Postgres + Debezium merged via foreachBatch CI
spark-retail-etl PySpark batch ETL (raw β†’ clean β†’ curated): incremental loads over a high-watermark, partitioned outputs, quality gates with volume anomaly detection, run summaries, ADRs CI
airflow-retail-orchestration Airflow TaskFlow DAG orchestrating a medallion pipeline: quality-gate branching, dynamic task mapping, idempotent partitioned writes, DAG-integrity and end-to-end tests CI
delta-lakehouse-mlflow Medallion lakehouse on Delta Lake with an MLflow-tracked churn model: idempotent silver via MERGE, full-recompute gold, tracked and registerable modelling CI
kanban-streaming-metrics Streaming flow metrics with Redpanda + PySpark: throughput, cycle time, WIP, and percentile-based forecasting (p85/p95, SLA attainment) CI
dbt-duckdb-analytics Analytics engineering with dbt + DuckDB: layered staging/marts and four layers of data tests, up to cross-layer revenue reconciliation CI
terraform-aws-datalake Reusable Terraform module for an AWS data lake (S3 + Glue + Athena): secure by default, plan-time input validation, tflint gate CI
data-mapping-framework Map multiple platforms/formats (CSV, JSON, SQL, Parquet) onto one canonical model, with business rules and full violation reporting per rejected row CI
nl-to-sql Natural-language-to-SQL with Claude: a deterministic read-only guard with lexer-based query normalization CI

Real products, code in the open β€” not demos: these run in production with real users.

Product What it is
meucomboio-pt "Which train can I still catch?" β€” next departures and trip planning for Portugal on CP's open GTFS data: daily sync pipeline with a quality gate, service-calendar-aware planning, live pricing, and a dbt analytics layer
inventario-familiar Self-hosted estate portal with a dbt + DuckDB analytics layer: financial marts and a reconciliation gate that fails CI if a single cent goes unaccounted for. Synthetic data only

πŸ—οΈ Products I build and operate

Beyond the open-source projects above, I build and run a small fleet of production SaaS products at AragΓ³n Tecnologia, mostly on my own, from the first commit to the 3am alert when something breaks.

Product What it is
🧠 PsycoHub Clinical management + AI for psychologists
πŸ₯— NutriPro AI-assisted platform for nutritionists
πŸ’… Sola Beauty Management platform for beauty studios

How I run them:

  • Each one is multi-tenant SaaS on AWS, released with zero-downtime blue-green deploys: a small Node agent flips nginx over once the health checks pass.
  • I operate them, not just ship them: self-hosted uptime monitoring, error tracking with Sentry, structured logging, scheduled jobs, and automated LGPD retention.
  • Some are realtime, using Server-Sent Events for live dashboards and WebRTC for video calls.
  • I pull data across products into a medallion lakehouse (bronze, silver, gold) on Databricks with an MLflow model on top, fed by anonymized exports.
  • And I add AI where it genuinely helps the person using the product, never for show.

The products and their source stay private. What you see in this profile are the same engineering patterns, rebuilt from scratch with synthetic data.

Open to business partnerships and collaboration on data platforms and products. Reach me at aragontecnologia.com or on LinkedIn.

Pinned Loading

  1. delta-lakehouse-mlflow delta-lakehouse-mlflow Public

    Medallion lakehouse on Delta Lake with an MLflow-tracked churn model, over synthetic SaaS product-analytics data. Runs locally, no cloud.

    Python

  2. kafka-retail-streaming kafka-retail-streaming Public

    Retail streaming done right: exactly-once into Iceberg, dead-letter queue, late-data handling and CDC (Kafka KRaft + Spark).

    Python

  3. spark-retail-etl spark-retail-etl Public

    PySpark batch ETL (raw to clean to curated) with data quality gates and a pytest suite.

    Python

  4. airflow-retail-orchestration airflow-retail-orchestration Public

    Apache Airflow orchestration of a retail medallion pipeline (bronze/silver/gold) with a data-quality gate, retries and idempotent backfills. Runs locally, no cloud.

    Python

  5. dbt-duckdb-analytics dbt-duckdb-analytics Public

    Analytics engineering with dbt + DuckDB - layered staging/marts models and data tests.

  6. terraform-aws-datalake terraform-aws-datalake Public

    Reusable Terraform module for an AWS data lake (S3 + Glue + Athena), secure by default.

    HCL