AirGuard-AI 2.0 — PM2.5 Forecasting for the Schwerte Sensor Network

Trains a PatchTST or PatchTSMixer transformer to forecast PM2.5 over a configurable horizon (default 6 hours) per station, using the past 168 hours of local PM and weather. Trained globally on the kept-stations subset (auto-filtered per data_prep run against a completeness threshold in the dataset config), evaluated per-horizon and per-station against persistence, seasonal-naive, and LightGBM baselines. Also ships a live-prediction CLI that pulls the most recent hours from a registered provider plugin (e.g. the external schwerte_provider_plugin) and runs a forecast end-to-end.

📚 CLI & YAML reference — every flag, every YAML field, every output path. 🛠️ Dataset creation · Training & evaluation — end-to-end walkthroughs. 🧠 Design & principles — the why behind the architecture and the rules that guard it.

📊 Overview

• What it does — fetch → preprocess → train a transformer forecaster → evaluate against three baselines → serve point predictions, either from on-disk parquets or from a live sensor feed.
• What's distinctive — pluggable provider layer (5 built-ins + auto-discovered external plugins for things like the schwerte feed), per-dataset outlier-clipping policy (percentile or iqr, configurable columns/floor), and a per-horizon directory layout so 6h and 12h experiments coexist cleanly on disk.
• What it produces — per-station forecasts on a configurable horizon, with MAE/RMSE/sMAPE per horizon hour, a combined leaderboard across models, and station-map / per-station / timeline plots.

⚙️ Install

uv sync           # installs Python 3.12+, PyTorch, Transformers, LightGBM, etc.

GPU is optional but strongly recommended for training. Live training metrics report to a Trackio project (airguard-pm25) when report_to: ["trackio"] is set in the training YAML — disable by setting it to ["none"].

🐳 Docker

A multi-stage, CUDA-enabled image is provided (Dockerfile). The image bundles the code and the locked dependency tree (uv sync --frozen) but no data and no secrets — data/, runs/ and eval/ are bind-mounted at run time, and credentials are passed via env vars / .env.

GPU prerequisite. GPU access needs nvidia-container-toolkit on the host and the native Docker Engine context — run docker context use default. Docker Desktop on Linux runs the daemon in a VM with no GPU passthrough, so --gpus all / the compose nvidia device fails there with could not select device driver "nvidia" with capabilities: [[gpu]]. If you see that, check docker context ls: if desktop-linux is active, switch back with docker context use default.

# Build (resolves uv.lock, installs the CUDA torch wheel).
docker build -t airguard:latest .

# Run any `python -m airguard.<module>` command. Mount data/ and runs/ so
# parquets and checkpoints persist on the host. --gpus all needs
# nvidia-container-toolkit installed on the host.
docker run --rm --gpus all \
    -v "$PWD/data:/app/data" -v "$PWD/runs:/app/runs" -v "$PWD/eval:/app/eval" \
    --env-file .env \
    airguard:latest \
    python -m airguard.train --config configs/patchtst_pm25.yaml

# Inference is one-shot (no HTTP server) — predict prints forecast JSON to stdout:
docker run --rm --gpus all -v "$PWD/data:/app/data" -v "$PWD/runs:/app/runs" \
    airguard:latest \
    python -m airguard.predict --checkpoint runs/<model>/best \
    --dataset configs/dataset_schwerte_pm25.yaml --station sc_10056 --asof 2026-03-30T12:00

For local development, docker-compose.yml wires up the GPU, the bind mounts and .env so you only choose the command:

docker compose run --rm airguard                                   # default: train
docker compose run --rm airguard python -m airguard.data_prep \
    --dataset configs/dataset_schwerte_pm25.yaml

🔐 Environment / credentials

Copy the template and fill in keys:

cp .env.example .env && $EDITOR .env

Loaded automatically by airguard.fetch and airguard.providers.plugin_loader:

• AIRGUARD_PLUGIN_PATH — colon-separated (Unix) or semicolon-separated (Windows) list of directories scanned for external provider plugins. Each plugin is a subdirectory with plugin.yaml + provider.py.
• OPENAQ_API_KEY — optional. Auto-injected into plugins.openaq.api_key for the built-in OpenAQ provider.
• SCHWERTE_API_KEY, SCHWERTE_TAG_ID — credentials for the external schwerte plugin (autoloaded when the plugin is discovered).
• General plugin convention — any env var named {PLUGIN_NAME_UPPER}_<KEY> is injected into plugin_config[plugin_name][key.lower()] for the matching discovered plugin (e.g. MYPLUGIN_API_KEY → plugin_config["myplugin"]["api_key"]). Env values override YAML values.

🚀 Quickstart

# 0. (Optional) Fetch fresh data. Skip if data/raw/<provider>/<city>/*.csv already exists.
python -m airguard.fetch --config configs/data_acquisition.yaml --dry-run    # plumbing check
python -m airguard.fetch --config configs/data_acquisition.yaml              # real fetch

# 1. Preprocess: CSV → per-station parquets + window indexes + clip stats.
python -m airguard.data_prep --dataset configs/dataset_schwerte_pm25.yaml

# 2. Baselines (persistence, seasonal-naive, LightGBM) on the test split.
python -m airguard.baselines --split test --dataset configs/dataset_schwerte_pm25.yaml

# 3. (Optional) Quick smoke train — 200 steps, no early stopping.
python -m airguard.train --config configs/patchtst_pm25.yaml --max-steps 200 \
    --limit-train 50000 --limit-val 10000

# 4. Full training.
python -m airguard.train --config configs/patchtst_pm25.yaml

# 5. Score the checkpoint on the test split (+ --plot for single-model PNGs).
python -m airguard.evaluate \
    --checkpoint runs/<model>-<dataset>-<timestamp>/best \
    --dataset configs/dataset_schwerte_pm25.yaml --plot

# 6. Multi-model leaderboard + combined horizon plot.
python -m airguard.eval.aggregate --split test --eval-dir eval/6h \
    --dataset-name schwerte_pm25_v1

# 7. (Optional) Per-station MAE geographic map for every model in eval/.
python -m airguard.eval.station_map --eval-dir eval

# 8. One-shot forecast from on-disk parquets.
python -m airguard.predict \
    --checkpoint runs/<model>-<dataset>-<timestamp>/best \
    --dataset configs/dataset_schwerte_pm25.yaml \
    --station sc_10056 --asof 2026-03-30T12:00

# 9. One-shot forecast from a LIVE sensor feed (provider plugin required).
python -m airguard.predict_live \
    --checkpoint runs/<model>-<dataset>-<timestamp>/best \
    --dataset configs/dataset_schwerte_pm25.yaml \
    --provider schwerte --station MCF88_33570-Gänsewinkel

Every flag is documented in docs/reference.md; narrative walkthroughs of these steps live in docs/dataset-creation.md and docs/training-and-evaluation.md.

📊 Results

In-distribution test split (2026-01 → 2026-03, 135 stations, 241,986 windows). The 12h model matches the 6h model on the short horizons it shares and extends cleanly out to 12 h — training for the longer horizon does not degrade short-horizon accuracy:

Per-station MAE distribution and geographic spread — same colour scale, side-by-side comparison of the two horizons:

6h model	12h model

Out-of-distribution evaluation on the Schwerte-plugin sensor network (MCF88, disjoint from training). Detail view of station Gänsewinkel — observed PM2.5 (black) vs. concatenated forecast windows (blue), with nighttime hours shaded; produced by airguard.eval.single_station:

Reproduce: run quickstart steps 1–6 with configs/dataset_schwerte_pm25_12h.yaml + configs/patchtsmixer_pm25_12h.yaml; PNGs land in eval/<H>h/. The single-station detail plot:

python -m airguard.eval.single_station \
    --checkpoint runs/<run>/best \
    --dataset configs/dataset_schwerte_plugin_12h.yaml \
    --station "MCF88_33570-Gänsewinkel"

📁 On-disk layout

• airguard/ — Python package (providers, data prep, dataset, model, train, baselines, evaluate, predict, predict_live, fetch CLI, eval/aggregator + map + plot helpers).
• airguard/providers/ — pluggable data-source layer. Built-ins: sensorcommunity, openaq, uba, luftqualitaet, beijing + weather/open_meteo. plugin_loader.py discovers externals via AIRGUARD_PLUGIN_PATH.
• configs/ — data_acquisition*.yaml (fetch), dataset_*.yaml (dataset definition; per-horizon variants live alongside, e.g. dataset_schwerte_pm25_12h.yaml), patchtst_pm25.yaml / patchtsmixer_pm25.yaml (training; horizon-suffixed variants likewise).
• scripts/ — run_data_prep.py and plot_all_models.py thin aliases.
• docs/ — extended documentation (reference.md for CLIs + YAML schemas; logo assets under docs/logos/; result PNGs embedded in the README under docs/figures/results/).
• data/ — .gitignored. Dataset-scoped paths:
  • data/raw/<provider>/<city>/<station>.csv + stations_static.csv — fetch output (provider-scoped, dataset-agnostic).
  • data/<dataset_name>/processed/<station_id>.parquet — per-station feature parquets.
  • data/<dataset_name>/splits/{train,val,test}.parquet — window-start indexes.
  • data/<dataset_name>/stats.json — outlier-clip bounds.
  • data/<dataset_name>/excluded_stations.json — per-station completeness + exclusion list.
• runs/ — training outputs, one directory per run: runs/<model_type>-<dataset_name>-YYYYMMDD-HHMMSS/{best/,model_config.json,train_config.json,val_metrics.json}. .gitignored.
• eval/ — .gitignored. Outputs are namespaced by prediction horizon so 6h and 12h experiments coexist cleanly:
  • eval/<H>h/<split>_summary_<name>.json — aggregated metrics (incl. skill_vs_persistence).
  • eval/<H>h/<split>_metrics_<name>.parquet — per-station MAE matrix (S × H).
  • eval/<H>h/horizon_plot_<name>.png, eval/<H>h/per_station_mae_<name>.png, eval/<H>h/test_timeline_<name>.png — evaluate.py --plot PNGs.
  • eval/<H>h/horizon_plot_combined.png — multi-model overlay, produced by airguard.eval.aggregate.
  • eval/<H>h/station_map_<name>.png — geographic per-station MAE map, produced by airguard.eval.station_map.
  • eval/<H>h/single_station_<station_id>_<name>.png — single-station detail plot, produced by airguard.eval.single_station.
  • eval/baselines/<dataset_name>/<H>h/baselines_<split>.json — baseline metrics, scoped per dataset and per horizon.

🏆 Contributors

A collaborative effort between Westfälische Hochschule Gelsenkirchen, Bocholt, Recklinghausen, the Urban-KI initiative, and the City of Schwerte.