ESI — U.S. Email Security Index

A Python data engineering pipeline that collects, normalizes, and scores email security posture for public-sector entities, nonprofits, and businesses across all 50 U.S. states.

Quick Start

# Install
python -m venv .venv
source .venv/bin/activate  # or .venv\Scripts\activate on Windows
pip install -e ".[dev]"

# Run pipeline
esi ingest TX              # Collect Texas entities
esi consolidate            # Merge all states
esi scan                   # DNS/MX scanning
esi score                  # SPF/DKIM/DMARC scoring
esi migrate                # Apply database schema
esi build-db               # Load into PostgreSQL

Data Sources

• Census Bureau — 2024 Gazetteer (counties, municipalities, townships)
• NCES CCD — K-12 school districts via Urban Institute API
• NCES IPEDS — Higher education institutions
• EPA SDWIS — Water system special districts
• IRS 990-N — Nonprofit filings (e-Postcard)

Architecture

Ingest → Consolidate → Scan → Score → Migrate → Build DB → Analyze

See CLAUDE.md for full technical documentation.

Built by Lokentra.