An agentic lead intelligence system that autonomously discovers IT service contractor partners, scores them with a self-improving ML model, and generates personalized outreach via LLM agents — with a full data pipeline tracking conversion outcomes.
🔗 Live Demo: contractor-intel-five.vercel.app
The main dashboard gives you a real-time overview of the entire system. The five stat cards show total businesses discovered (17), hot leads identified by the ML model (7), warm leads, average lead score across all businesses (0.408), and outreach emails sent. The bar chart below breaks down lead distribution by city red bars are hot leads, blue are cold so you can instantly see which markets have the most opportunity. Boston, Chicago, and New York are the three target cities in this run.
Every discovered business gets scored by the XGBoost model using 24 engineered features Google rating, review count, website quality, contact completeness, and more. The score bar visually shows relative quality, and each lead is classified as 🔥 hot (score ≥ 0.70), 🌤 warm (0.40–0.70), or ❄️ cold (below 0.40). Hot leads like Micro-Tech USA, Boston IT, and Power Consulting Group score 0.738 and are immediately queued for AI agent outreach. The table is sortable and filterable by tier and city.
The interactive bar chart shows how leads are distributed across cities and tiers. Hovering over a city (Boston shown here) reveals the exact breakdown 3 hot leads and 4 cold leads in Boston. This view helps identify which cities are producing the highest quality leads so the pipeline can be prioritized accordingly. Chicago has the most hot leads in this run, making it the top target market for outreach.
Most lead generation tools are static scripts. This system:
- Learns over time — XGBoost model retrains automatically as outcome data accumulates
- Multi-agent research — CrewAI agents research each business before writing personalized outreach
- Full data flywheel — Discovery → Scoring → Outreach → Outcomes → Retraining, all automated
- 5 database paradigms — PostgreSQL, MongoDB, ClickHouse, Qdrant, Redis in one system
- Production-grade infra — Temporal workflows, Prometheus monitoring, 11 Docker containers
| Tool | Purpose |
|---|---|
| XGBoost + SHAP | Lead scoring with explainability |
| MLflow | Experiment tracking + model versioning |
| CrewAI | Multi-agent orchestration |
| Groq (llama-3.3-70b) | Ultra-fast LLM inference |
| Tool | Purpose |
|---|---|
| Temporal | Workflow orchestration |
| Apache Flink | Real-time scoring stream |
| dbt | Data transformation layer |
| Database | Type | Purpose |
|---|---|---|
| PostgreSQL | Relational | Scores, outreach, outcomes |
| MongoDB | Document | Raw profiles, agent research |
| ClickHouse | Columnar | Analytics + funnel metrics |
| Qdrant | Vector | Semantic lead matching |
| Redis | Cache | Job queues + caching |
| Tool | Purpose |
|---|---|
| FastAPI | REST API (12 endpoints) |
| Strawberry | GraphQL layer |
| Next.js | Dashboard UI |
| Recharts | Data visualization |
| Tool | Purpose |
|---|---|
| Docker Compose | 11 containers |
| Prometheus + Grafana | Monitoring |
| GitHub Actions | CI/CD |
| Terraform | GCP IaC |
- Docker + Docker Compose
- Python 3.12+
- Node.js 18+
- API keys: Google Places, Groq, Hunter.io
git clone https://github.com/Sakshi3027/contractor-intelligence.git
cd contractor-intelligence
cp .env.example .env
# Fill in your API keys in .env
docker-compose up -d
docker exec -i contractor_postgres psql -U contractor_user -d contractor_db < infrastructure/docker/init_postgres.sql
docker exec -i contractor_mongodb mongosh < infrastructure/docker/init_mongodb.js
docker exec -i contractor_clickhouse clickhouse-client --multiquery < infrastructure/docker/init_clickhouse.sql
cd services
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Terminal 1 — Start Temporal worker
python -m pipeline.temporal.worker
# Terminal 2 — Trigger pipeline
python -m pipeline.temporal.scheduler
python -m uvicorn api.main:app --host 0.0.0.0 --port 8001 --reload
cd ../frontend
npm install
npm run dev
# Open https://contractor-intel-five.vercel.app
Phase 1: Discovery
- Google Places API — 15+ IT businesses per city
- Playwright scraper — website content
- Hunter.io — email discovery (87% success rate)
Phase 2: ML Scoring
- 24 engineered features per business
- XGBoost classifier with cross-validation
- SHAP explainability per lead
- MLflow experiment tracking
Phase 3: AI Agents
- Research Agent — analyzes each business profile
- Email Agent — writes personalized outreach
- CrewAI orchestration with Groq inference
Phase 4: Pipeline Automation
- Temporal workflows — full orchestration
- Auto-retry on failure
- Scheduled runs across cities
Phase 5: Feedback Loop
- Track outreach outcomes (opened, replied, converted)
- Auto-retrain model when 10+ outcomes collected
- Model improves over time — real data flywheel
| Service | URL | Description |
|---|---|---|
| Dashboard | https://contractor-intel-five.vercel.app | Next.js UI |
| API Docs | http://34.23.97.178:8001/docs | FastAPI Swagger |
| MLflow | http://34.23.97.178:5002 | Experiment tracking |
| Temporal UI | http://34.23.97.178:8888 | Workflow monitoring |
| Grafana | http://34.23.97.178:3001 | System monitoring |
| Prometheus | http://34.23.97.178:9090 | Metrics |
| Qdrant | http://34.23.97.178:6333/dashboard | Vector DB UI |
contractor-intelligence/
services/
discovery/ # Google Places + email finder
scoring/ # XGBoost + SHAP + MLflow
agents/ # CrewAI research + email agents
api/ # FastAPI REST + GraphQL
pipeline/
temporal/ # Workflows + activities + worker
frontend/ # Next.js dashboard
infrastructure/
docker/ # DB init scripts
terraform/ # GCP IaC
monitoring/ # Prometheus config
data/
models/ # Trained model artifacts
docker-compose.yml # 11 services
New businesses are discovered → ML model scores them → AI agents research and write personalized emails → outreach is sent and tracked → outcomes are recorded → model retrains on new data → better scores next time.
This feedback loop is what separates this from a simple scraper. The system gets smarter with every outreach cycle.
Sakshi Chavan — Software Engineer | Data Scientist | ML Engineer | New York, USA
GitHub: https://github.com/Sakshi3027