⚡ OptiFlow

Intelligent Workflow Orchestrator & Self-Optimizing System

---

Define → Execute → Profile → Optimize → Repeat.
OptiFlow turns complex multi-step workflows into self-improving pipelines.

🧠 What is OptiFlow?

OptiFlow is a full-stack application that lets you define complex, multi-step workflows as directed acyclic graphs (DAGs), executes them with full parallelism and error handling, profiles every run down to the millisecond, automatically detects bottlenecks, and uses AI to suggest and apply optimizations.

The system embodies a continuous improvement loop: build → run → measure → learn → improve.

✨ Features

🎯 Core DAG Workflow Editor — Visual drag-and-drop with React Flow Natural Language Creation — Describe a workflow, AI generates the DAG 4 Task Types — Python, HTTP, File, Simulate (for profiling demos) Execution Engine — Async DAG scheduler with parallelism, retries, timeouts Real-time Dashboard — Run history, execution timelines, live status

🚀 Intelligence Deep Profiling — Per-task timing, resource estimates, variance analysis Bottleneck Detection — Automatic identification of slow paths AI Optimization — DeepSeek-powered suggestions (parallelize, cache, restructure) One-Click Apply — Apply suggestions and re-run to measure impact Analytics — Success rates, avg durations, optimization impact over time

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                    🖥️  Frontend (Next.js 16)                 │
│  ┌──────────┐  ┌───────────┐  ┌──────────┐  ┌───────────┐  │
│  │Dashboard │  │ DAG Editor│  │  Profile │  │Analytics  │  │
│  │          │  │React Flow │  │  Viewer  │  │ Dashboard │  │
│  └────┬─────┘  └─────┬─────┘  └────┬─────┘  └─────┬─────┘  │
└───────┼──────────────┼──────────────┼──────────────┼────────┘
        │              │              │              │
        ▼              ▼              ▼              ▼
┌─────────────────────────────────────────────────────────────┐
│                   ⚙️  Backend (FastAPI)                       │
│  ┌────────────┐  ┌───────────┐  ┌────────┐  ┌───────────┐  │
│  │  REST API  │  │ Execution │  │Profiler│  │Optimizer  │  │
│  │            │  │  Engine   │  │Service │  │Service    │  │
│  └────────────┘  └───────────┘  └────────┘  └─────┬─────┘  │
│                                                    │        │
│                                         ┌──────────▼──────┐ │
│                                         │  AI Service     │ │
│                                         │ (DeepSeek API)  │ │
│                                         └─────────────────┘ │
└──────────────────┬─────────────────────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────────────────────┐
│                     🗄️  PostgreSQL 16                         │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌─────────────┐  │
│  │Workflows │  │   Tasks  │  │   Runs   │  │Suggestions  │  │
│  │  + DAGs  │  │          │  │+ Profiles│  │+ Optimiz.   │  │
│  └──────────┘  └──────────┘  └──────────┘  └─────────────┘  │
└──────────────────────────────────────────────────────────────┘

Tech Stack

Layer	Technology	Why
🖥️ Frontend	Next.js 16 + Tailwind + React Flow + Recharts	Modern, fast, best-in-class DAG visualization
⚙️ Backend	FastAPI + SQLAlchemy + asyncio	Async-native Python, seamless AI/ML integration
🗄️ Database	PostgreSQL 16	Mature, reliable, native JSON support
🤖 AI	DeepSeek API (OpenAI-compatible SDK)	Superior reasoning for optimization tasks
🚦 Queue	In-process asyncio (Redis-ready)	Zero overhead for MVP, drop-in Celery later
🐳 Infra	Docker Compose	One-command full-stack deployment

🚀 Quick Start

🐳 Docker (Recommended)

# Clone and launch everything
git clone https://github.com/NullLabTests/optiflow.git
cd optiflow
docker compose up --build

Then open http://localhost:3000 🎉

🔧 Manual Setup

# Backend
cd backend
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env          # Edit DB URL + DeepSeek key
uvicorn app.main:app --reload --port 8000 &

# Frontend
cd ../frontend
npm install
npm run dev

Service	URL
🌐 Frontend	http://localhost:3000
📡 API	http://localhost:8000
📖 Docs	http://localhost:8000/docs
📘 ReDoc	http://localhost:8000/redoc

🎮 Usage

Creating a Workflow

Method	How
🖱️ Visual Editor	Navigate to Workflows → New Workflow → Add Tasks → Drag edges between nodes
💬 Natural Language	On Dashboard, type "Fetch data from an API, transform it with Python, then simulate heavy processing" → click Create with AI

The Optimization Loop

┌─────────┐    ┌──────────┐    ┌───────────┐    ┌────────┐    ┌─────────┐
│  Build  │───▶│  Execute │───▶│  Profile  │───▶│Analyze │───▶│ Optimize│
│ Workflow│    │  DAG     │    │  Metrics  │    │  Bottl.│    │  +Apply │
└─────────┘    └──────────┘    └───────────┘    └────────┘    └────┬────┘
      ▲                                                            │
      └───────────────────────── Re-run ───────────────────────────┘

1. Run → Click Execute on any workflow
2. Profile → View per-task timing, bottlenecks, critical path
3. Analyze → Click Analyze for AI/rule-based optimization suggestions
4. Apply → One-click apply suggestions (parallelize, cache, restructure)
5. Re-run → Measure the improvement, rinse and repeat

📡 API Reference

All endpoints under /api/v1/. Auth via Bearer token.

Authentication

Method	Endpoint	Description
POST	/auth/register	Create account
POST	/auth/login	Sign in
GET	/auth/me	Current user

Workflows

Method	Endpoint	Description
POST	/workflows	Create workflow
GET	/workflows	List all workflows
GET	/workflows/{id}	Get workflow + DAG
PUT	/workflows/{id}	Update workflow
DELETE	/workflows/{id}	Delete workflow
POST	/workflows/from-nl	Create from natural language
POST	/workflows/{id}/execute	Execute workflow
GET	/workflows/{id}/runs	List run history

Profiling & Optimization

Method	Endpoint	Description
GET	/runs/{id}/profile	Profile report with bottlenecks
POST	/workflows/{id}/analyze	Generate optimization suggestions
GET	/workflows/{id}/suggestions	List all suggestions
POST	/optimizations/apply	Apply a suggestion

Analytics

Method	Endpoint	Description
GET	/analytics/summary	Aggregate stats

🧪 Testing

cd backend
source venv/bin/activate
python -m pytest tests/ -v

✓ test_password_hashing
✓ test_jwt_tokens
✓ test_profile_report_generation
✓ test_profile_with_failures
✓ test_profile_parallelism_efficiency
✓ test_empty_run
══════════════════════════════════════
6 passed in 0.61s

📁 Project Structure

optiflow/
├── 🐳 docker-compose.yml
│
├── ⚙️ backend/
│   ├── app/
│   │   ├── api/          # FastAPI route handlers
│   │   ├── core/         # Config, DB engine, auth
│   │   ├── models/       # SQLAlchemy ORM models
│   │   ├── schemas/      # Pydantic request/response schemas
│   │   └── services/     # Business logic layer
│   │       ├── execution_service.py   # 🧠 DAG scheduler + task runner
│   │       ├── profiler_service.py    # 📊 Metrics + bottleneck detection
│   │       ├── optimization_service.py # 💡 Rule-based suggestions
│   │       ├── ai_service.py          # 🤖 DeepSeek integration
│   │       └── workflow_service.py    # 📋 CRUD operations
│   ├── tests/
│   └── Dockerfile
│
└── 🖥️ frontend/
    └── src/
        ├── app/           # Next.js 16 app router pages
        ├── components/    # React components
        └── lib/           # API client, utilities

🤖 AI Integration

OptiFlow leverages DeepSeek (via OpenAI-compatible SDK) for three capabilities:

Feature	Without API Key	With DeepSeek API Key
🧩 Natural Language → Workflow	Mock template (3-task pipeline)	AI generates optimal DAG from description
🔍 Optimization Analysis	Rule-based (parallelize, cache, structure)	Context-aware AI suggestions with code diffs
📝 Code Review	Static tips	Deep analysis with specific improvements

# In backend/.env
DEEPSEEK_API_KEY=sk-your-key-here
DEEPSEEK_MODEL=deepseek-chat

📊 The Optimization Loop in Action

Before Optimization                    After Optimization
┌──────────────────┐                  ┌──────────────────┐
│ Task A → 1200ms  │                  │ Task A → 800ms   │
│ Task B → 1100ms  │    AI Analyzes  │ Task B → 800ms   │◀── Parallelized!
│ Task C → 1300ms  │ ──────────────▶ │ Task C → 900ms   │
│ Total:  3600ms   │    + Caching    │ Total:  2500ms   │
│ Bottleneck: A,B,C│                  │ 31% faster! 🎉   │
└──────────────────┘                  └──────────────────┘

📜 License

MIT License — see LICENSE.

---

Built with 🧠 by NullLabTests

⭐ Star this repo if you find it useful!
🐛 Report a Bug · 💡 Request Feature