title	GridMind-RL
emoji	⚡
colorFrom	green
colorTo	blue
sdk	docker
app_port	7860
pinned	false
license	mit

GridMind-RL — Train LLMs to manage industrial buildings under faults, grid stress, and natural language objectives.

---

Why This Environment Is Novel

Industrial buildings consume ~40% of global electricity yet rely on naive "always-on" HVAC policies. LLMs can reason about pricing curves, fault alerts, and natural language objectives—but no environment trains them for this. GridMind-RL simulates a full 24-hour building energy system with stochastic electricity prices, equipment faults, and instruction cards, creating a genuinely challenging domain where learned policies translate to real operational value.

Live Demo

	URL
Environment API	https://prajwal782007-gridmind.hf.space
Live Dashboard	https://prajwal782007-gridmind.hf.space/dashboard

Quick test:

curl https://prajwal782007-gridmind.hf.space/health
curl https://prajwal782007-gridmind.hf.space/tasks

---

Environment

	Description
Observation	13 fields: temperature, storage, price, stress, carbon, faults, HVAC efficiency, process demand, batch queue, price forecast
Actions	HVAC level (0-1), thermal charge (-1 to 1), batch slot (0-4), load shed (0-0.5)
Reward	9-component weighted sum: cost, temperature, grid, deadline, efficiency, stability, carbon, instruction, fault_mitigation
Episode	96 steps = 24 simulated hours @ 15-min resolution
Tasks	4 tasks: (1) cost, (2) temperature, (3) demand_response, (4) instruction_following

Reward Weight Rationale

Weights reflect real-world building operator priorities — not arbitrary values:

Component	Weight	Rationale
cost_savings	0.28	Primary operator KPI — energy spend is the main business metric
carbon_reward	0.20	ESG compliance — increasingly mandatory for industrial operators
temp_constraint	0.20	Hard safety constraint — comfort SLA violations incur penalties
grid_response	0.20	Regulatory SLA — demand response programs pay operators to shed load
batch_deadline	0.12	Production continuity — missing batch deadlines causes downstream losses
efficiency_bonus	0.05	Storage arbitrage — incentivises smart charge/discharge timing
stability_penalty	-0.05	Anti-cycling — prevents HVAC thrashing that causes equipment wear
task_satisfaction	0.50*	Task 4 only — weighted per the episode's instruction card
fault_mitigation	dynamic	Emergency response — computed based on fault type and response

*Task 4 instruction reward weight comes from the sampled instruction card, not a fixed value.

Observation Fields

Field	Type	Description
indoor_temperature	float	°C
thermal_storage_level	float	0-1 (0=empty, 1=full)
process_demand	float	kW current industrial power demand
current_price	float	$/kWh
grid_stress_signal	float	0-1 (>0.7 = critical)
carbon_intensity	float	gCO2/kWh
hour_of_day	int	0-23
batch_queue	int[]	pending job deadline slots
cumulative_cost	float	$ total incurred this episode
hvac_efficiency	float	1.0 → degrades to 0.5 over episode
active_faults	string[]	Active fault alarm strings
instruction_card	object	Task 4 objective only
price_forecast	float[]	4-step upcoming price preview

Action Fields

Field	Type	Range
hvac_power_level	float	0.0-1.0
thermal_charge_rate	float	-1.0 to 1.0
batch_job_slot	int	0-4
load_shed_fraction	float	0.0-0.5

---

Core Capabilities

Multi-Agent Coordination

A single oversight LLM coordinates multiple buildings through price signals. The coordinator reads /feeder to see fleet-wide demand, then sets per-building price multipliers via /coordinate to orchestrate behavior.

Long-Horizon Instruction Following

Task 4 presents a natural language objective card like "Keep total energy cost under $2.50 while maintaining 19-23°C". Agents must plan across all 96 steps—not greedy per-step control.

These two capabilities map directly to Theme 1 and Theme 3 of the OpenEnv Hackathon.

---

Results

What the Agent Learns

A naive heuristic runs HVAC at fixed levels based on time-of-day. After GRPO training on GridMind-RL, the agent learns to charge thermal storage during off-peak hours (4¢/kWh) and discharge during peak (32¢/kWh), voluntarily shed load during grid stress signals above 0.7, and adjust HVAC intensity as efficiency degrades over the episode. None of these behaviors are hardcoded — the agent discovers them through the reward signal alone.

Policy	Task 1	Task 2	Task 3	Task 4
Heuristic Baseline	0.494	0.471	0.748	0.478
Zero-shot LLM	0.715	0.645	0.610	0.582
GRPO Fine-tuned LLM	—	—	—	—

GRPO fine-tuned scores updating after full training run on T4 GPU. Training plots below show live progress from the actual run.

Reward vs training step. Blue = per-step reward, red dashed = smoothed average.

Training loss decreasing over steps — confirms the model is updating.

Grade scores per task: heuristic baseline vs GRPO-trained LLM.

Scores are episode grade scores (0.0–1.0, clamped open interval). Heuristic = fixed policy with no learning. Zero-shot = Qwen2.5-1.5B-Instruct prompted with task description, no fine-tuning, evaluated over 1 episode per task. Fine-tuned = GRPO-trained on GridMind-RL environment.

🔄 Live update: GRPO fine-tuned scores will be filled in here immediately after the final training run completes on the T4 GPU.

---

How to Run

Start the environment server

go run main.go

Run the LLM agent (task 1-4)

# Set up your API token
cp .env.example .env
# Edit .env with HF_TOKEN

# Task 1: Cost minimization
python inference.py --task 1 --episodes 5

# Task 2: Temperature management  
python inference.py --task 2 --episodes 5

# Task 3: Full demand response
python inference.py --task 3 --episodes 5

# Task 4: Instruction following
python inference.py --task 4 --episodes 5

# Heuristic baseline (fast, no LLM)
python inference.py --fast-mode --task 3 --episodes 5

Run multi-building coordinator demo

python scripts/multi_building_demo.py

Run training (requires GPU)

python scripts/train_unsloth.py --steps 500 --output-csv results/training_log.csv

Generate training curve plot

python scripts/plot_results.py

---

Architecture

Agent (python/inference.py)
    → HTTP POST /step, /reset, /grade
    ↓
Go Environment Server (main.go) → Port 7860
    ↓
Physics Engine (env/environment.go) + Rewards (env/rewards.go) + Tasks (env/tasks.go)
    ↓
Web Dashboard (dashboard/server.py) → Port 7861

Design philosophy:

• Separation of concerns: Physics engine (Go) decoupled from policy layer (Python)
• OpenEnv compliance: Standardized REST API enables any language agent
• Deterministic simulation: Seeded RNG for reproducible experiments
• Dense rewards: 9-component reward for effective learning

---

API Reference

Method	Endpoint	Description
GET	/health	Health check
GET	/ping	Liveness probe
POST	/reset	Start new episode
POST	/step	Take action step
GET	/state	Get current state
GET	/grade	Grade episode (0.0-1.0 score)
GET	/tasks	Available tasks
GET	/metrics	Prometheus metrics
GET	/replay	Episode history
GET	/feeder	Aggregate fleet state
POST	/coordinate	Set price multipliers
POST	/simulate	World model prediction
POST	/coordinator/reset	Reset multi-building episode
POST	/coordinator/step	Step with per-building actions
GET	/info	OpenEnv metadata
GET	/ws	WebSocket endpoint

---

Project Structure

gridmind-rl/
├── main.go                    # HTTP server & OpenEnv API
├── inference.py              # Agent entry point (LLM + heuristic)
├── openenv.yaml              # OpenEnv spec
├── Dockerfile                # Container build
├── HF_BLOG_POST.md           # Blog write-up
├── baseline_scores.json      # Heuristic baseline scores
├── env/
│   ├── environment.go        # Physics simulation
│   ├── models.go           # Data models
│   ├── rewards.go         # Reward computation
│   ├── tasks.go           # Task grading
│   └── faults.go         # Fault injection
├── scripts/
│   ├── train_unsloth.py   # GRPO training
│   ├── plot_results.py   # Training curve visualizer
│   ├── multi_building_demo.py  # Fleet AI demo
│   └── gridmind_grpo_colab.ipynb  # Colab training notebook
├── server/
│   └── app.py            # Python fallback server
├── dashboard/
│   ├── server.py         # Web server (port 7861)
│   └── static/           # Frontend assets
├── curves/               # Training curves (train N/)
│   └── train N/         # Per-run plots
├── results/              # Training outputs (generated)
└── README.md

---

Links

• 🤗 HuggingFace Space: GridMind-RL
• 📓 Training Notebook: gridmind_grpo_colab.ipynb
• 📝 Blog Post: Read the write-up
• 🐙 GitHub: Code Repository

---

License

MIT License. See LICENSE file.

---

Questions? Open an issue on GitHub.