Measure how, not just whether
A graph-based coordination benchmark where a "Green Agent Judge" dissects multi-agent interactions into network graphs and evaluates collaboration quality through structural analysis and LLM assessment.
GraphJudge transforms multi-agent interactions into coordination graphs and evaluates collaboration quality through graph theory, LLM assessment, and performance analysis.
Graph-based runtime coordination analysis - measuring collaboration quality through NetworkX metrics and LLM-as-judge evaluation.
| Tier | Type | Description |
|---|---|---|
| 1 | Graph | NetworkX metrics (centrality, efficiency) |
| 2 | LLM-Judge | Coordination quality assessment |
| 3 | Text | Similarity metrics (plugin for PeerRead) |
- Research Agent
- Multi-Agent
- AAA (Agentified Agent Assessment)
See docs/GreenAgent-UserStory.md for full problem statement and value proposition.
TL;DR: Graph-based structural analysis + LLM-as-judge + text similarity for multi-tier coordination evaluation. A2A-compliant trace capture enables reproducible assessment (0% variance) of multi-agent collaboration patterns.
Status: Coming Soon
Demo video (max 3 minutes) will cover: server startup, evaluation flow, and results interpretation.
- ✅ Phase 1: A2A + Graph + Basic eval (current)
- 🔜 Phase 2 (outlook): ART training on traces, potentially using WeightWatcher or PerforatedAI
- 🔮 Phase 3 (outlook): Self-evolving GreenAgent, e.g., DGM
Note: Time constraints limited full implementation of advanced features planned in Phase 2 and 3. Current release focuses on core graph-based evaluation with proven reproducibility.
docker pull ghcr.io/qte77/agentbeats-greenagent:latest
docker run -p 9009:9009 ghcr.io/qte77/agentbeats-greenagent:latest
curl localhost:9009/.well-known/agent.jsonSee docs/AgentBeats/QUICKSTART.md for local development, Docker build, and environment configuration.
| Document | Purpose |
|---|---|
| COMPETITION-ALIGNMENT.md | Alignment analysis with competition requirements |
| SUBMISSION-GUIDE.md | Phase 1 submission checklist |
| LIMITATIONS.md | Scope boundaries and deferred features |
| RESOURCES.md | External links and references |