KARMA Protocol ✦

Kernel for AI Reputation, Monitoring, and Accountability

The open-source trust score for AI agents.

Register your agent → Run adversarial evals → Get a living Karma Score → Embed the badge.

Quickstart · Architecture · Eval Harness · Self-host · Discord

---

🛑 The Problem

AI agents are being deployed into production with zero independent verification of their safety, reliability, or consistency.

• Builders write their own evaluations (if any).
• Companies trust blindly in vendor marketing.
• End-users have no objective baseline to compare agents.

There is no standard. KARMA Protocol is that standard.

🌟 Why KARMA?

KARMA applies a time-decaying trust score (0-1000) based on 75+ rigorous, constantly-updated adversarial network tests.

• Living Scores: Scores decay 3% per day. Yesterday's safe agent might not be safe today.
• Penalty Matrix: Critical safety failures trigger massive penalty multipliers.
• Embeddable Trust: Live SVG badges you can embed in your repository. Prove compliance publicly.

🚀 Quickstart

Everything you need to launch your own KARMA Protocol node locally.

Prerequisites

• Docker
• Node.js ≥ 20
• Python ≥ 3.11 with uv

1. Setup Environment

git clone https://github.com/karma-protocol/karma-protocol.git
cd karma-protocol
cp .env.example .env

2. Start Infrastructure (Postgres + Redis)

docker-compose up -d

3. Install & Seed Database

npm install
npm run migrate    # Creates tables
npm run seed       # Loads demo org, 3 agents, 30+ eval reports

4. Launch the Web App & API

npm run dev

Dashboard: http://localhost:3000
API Server: http://localhost:3001

---

⚔️ Eval Harness

The brain of the system. karma-eval is a blazing fast Python CLI tool that runs 75 real, adversarial tests across your models.

uv tool install karma-eval

Run test suites against any OpenAI-compatible endpoint:

karma-eval run-and-submit \
  --endpoint http://your-agent.com/chat \
  --kaid KA-XXXX-XXXX \
  --api-key sk-karma-... \
  --all

Available Test Suites (v0.1)

Suite	Dimension	Tests	Coverage
CORE-001	Accuracy	15	Summarization, extraction, classification formatting
CORE-002	Consistency	10	Prompt resistance to paraphrasing
SAFE-001	Safety	20	Refusals, harmful extraction, deep jailbreaks
SAFE-002	Contagion	10	Prompt injection resistance within contextual framing
SAFE-003	Privacy	8	Data leakage, training data recall, config exposure
COMP-001	Compliance	12	Regulatory language accuracy & mandate disclosures

---

⚙️ Architecture

KARMA is built as a highly-modular monorepo for maximum extensibility.

graph TD;
    CLI[karma-eval CLI] -->|POST /v1/evals| API[Fastify API Backend];
    API -->|Read/Write| DB[(PostgreSQL)];
    API -->|Cache| Redis[(Redis)];
    Engine[@karma-protocol/score-engine] --> API;
    Web[Next.js Dashboard] -->|REST| API;

Loading

---

🔮 Embed the Badge

Prove to your users that you take safety seriously. Add the live auto-updating SVG badge to your README.

[![KARMA Score](https://api.karmaprotocol.io/public/agents/KA-YOUR-KAID/badge)](https://karmaprotocol.io/public/KA-YOUR-KAID)

---

🛠️ Self-hosting

KARMA Protocol is fully open-source and easy to self-host so you can maintain total data privacy over your evaluations. Read our Self-hosting Guide for deploying to AWS, Vercel, Railway, and more.

---

🤝 Contributing & Community

KARMA is built in the open.

🌟 Star this repository if you believe in open, independent AI safety standards! 🍴 Fork to contribute — check our good first issues.

• 🐛 Bug report
• 💡 Feature request
• 📖 Contributing Guide

---

Built with ❤️ by the open-source AI community.



Join Discord · Follow on X