An orchestration backbone for AI pipelines.
Event-sourced orchestration for Fabric and beyond.
Quick Start • Why arkai? • Features • Docs
Building AI workflows today means:
- Results vanish after each chat session
- Pipelines fail with no way to resume
- No audit trail of what happened or why
- Spaghetti code as complexity grows
arkai gives your AI a spine, a Rust-based orchestration layer that:
- Remembers everything (event-sourced state)
- Recovers from failures (idempotent resume)
- Replays any operation (full audit trail)
- Scales without spaghetti (YAML pipelines)
# Install arkai
cargo install --git https://github.com/arkaigrowth/arkai
# Install Fabric (AI pattern library)
go install github.com/danielmiessler/fabric@latest
fabric --setup
# Ingest your first video
arkai ingest "https://youtube.com/watch?v=..." --tags "ai,learning"
# Search your library
arkai search "transformers"That's it. Your knowledge base grows with every ingest.
If Arkai picks the wrong fabric on your host, override it explicitly:
export ARKAI_FABRIC_BIN=/opt/homebrew/bin/fabric-ai
arkai doctor --jsonOr set it in .arkai/config.yaml:
fabric:
binary: /opt/homebrew/bin/fabric-aiVerify the selected binary with:
arkai doctor --json| Without arkai | With arkai |
|---|---|
| Results vanish after chat | Searchable library forever |
| Pipeline fails → start over | Resume from exact failure point |
| "What did the AI do?" 🤷 | Full event log, replay any step |
| Copy-paste prompt spaghetti | Composable YAML pipelines |
┌─────────────────────────────────────────────────────────────┐
│ NATURAL LANGUAGE │
│ (Claude Code + /arkai skill) │
│ │
│ "ingest this video" → arkai ingest "https://..." │
└─────────────────────────────┬───────────────────────────────┘
│
┌─────────────────────────────▼───────────────────────────────┐
│ RUST SPINE (arkai) │
│ Orchestration • State • Reliability │
│ │
│ ✓ Event-sourced ✓ Idempotent ✓ Auditable │
└─────────────────────────────┬───────────────────────────────┘
│
┌─────────────────────────────▼───────────────────────────────┐
│ FABRIC (patterns) │
│ AI Transformation • 200+ Prompts │
└─────────────────────────────┬───────────────────────────────┘
│
┌─────────────────────────────▼───────────────────────────────┐
│ LLM PROVIDER │
│ (Claude, GPT, Ollama, local models, etc.) │
└─────────────────────────────────────────────────────────────┘
📊 Mermaid: Data Flow
flowchart LR
subgraph Input
URL[URL/Text]
end
subgraph arkai [arkai Spine]
ES[Event Store]
ORCH[Orchestrator]
CAT[Catalog]
end
subgraph Fabric
PAT[Patterns]
end
subgraph Output
LIB[Library]
end
URL --> ORCH
ORCH --> ES
ORCH --> PAT
PAT --> ORCH
ORCH --> CAT
ORCH --> LIB
📊 Mermaid: Event Sourcing
sequenceDiagram
participant User
participant arkai
participant EventStore
participant Fabric
participant LLM
User->>arkai: ingest URL
arkai->>EventStore: RunStarted
arkai->>Fabric: fetch transcript
Fabric->>LLM: extract_wisdom
LLM-->>Fabric: wisdom.md
arkai->>EventStore: StepCompleted
arkai->>User: ✅ Content ingested
Note over EventStore: Full replay possible
# YouTube (auto-detected)
arkai ingest "https://youtube.com/watch?v=..." --tags "ai,ml"
# Web articles
arkai ingest "https://example.com/article" --tags "tech"# Analyze entire repositories that exceed context windows
# Uses a Recursive Language Model approach for context that exceeds the window
# In Claude Code, RLM tools are available as MCP:
rlm_load_context # Load files as external variables
rlm_filter_context # Regex-based filtering (deterministic)
rlm_sub_query # LLM call on chunks (budgeted)
rlm_exec # Sandboxed Python execution (HITL)arkai library # List all
arkai library --content-type youtube # Filter
arkai search "transformer" # Full-text search
arkai show <id> --full # View contentarkai run my-pipeline # Execute pipeline
arkai status <run_id> # Check status
arkai resume <run_id> # Resume failed runarkai config # Show resolved paths
arkai runs # List recent runsyour-project/
├── .arkai/
│ ├── config.yaml # Project config
│ ├── catalog.json # Searchable index
│ └── runs/ # Event logs (gitignore)
│
├── library/ # Knowledge base (git-track!)
│ ├── youtube/
│ └── articles/
│
└── pipelines/ # Custom workflows
└── my-workflow.yaml
name: youtube-wisdom
description: Extract wisdom from YouTube videos
safety_limits:
max_steps: 10
step_timeout_seconds: 300
steps:
- name: fetch
action: __youtube__
input_from: pipeline_input
- name: wisdom
action: extract_wisdom
input_from: fetch
- name: summary
action: summarize
input_from: wisdom| Feature | Raw LLM | LangChain | Fabric | arkai + Fabric |
|---|---|---|---|---|
| Persistent state | ❌ | ❌ | ✅ | |
| Resume failed runs | ❌ | ❌ | ❌ | ✅ |
| Full audit trail | ❌ | ❌ | ❌ | ✅ |
| 200+ AI patterns | ❌ | ✅ | ✅ | |
| Content library | ❌ | ❌ | ❌ | ✅ |
| Single binary | ❌ | ❌ | ✅ | ✅ |
| Massive context (RLM) | ❌ | ❌ | ✅ | |
| Evidence provenance | ❌ | ❌ | ❌ | ✅ |
- AI OS Architecture: Full philosophy and design
- Pitch: Quick shareable summary
- Architecture Overview: Technical deep-dive
- Early-stage and under active development. Commands and interfaces may change.
- Developed on macOS. Linux and Windows are not tested, and voice capture relies on macOS-specific paths.
- Requires an external Fabric install and your own LLM API key.
- The RLM (large-context) integration depends on a separate RLM server and is optional.
- Some items in the docs and roadmap describe planned or in-progress work rather than shipped features.
MIT. Use it, fork it, build on it.
Built with 🦀 Rust + 🧵 Fabric
"A really smart AI with a bad system is way worse than a well-designed system with a less smart model."