ORCAI

Purpose

ORCAI is a multi-agent AI orchestration platform designed to solve complex problems that require expertise across multiple domains. Instead of relying on a single LLM to analyze all available information, ORCAI delegates tasks to specialized AI agents, each responsible for understanding a specific type of data. The results from these agents are then consolidated into a single, easy-to-understand response for the user.

Objectives

• Improve analysis accuracy by using specialized AI agents.

• Support multiple data types (logs, metrics, configurations, documents, etc.).

• Execute agents in parallel to reduce analysis time.

• Detect and recover from agent failures automatically.

• Provide users with a single summarized response.

Scope

In Scope

• AI planning

• Task orchestration

• Specialized AI agents

• Fault detection and recovery

• Response summarization

• Multi-file and multi-data-type analysis

Out of Scope

• Infrastructure provisioning

• Manual log analysis

• Agent model training

---

Problem Statement

Traditional LLMs struggle when:

• Multiple data sources must be analyzed together.

• Data exceeds the model's context window.

• Different datasets require domain-specific expertise.

• Long-running analysis must tolerate failures.

ORCAI addresses these limitations by orchestrating multiple specialized AI agents under a centralized execution framework.

---

High-Level Architecture

Components

User (Actor)

Initiates a request by providing a prompt and optional input data such as logs, metrics, or configuration files.

---

Planner

Responsibilities

• Receive user prompt and input files.

• Understand the user's objective.

• Determine which specialized agents are required.

• Generate an execution plan in JSON format.

• Send the execution plan to the Executor.

Inputs

• User Prompt

• Uploaded Data

Outputs

• Execution Plan (JSON)

---

Executor

Responsibilities

• Read the execution plan.

• Create and manage specialized agents.

• Assign work to each agent.

• Monitor execution progress.

• Collect agent outputs.

• Forward results to the LLM.

Inputs

• Planner Execution Plan

Outputs

• Aggregated Agent Results

---

Specialized Agents

Each agent is responsible for one data type or domain.

Examples:

• Linux Log Agent

• Database Agent

• Network Agent

• Kubernetes Agent

• Application Log Agent

• Security Agent

Responsibilities

• Analyze assigned dataset.

• Produce structured findings.

• Return results to the Executor.

---

Fault Terminal

Responsibilities

• Monitor agent health.

• Detect failures or timeouts.

• Restart failed agents.

• Notify the Executor when recovery is complete.

---

LLM Response Generator

Responsibilities

• Receive outputs from all agents.

• Correlate findings.

• Generate a single natural-language response.

• Present root cause analysis and recommendations.

---

End-to-End Workflow

1. User submits a prompt and uploads supporting data.

2. Planner analyzes the request.

3. Planner generates an execution plan.

4. Executor receives the plan.

5. Executor launches required specialized agents.

6. Each agent analyzes its assigned dataset.

7. Fault Terminal monitors all running agents.

8. Failed agents are restarted automatically.

9. Executor collects successful agent outputs.

10. LLM consolidates all findings.

11. Final response is returned to the user.

---

Use Cases

Use Case UC-001

Name

Server Failure Root Cause Analysis

Objective

Identify the root cause of a server outage by analyzing multiple datasets simultaneously.

Business Value

Reduce Mean Time to Resolution (MTTR) by automating cross-domain analysis.

Actors

• User

• Planner

• Executor

• Linux Log Agent

• Network Agent

• Database Agent

• Fault Terminal

• LLM

Preconditions

• User is authenticated.

• Input files are uploaded.

• Required agents are available.

• Executor service is running.

Trigger

User submits:

"Why did Production Server A go down yesterday?"

Inputs

• System Logs

• Application Logs

• Database Logs

• Network Logs

• User Prompt

Main Flow

1. User uploads all available logs.

2. Planner analyzes the request.

3. Planner identifies required agents.

4. Planner generates execution JSON.

5. Executor creates specialized agents.

6. Linux Agent analyzes system logs.

7. Database Agent analyzes database logs.

8. Network Agent analyzes network events.

9. Agents return structured findings.

10. Executor aggregates results.

11. LLM summarizes findings.

12. User receives the final report.

Alternative Flows

A1
Some logs are missing.

System notifies the user and continues analyzing available data.

A2
Only one dataset is uploaded.

Planner creates a single-agent execution plan.

Exception Flow

E1
Agent crashes.

Fault Terminal restarts the failed agent.

E2
Agent timeout.

Executor retries according to the retry policy.

E3
Planner cannot determine the file type.

System requests additional user input.

Postconditions

• Root cause report generated.

• Analysis stored.

• User receives recommendations.

Expected Output

Example:

Root Cause:
Database storage reached 100%, causing application failures.

Supporting Evidence:

• Linux Agent detected disk full.

• Database Agent detected write failures.

• Application Agent detected connection errors.

Recommendation:
Increase storage capacity and enable storage monitoring alerts.

---

Functional Requirements

ID	Requirement
FR-001	Accept user prompts.
FR-002	Accept multiple file uploads.
FR-003	Planner shall generate execution plans.
FR-004	Executor shall create specialized agents.
FR-005	Agents shall process assigned datasets independently.
FR-006	Executor shall collect all agent outputs.
FR-007	Fault Terminal shall restart failed agents automatically.
FR-008	LLM shall generate a unified response.

---

Non-Functional Requirements

Performance

• Parallel execution of agents.

• Support large datasets.

Reliability

• Automatic retry for failed agents.

• Fault tolerance.

Scalability

• Dynamically add new agent types.

• Execute multiple agents concurrently.

Security

• Secure file upload.

• Role-based access control.

• Encryption for data in transit.

Observability

• Execution logs.

• Agent metrics.

• Audit trail.

---

Assumptions

• Planner always returns valid execution JSON.

• All agents implement a common interface.

• Executor can communicate with all agents.

• LLM is available during execution.

---

Future Enhancements

• Dynamic agent discovery

• Agent marketplace

• Agent prioritization

• Multi-LLM support

• Human approval workflow

• Learning from previous executions

• Agent caching

• Distributed execution across multiple servers

Minimal POC

This repository contains a small FastAPI proof of concept for the ORCAI flow:

1. The user submits a prompt and one or more data artifacts.
2. A planner uses the OpenAI Python SDK Responses API to create a JSON plan.
3. An executor summons specialist agents through the OpenAI Agents Python SDK.
4. A fault terminal wraps agent execution with retry, circuit breaker, and fallback behavior.
5. A final LLM response summarizes the specialist findings into one answer.

The app runs in demo mode without credentials. Set OPENAI_API_KEY to use OpenAI-backed planner, agents, and summarization.

Run

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
export OPENAI_API_KEY="sk-..."
uvicorn app.main:app --reload

Open http://127.0.0.1:8000.

Useful endpoints

• GET / - Bootstrap UI
• GET /health - SDK and resilience library status
• POST /api/analyze - JSON API for planner, executor, agents, and final summary

Fault tolerance module

app/fault_tolerance.py mixes:

• tenacity for async retries with exponential jitter.
• aiobreaker for async circuit breaking.
• pybreaker as a synchronous circuit-breaker fallback.
• pyresilience as an optional resilience provider indicator, kept optional because package APIs differ across releases.