IntelAgent: A Privacy-First AI Tool for Journalists and Researchers

IntelAgent is a local, privacy-first AI-powered assistant engineered for complex, multi-step tasks that are executed by an original agentic harness. Designed specifically for investigative journalists and scientific researchers, the architecture prioritizes data sovereignty and operational security. To support professionals in critical environments, the application features an interface available in English, as well as the native languages of two fragile democracies where journalists and researchers face immense pressure: Turkey and Hungary. By utilizing local large language models (LLMs) and open-weights alternatives, the system mitigates the privacy risks associated with proprietary, cloud-based application programming interfaces (APIs). Furthermore, this app democratizes access to programming; the user can create and execute Python functions by simply chatting with the bot to conduct data mining, analysis and visualization.

⚠️ Security Notice — Local, Single-User Tool

IntelAgent is intended to be run as a local, single-user tool on a machine you control and trust. By design, the agent can run arbitrary shell commands, write and execute Python code, install packages, and (optionally) control your mouse, keyboard, and open desktop applications. These are intended capabilities — like a terminal, they are powerful by nature.

Because of this, please observe the following:

• Run it locally for yourself only. Do not expose the Streamlit server to a public network or share an instance with untrusted users. The app ships with no authentication.
• Treat external content as untrusted. When the agent reads web pages, PDFs, or RSS feeds, that content could contain prompt-injection instructions. Review what an autonomous run is doing, especially before enabling high-power skills (LOCAL_SEARCHER, DATA_ANALYST, COMPUTER_USE).
• Keep your secrets out of git. Your real API keys live in .env (and Google tokens in skills/*/token.json); both are excluded by .gitignore. Never force-add them.
• Remote LLMs send data off-machine. If you enable OpenRouter, prompts and context (which may include file contents) are transmitted to a third party. For sensitive material, prefer the local model.

See SECURITY.md for the full threat model and reporting guidance.

Initialization and Configuration

1. Install Dependencies

Before running the application, make sure to install the required Python packages. Execute the following command in your terminal:

pip install -r requirements.txt

2. Environment Variables (.env)

Users must configure a .env file in the root directory. To do this, duplicate the .env.example file and rename it to .env. Set the PROJECTS_DIR variable to define the agent's primary operational environment. If omitted, the system defaults to the application's parent directory.

3. Run the Application

To initiate the agent, execute the following command in your terminal:

streamlit run app.py

The repository includes a batch (.bat) file for automated startup. Upon initialization, the user interface prompts the selection of either a local LM Studio model or a model on OpenRouter, an LLM platform.

Repository Configuration Note: The agent_memory, agent_workspace, and agent_references directories are excluded from version control to maintain a lightweight repository. The application generates these directories automatically during the initial execution.

Users must configure a .env file in the root directory. Set the PROJECTS_DIR variable to define the agent's primary operational environment. If omitted, the system defaults to the application's parent directory.

# Local Model Configuration
LOCAL_MODEL_NAME=holo3-35b-a3b

# System Directories and API Keys
PROJECTS_DIR=C:\path\to\your\projects
OPENROUTER_KEY=your_openrouter_api_key
BRAVE_API_KEY=your_brave_search_key
S2_API_KEY=your_semantic_scholar_api_key
GITHUB_PERSONAL_ACCESS_TOKEN=your_github_token

# Optional: Corporate Integrations
GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret

Model Selection and Data Sovereignty

The system architecture encourages the use of local LLMs, which is critical when processing sensitive journalistic or research data. IntelAgent serves as a secure alternative to popular agentic assistants such as OpenClaw, which presents documented security vulnerabilities acknowledged even by its own development team. The default local configuration utilizes the holo3-35b-a3b open-weights model via an LM Studio server, offering high capability on mid-tier hardware. Executing it locally ensures that all data processing remains strictly confined to the user's hardware by default, eliminating unauthorized external data transmission. Users can substitute this with any compatible local model.

For tasks requiring greater computational power without local hardware constraints, the system supports OpenRouter, a platform that enables to use any LLM through an API. The default OpenRouter selection in the app is Mistral Large, an Apache-2.0 licensed, open-weights architecture developed in Europe. This provides a robust alternative to foundational models originating from the United States and China, aligning with the project's emphasis on open-source technology.

Corporate Integrations (Google Workspace)

While the developer strongly advocates for open-source software to ensure maximum privacy, IntelAgent permits integration with corporate services like the U.S.-based Google Workspace (Docs, Sheets, Drive, Calendar) for users who require them. To prevent data leakage, the repository's .gitignore file strictly excludes authentication tokens (token.json, credentials.json).

To configure these optional integrations:

1. Access the Google Cloud Console.
2. Generate a new project and enable the necessary APIs.
3. Establish an OAuth Client ID configured as a Desktop or Web application.
4. Input the GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET into the .env file.

The initial execution of a Google-dependent skill triggers a secure, localized OAuth authentication process, generating an untracked token file within the respective skill directory.

System Architecture

The application operates on a dynamic, modular architecture. An orchestration engine evaluates user prompts and routes tasks to specialized sub-agents.

+------------------------------------------------------------------------------+
|                            Streamlit UI (app.py)                             |
|    [ Model Selection ]  [ Memory Controls ]  [ Chat ]  [ Stop Execution ]    |
+-------------------------------------+----------------------------------------+
                                      |
                                      v
+------------------------------------------------------------------------------+
|                             Orchestration Engine                             |
|  1. Evaluates request (Manager LLM: local LM Studio or OpenRouter)           |
|  2. Retrieves context (Hybrid Memory: ChromaDB + BM25/FTS5, RRF + MMR)       |
|  3. Routes to the next best skill (history-aware; optional Deep Planning)    |
+----------------+---------------------------------------------+---------------+
                 |                                             |
                 v                                             v
+----------------+----------------+        +-------------------+--------------+
|          Memory & RAG           |        |         Agent Workspace          |
|  - ChromaDB store (history,     |        |  - agent_workspace/sandbox       |
|    prior task context)          |        |    (uploaded inputs / outputs)   |
|  - Hybrid keyword index (FTS5)  |        |  - timestamped run logs          |
|  - agent_references/ (LIBRARIAN |        |  - agent-written scripts & files |
|    RAG documents)               |        |                                  |
+---------------------------------+        +----------------------------------+
                 ^                                             ^
                 |                                             |
                 +---------------------+-----------------------+
                                       |
                                       v
+------------------------------------------------------------------------------+
|                          MCP Skills Hub  (skills/*)                          |
|             stdio (local Python / Node) + SSE (remote) servers               |
|                                                                              |
|  Web & Research           Academic & Library        Documents & Data         |
|  - WEB_SEARCHER           - ACADEMIC_SEARCHER        - DATA_ANALYST          |
|  - DEEP_RESEARCH          - LITERATURE_REVIEW_HELPER - DOCUMENT_SPECIALIST   |
|  - ADVANCED_WEB_SCRAPER   - LIBRARIAN                - LITEPARSE             |
|  - NEWS_DISCOVERY                                                            |
|                                                                              |
|  Writing & Legal          System & Automation        Integrations            |
|  - NEWSROOM_EDITOR        - LOCAL_SEARCHER (shell)   - GITHUB_MANAGER        |
|  - FOI_REQUEST_WRITER     - COMPUTER_USE (desktop)   - GOOGLE_WORKSPACE      |
|  - TURKISH_LEGAL_EXPERT   - REPAIR_AGENT                                     |
|                           - ENV_MANAGER                                      |
+------------------------------------------------------------------------------+
                                       |
                                       v
+------------------------------------------------------------------------------+
|                        Execution & Output Generation                         |
|  - Runs the selected tool calls (MCP servers) within the agent workspace.     |
|  - Final answer & execution trace are streamed to the UI and saved to memory. |
+------------------------------------------------------------------------------+

Integrated Skills

The system includes 19 default skills loaded dynamically from the skills/ directory.

Skill	Primary Function	Core Technology / Methodology
Academic Searcher	Synthesizes academic literature with APA citations.	Semantic Scholar API + Consensus MCP
Advanced Web Scraper	Extracts structured datasets and dynamic HTML.	Puppeteer, BeautifulSoup
Computer Use	Controls mouse, keyboard, and live desktop applications to automate GUI tasks.	pyautogui, mss, Playwright, win32com
Data Analyst	Executes statistical analyses and generates visualizations.	Python (Pandas, Seaborn, Plotly)
Deep Research	Conducts multi-step autonomous investigations.	Brave Search, Semantic Scholar
Document Specialist	Formats and manipulates office documents and PDFs; regex PDF search.	Python Document Libraries, PyMuPDF
Environment Manager	Handles .env variables securely.	Local File System
FoI Request Writer	Drafts more effective Freedom of Information requests by utilizing best practices from the International Press Institute (IPI).	Legal text synthesis
GitHub Manager	Oversees repository workflows and pull requests, if the user is on GitHub.	GitHub MCP
Google Workspace Manager	Integrates Drive, Docs, Sheets, and Calendar tasks.	Google APIs (Optional)
Librarian	Queries indexed reference documents locally with semantic search.	sentence-transformers
LiteParse	Parses complex PDFs locally with spatial bounding boxes and OCR.	Node.js (LiteParse CLI)
Literature Review Helper	Automates gathering, categorizing, and synthesizing academic papers into a Word literature guide.	Consensus + Semantic Scholar, python-docx
Local File Manager	Manages local file system operations and runs terminal commands for system tasks.	OS Libraries, custom Terminal MCP
News Discovery	Retrieves and analyzes recent news from curated RSS feeds.	RSS / XML Parsing
Newsroom Editor	Copyedits journalistic texts according to configurable newsroom guidelines, defaulting to AP style.	Text Processing
Repair Agent	Diagnoses and fixes errors, missing files, and sandbox/terminal issues from previous runs.	Diagnostics + Terminal MCP
Turkish Legal Expert	Searches and retrieves Turkish legislation and case law.	mevzuat-mcp, yargi-mcp
Web Searcher	Executes general internet queries.	Brave Search API

💡 Best Practice for Bulk Data Parsing

The agent comes equipped with an Advanced Web Scraper that uses interactive Puppeteer tools to browse the web step-by-step. While this is great for exploring a few pages or bypassing simple Javascript walls, it is not designed for bulk parsing (e.g., iterating through thousands of pagination pages or downloading hundreds of PDFs). If you try to do this, the agent will likely exhaust its context window or hit a timeout.

For complex or bulk parsing tasks, specifically instruct the agent to write a standalone Python script instead:

❌ Bad Prompt: "Go to [website], click through all 4,000 pages of pagination, and download every PDF." (This triggers the interactive scraper and will fail).

✅ Good Prompt: "Write a standalone Python scraping script using playwright (or requests and BeautifulSoup) to iterate through 400 pages on [website] and download the PDFs to a local folder. Do not use your interactive browser tools. Use write_file to save the script to the sandbox, and I will run it myself."

Advanced Features & Enhancements

Beyond the core orchestration loop, IntelAgent ships with a number of enhancements that improve its autonomy, accuracy, and reach.

Daily Start-up Routine

When the application loads and the server is ready, the chatbot greets the user and asks: "Would you like a summary of the day?". If the user responds with an affirmative keyword (e.g., "yes", "y", "evet"), the system intercepts the response and assigns a predefined startup_prompt. This comprehensively tasks the required agents to:

• Check the Google Calendar for the next three upcoming events.
• Check Google Tasks for the next three upcoming tasks.
• Summarize the latest news using the News Discovery skill.
• Provide a weather forecast for today and tomorrow in Istanbul via Web Searcher.
• Summarize the top 3 most interesting recent academic articles from top communication and media journals (via the Academic Searcher's RSS capability).
• Output an inspirational quote from Marcus Aurelius, Immanuel Kant, or Viktor Frankl.

Note on Google Workspace Integration (Calendar & Tasks): If you encounter permission restrictions when accessing Google Tasks or adding new Google services, you must: (1) delete the existing token.json file in skills/GOOGLE_WORKSPACE/ to clear outdated cached credentials; (2) ensure the Google Tasks API is enabled in your Google Cloud Console project (APIs & Services > Library > "Google Tasks API"); (3) restart the application and complete the newly launched authentication flow in your browser to grant the new scope access.

Terminal Access for Local File Manager

The Local File Manager agent (LOCAL_SEARCHER) is backed by a custom standalone MCP server (terminal_mcp.py) that executes operating-system commands directly from the workflow. Because it interfaces with the host's shell, it acts as a highly capable gateway to practically any task you could run manually in a terminal. Key capabilities include:

• System Diagnostics & Monitoring: disk space, CPU/memory of running processes, network pings.
• Advanced PowerShell Access: defaults to cmd.exe on Windows, but unlocks full PowerShell (object manipulation, registry, WMI/CIM) by prefixing with powershell -Command or pwsh -Command.
• File Management & Batch Operations: searching for large files, bulk renaming, .zip archiving, modifying permissions.
• Source Control (Git): init, status, pulls, history analysis, branch management within the workspace.
• Software & Package Management: installing/upgrading dependencies via pip, npm, or apt.
• Environment Initialization: bootstrapping local dev servers (Flask, FastAPI, React) or orchestrating Docker containers.
• Executing Code & Tests: running scripts, pytest, or code formatters.
• Log Analysis: tailing logs dynamically or filtering large chunks with findstr/grep.

⚠️ This terminal capability is powerful and intentionally unrestricted. See the Security Notice — run IntelAgent only on a machine and account you trust.

Academic and Scientific Rigor Prompting

The main orchestration system prompt enforces a higher standard of information reliability. The agent is instructed to: write with scientific rigor using only verified and reputable sources; cite sources in APA style with a bibliography for academic requests; credit sources and provide direct links for news stories; and ask clarifying questions to fully understand intent, prioritizing clarity, critical reasoning, and factual accuracy.

Import External Memories (Cross-Chatbot State Transfer)

Users switching from other generative AI platforms (ChatGPT, Claude, etc.) can port over stored memory states. An "Import External Memories" panel under "Memory & Context" in the sidebar accepts raw text memory exports.

• Automated Chunking: the text is split structurally by categorical headers (## Instructions, ## Identity, ## Projects, etc.).
• Long-Term Chroma Injection: chunks are embedded directly into the research_memory vector database so guidelines, tonal preferences, and project history immediately influence future interactions.
• Reference Document Backup: a raw markdown file of the full profile is also written to agent_references/, which can then be indexed via the Librarian (RAG) agent.

Deep Planning Orchestrator Mode

For complex tasks requiring strategy and sequential reasoning, a "Deep Planning" mode is available.

• Settings Toggle: a "Deep Planning Settings" panel in the sidebar toggles it on/off. By default it uses a highly capable model via OpenRouter (e.g., anthropic/claude-opus-4.6).
• Phased Decomposition: the task is intercepted by run_deep_planning_flow, and the Orchestrator LLM breaks the request into a detailed JSON schema of sequenced steps, sub-agent instructions, and expected outcomes.
• Progressive Execution: the main loop sequentially invokes specialized sub-agents for each step.
• Context Accumulation: each completed step produces a brief summary and guidance that is passed into the next sub-agent's prompt, ensuring cohesive continuation.

Stronger Argumentation for the FoI Agent

The FoI Agent's instructions have been reinforced with Turkish case law, citing the relevant BEDK verdicts to pre-counter common rejection justifications.

Hybrid Search Memory (Vector + BM25 + MMR + RRF)

The memory architecture was upgraded to a modern hybrid-search engine (hybrid_memory.py) operating alongside ChromaDB to eliminate the keyword/semantic gap of pure vector search.

• Reciprocal Rank Fusion (RRF): merges semantic (ChromaDB) results with exact keyword matching via SQLite FTS5, locking onto specific IDs, error codes, and proper nouns while still pulling conceptual memories.
• Maximal Marginal Relevance (MMR): re-ranks candidates (λ=0.8) to penalize redundancy and inject a diverse, token-efficient snapshot into context.
• Temporal Memory Decay: a chronological exponent downscales aging memories, favoring recent procedural continuity.
• Synchronized Data Engineering: new memories are indexed in both stores instantly, and "🧹 Clear All Memory & Context" tears down both holistically so no ghost data remains.

Comprehensive Literature Review Engine

The LITERATURE_REVIEW_HELPER skill automates gathering, categorizing, and synthesizing academic papers into comprehensive Microsoft Word (.docx) literature guides.

• PICO Framework Integration: analyzes requests using the PICO (Population, Intervention, Comparison, Outcome) or SPIDER framework to structure sub-queries.
• Dynamic Research Pacing: evaluates Consensus rate limits in real time, executing targeted searches across "Quick Scan," "Standard Review," and "Deep Dive" depths chosen by the user.
• Python-docx Synthesis: compiles a structured primer with chronological evolution tables, thematic reading lists, methodological gap assessments, boolean query cheat sheets, and linked bibliographies into the workspace export directory.

Dual-Engine Academic Search (Consensus + Semantic Scholar)

Both ACADEMIC_SEARCHER and LITERATURE_REVIEW_HELPER use a dual-engine paradigm combining a custom Python Semantic Scholar client with the official Consensus MCP Server.

• Flexible Tier Compatibility: Consensus usage limits depend on your subscription, but the integration is fully functional even on the free tier.
• Primary vs. Fallback Reliability: connects to Consensus via Server-Sent Events (SSE) for AI-digested queries, transparently falling back to local Semantic Scholar (academic_searcher_mcp.py) on rate-limit or timeout.
• Strict Network Error-Handling: the Consensus initialize() call is isolated with asyncio.wait_for, so a stalled endpoint unloads gracefully without freezing the Streamlit interface.
• Expanded Search Paradigms: leverages SCImago Journal Rank (Q1–Q4) quality filtering and year bounds via conversational queries before searching.

Advanced Local PDF Parsing (LiteParse)

The LITEPARSE skill is a local alternative for problematic PDFs that standard text extraction fails on.

• Spatial Bounding Boxes: processes multi-column layouts, dense tables, and charts with pixel precision, returning structured JSON or Markdown.
• Embedded Local OCR: uses Tesseract.js to process scanned images and handwritten text completely offline.
• Visual Intelligence via Screenshots: generates high-DPI page screenshots on demand for visual LLMs when text alone fails.
• Zero Cloud Dependencies: a standalone OSS solution via Node.js (npx @llamaindex/liteparse) so raw documents never leave the host.

Turkish Legal Expert Skill

The TURKISH_LEGAL_EXPERT skill is a specialized legal research assistant powered by two MCP servers:

• mevzuat-mcp: interfaces with mevzuat.gov.tr and bedesten.adalet.gov.tr to search and retrieve full texts of laws, decrees, and regulations.
• yargi-mcp: connects to major Turkish legal databases (Yargıtay, Danıştay, Anayasa Mahkemesi, Emsal, etc.) for case law and judicial decisions.

It uses uvx to execute these servers, prioritizes fetching actual laws over generalized AI responses, and can cite articles and decisions accurately in Turkish.

Robust PDF Search & Discovery

The DOCUMENT_SPECIALIST skill includes a search_pdf_text tool to solve hallucination and context-dropping when reading large PDFs.

• Regex-Based Phrase Matching: an r'\s+'.join(words) pattern ignores odd whitespace, line-wraps, and carriage returns that break standard string matching.
• Context-Aware Extraction: returns each match with a 300-character buffer of surrounding text.
• PyMuPDF (fitz) Backend: high-performance local indexing and extraction.

Cross-Agent Orchestration Feedback

The central run_agent loop handles "domain jumping" — when an agent mistakenly calls another expert's tool. Instead of failing silently, the system injects a structured message explaining which tools the agent has access to and instructs it to write_file to save progress before stopping, preventing infinite loops and enabling clean hand-offs.

History-Aware Routing

The Manager Agent (route_task) is a context-aware router: it injects the last 5 conversation messages into its prompt so that, for example, after WEB_SEARCHER finishes, the Manager can intelligently route the next step to DATA_ANALYST or DOCUMENT_SPECIALIST with full awareness of preceding results.

Hybrid Computer Use Architecture

The COMPUTER_USE sub-agent performs tasks directly on the user's desktop using a hybrid of visual GUI automation and programmatic APIs.

• Hybrid Interaction Model: visual coordinate mapping (pyautogui for input, mss for screen capture) combined with programmatic APIs for reliability — Playwright for DOM-based web navigation and win32com.client to manipulate live, open Windows applications (e.g., PowerPoint, Photoshop).
• Strict Agent Boundaries: DOCUMENT_SPECIALIST is scoped to creating/reading files on disk, while COMPUTER_USE is exclusively for the live desktop and open applications.
• "Sense → Plan → Act → Verify" Cycle: after each action, a screen capture verifies the UI changed as expected before proceeding.
• Amnesia Protocol (Visual Memory Pruning): older base64 screenshots are stripped from history (replaced by a [Previous Screen State Removed] placeholder), keeping only the most recent screen in context.
• User Confirmation Flow: before activation, the user is prompted to confirm a vision-capable model is running on local LM Studio.

⚠️ Computer Use grants the agent control of your mouse, keyboard, and open applications. Use it deliberately and supervise active runs. See the Security Notice.

Settings Sidebar Quality of Life

• Infinite Iterations Toggle: flips max_iterations to float('inf') so the agent can run indefinitely with stable progress messaging.
• Danger Zone Placement: "Clear All Memory & Context" lives at the bottom under a dedicated danger heading as a red primary button.
• Document Workflow Grouping: "Upload Documents" sits directly above the "Library (References)" metrics and "Index References" action; "Import External Memories" was moved lower as it is rarely used.

Extending Capabilities

Users can engineer new sub-agents without modifying the core orchestration code (app.py). To integrate a new skill:

1. Construct a dedicated directory within the skills/ folder.
2. Formulate a manifest.json file defining the metadata, routing description, and Model Context Protocol (MCP) server arguments.
3. Draft an instructions.md file establishing the strict parameters and rules for the language model utilizing the skill.
4. Restart the application. The orchestrator will detect the new directory and integrate it into the routing logic.

Settings & Data Management

The sidebar in IntelAgent's UI provides several tools to manage the agent's memory, files, and reference data. While some of these "cleaning" actions may seem similar, they serve distinctly different purposes in managing the agent's local environment:

• Clear Memory: IntelAgent continuously learns by saving the reasoning and output of every successful task you give it into a local vector database (Chroma). This allows the agent to recall past context in future conversations automatically. Clicking "Clear Memory" resets this database, wiping the agent's contextual history of your past tasks without disabling its core capabilities.
• Clean Sandbox: The agent performs its active work in a temporary "sandbox" folder (agent_workspace/sandbox/inputs and outputs). This button lets you selectively delete the files you uploaded (Inputs) or the files the agent generated (Outputs) for your current task. This is useful for resetting the agent's immediate working directory before starting a new data analysis or document formatting job.
• Clean Old Workspaces: Every time you run a task, the agent leaves behind an execution log within a timestamped backup folder (e.g., agent_workspace/2026-03-12.../). Over time, these archived logs can take up disk space. Clicking "Clean Old Workspaces" deletes all historical timestamped workspace folders to clear up storage, leaving your current sandbox and memory intact.
• Index References: If you upload documents (PDFs, TXTs, or Markdown) to the "Library (Reference Documents)" target in the UI, they go into the agent_references/ folder. Clicking "Index References" parses these files and generates deep local embeddings (sentence-transformers), allowing the "Librarian" sub-agent to query your private documents securely via Retrieval-Augmented Generation (RAG) without relying on an external search engine.

Security

IntelAgent is designed as a local, single-user tool. For the full threat model, the security expectations for running and extending the app, and how to report a vulnerability, see SECURITY.md.

License

This project is open-sourced under the Mozilla Public License 2.0.