Developer Tools Research Agent

An AI-powered agent that automates the research and analysis of developer tools. Provide a query, and the agent will identify relevant tools, gather detailed information, and generate a summary report with recommendations.

Features

• Natural Language Queries: Ask questions about developer tools in plain English.
• Automated Research: Scrapes articles and official websites to gather up-to-date information.
• In-Depth Analysis: Extracts key details for each tool, including:
  • Pricing Model
  • Open Source Status
  • Tech Stack
  • API Availability
  • Language Support
  • Integration Capabilities
• AI-Powered Recommendations: Generates a final analysis and recommendation based on the research.
• Detailed Reports: Saves a comprehensive report of the findings to a text file for future reference.

Agent Workflow

The agent operates in a multi-step process orchestrated by LangGraph to ensure a comprehensive and structured research process.

1. Tool Extraction:

   • The agent first refines the user's query to search for comparison articles and alternative lists (e.g., "best X alternatives").
   • It uses Firecrawl to find and scrape the content of the top search results.
   • An LLM call is then made to analyze the scraped text and extract a list of relevant tool names.

2. In-Depth Research:

   • For each extracted tool, the agent performs a targeted search to find its official website.
   • It scrapes the content of the official site to gather primary source information.
   • A structured LLM call analyzes the website content to populate key data points for each company, such as pricing, tech stack, and API availability. If this step fails, it gracefully handles the missing information.

3. Analysis & Recommendation:

   • All the gathered data on the researched tools is compiled.
   • A final LLM call synthesizes this information in the context of the original user query.
   • This step generates a final, human-readable analysis and provides a clear recommendation.

4. Report Generation:

   • The entire state of the research, including the extracted tools, detailed company profiles, and the final analysis, is formatted into a clean text report.
   • This report is saved to a timestamped .txt file in the project's root directory for persistent storage and easy access.

Tech Stack

• Python: The core programming language.
• LangChain & LangGraph: For building the AI workflow and orchestrating the agent's steps.
• OpenAI: Powers the language understanding and generation capabilities.
• Firecrawl: Used for web scraping and data extraction.
• Pydantic: For data validation and settings management.

Getting Started

Prerequisites

• Python 3.13+
• An OpenAI API key
• A Firecrawl API key

Installation

1. Clone the repository:

   git clone <repository-url>
   cd reaserch-agent

2. Install the dependencies:

   pip install -r requirements.txt

   (Note: If you are using uv, you can use uv pip install -r requirements.txt)

Configuration

1. Create a .env file in the root of the project.
2. Add your API keys to the .env file:

   OPENAI_API_KEY="your-openai-api-key"
   FIRECRAWL_API_KEY="your-firecrawl-api-key"
   

Running the Agent

Execute the main.py script to start the agent:

python main.py

Usage

Once the agent is running, you will be prompted to enter a query. Here are a few examples:

• What are some open-source alternatives to Retool?
• Find me low-code platforms for building internal tools.
• Compare monitoring solutions for a Kubernetes cluster.

The agent will then perform the research and print the results to the console. A detailed report will also be saved as a .txt file in the project's root directory.

Project Structure

.
├── .gitignore
├── main.py
├── pyproject.toml
├── README.md
├── research_results_...
├── src
│   ├── __init__.py
│   ├── firecrawl.py
│   ├── models.py
│   ├── prompts.py
│   └── workflow.py
└── uv.lock

• main.py: The entry point for the application. Handles user input and displays the final results.
• src/workflow.py: Defines the main research workflow using LangGraph. Orchestrates the different steps of the research process.
• src/models.py: Contains the Pydantic models used for structuring the data throughout the agent.
• src/prompts.py: Manages the prompts used to interact with the language model.
• src/firecrawl.py: A service class for interacting with the Firecrawl API.
• research_results_...: The output files generated by the agent.

Contributing

Contributions are welcome! If you have any ideas, suggestions, or bug reports, please open an issue or submit a pull request.

License

This project is licensed under the MIT License. See the LICENSE file for more details.