Skip to content

feat(cookbook): add llamaindex-rag-tracing-eval example with traceAI Instrumentation - #10

Open
pm32900 wants to merge 1 commit into
future-agi:mainfrom
pm32900:feat/llamaindex-rag-tracing-eval
Open

feat(cookbook): add llamaindex-rag-tracing-eval example with traceAI Instrumentation#10
pm32900 wants to merge 1 commit into
future-agi:mainfrom
pm32900:feat/llamaindex-rag-tracing-eval

Conversation

@pm32900

@pm32900 pm32900 commented Jun 9, 2026

Copy link
Copy Markdown

Summary

Adds a new cookbook example: llamaindex-rag-tracing-eval.

This is a minimal, self-contained RAG pipeline that demonstrates:

  • Building a VectorStoreIndex from a local document using LlamaIndex
  • Tracing all LLM calls, retrieval steps, and token usage with traceAI-llamaindex
  • Attaching CONTEXT_ADHERENCE, CONTEXT_RELEVANCE, and COMPLETENESS eval tags so Future AGI automatically scores each response
  • Viewing full span trees and eval scores in the Future AGI dashboard

Motivation

The existing external_gdrive_search cookbook uses LlamaIndex but requires
Google Drive credentials, a database, and Gradio. There is currently no
minimal LlamaIndex RAG example that shows the tracing + eval pattern in
isolation. This fills that gap for developers who want to add observability
to a simple RAG pipeline before scaling to production.

Files Added

llamaindex-rag-tracing-eval/ ├── README.md ├── app.py ├── eval_tags.py ├── requirements.txt ├── .env.example └── data/ └── sample_policy.txt

Testing

Tested locally with Python 3.11. Run python app.py after filling in .env.
Traces and eval scores confirmed visible in the Future AGI dashboard.

Checklist

  • New folder only — no existing files modified
  • requirements.txt uses real published package names from PyPI
  • .env.example contains no real secrets
  • README includes prerequisites, install, run, expected output, and troubleshooting
  • Follows the same structure as multi_agent and external_gdrive_search

@pm32900

pm32900 commented Jun 30, 2026

Copy link
Copy Markdown
Author

Hi! Just checking in to see if there's anything I can update or clarify to help move this PR forward. Happy to make any requested changes. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant