Add Agent Lightning showcase application by stephenspreetail · Pull Request #1 · stephenspreetail/test-5481

stephenspreetail · 2026-01-31T03:54:21Z

This commit adds a comprehensive showcase application demonstrating
Microsoft's Agent Lightning framework for training AI agents.

Features:

Sentiment analysis agent with functional and class-based approaches
Question answering agent with chain-of-thought reasoning
Math problem solver with step verification
Multi-agent pipeline demonstrating selective optimization
Sample datasets for training and validation
Grading utilities for computing rewards
5 example scripts showing different use cases
Comprehensive README with documentation

https://claude.ai/code/session_01ThrVrGQcWmzDpoF14Lyegv

This commit adds a comprehensive showcase application demonstrating Microsoft's Agent Lightning framework for training AI agents. Features: - Sentiment analysis agent with functional and class-based approaches - Question answering agent with chain-of-thought reasoning - Math problem solver with step verification - Multi-agent pipeline demonstrating selective optimization - Sample datasets for training and validation - Grading utilities for computing rewards - 5 example scripts showing different use cases - Comprehensive README with documentation https://claude.ai/code/session_01ThrVrGQcWmzDpoF14Lyegv

Copilot

Pull request overview

This pull request adds a comprehensive showcase application demonstrating Microsoft's Agent Lightning framework for training AI agents. The showcase includes multiple example implementations showing different aspects of the framework, from basic sentiment analysis to complex multi-agent pipelines.

Changes:

Added complete package structure with agents, datasets, and utilities modules
Implemented 5 runnable examples demonstrating various Agent Lightning features
Provided sample datasets and grading utilities for sentiment analysis, question answering, and math problem solving
Included comprehensive README with installation instructions, usage examples, and API reference

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
setup.py	Package configuration with dependencies and console script entry point
requirements.txt	Core dependencies for the showcase application
agent_lightning_showcase/init.py	Package initialization with version information
agent_lightning_showcase/agents/*.py	Agent implementations (sentiment, QA, math, multi-agent)
agent_lightning_showcase/datasets/sample_datasets.py	Training and validation datasets with TypedDict definitions
agent_lightning_showcase/utils/graders.py	Reward grading functions for evaluating agent outputs
agent_lightning_showcase/utils/prompt_templates.py	Baseline prompt templates for different agent types
examples/01_basic_sentiment_training.py	Basic sentiment classification training example
examples/02_qa_agent_training.py	Question answering with functional and class-based approaches
examples/03_math_solver_training.py	Math problem solver with verification capabilities
examples/04_multi_agent_pipeline.py	Multi-agent pipeline demonstrating selective optimization
examples/05_custom_store_and_strategy.py	Advanced configuration with custom stores and strategies
README.md	Comprehensive documentation with quick start guide and examples

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-31T04:00:16Z

+)
+from agent_lightning_showcase.utils.prompt_templates import (
+    SENTIMENT_BASELINE_PROMPT,
+    QA_BASELINE_PROMPT,


The imported QA_BASELINE_PROMPT is never used in this file. It should be removed to avoid confusion.

Suggested change

QA_BASELINE_PROMPT,

Copilot · 2026-01-31T04:00:16Z

+
+from agent_lightning_showcase.datasets import create_math_dataset
+from agent_lightning_showcase.agents.math_agent import MathAgent, math_solver
+from agent_lightning_showcase.utils.prompt_templates import MATH_BASELINE_PROMPT


The imported MATH_BASELINE_PROMPT is never used in this file. It should be removed to avoid confusion.

Suggested change

from agent_lightning_showcase.utils.prompt_templates import MATH_BASELINE_PROMPT

Copilot · 2026-01-31T04:00:17Z

+    formatted_prompt = prompt_template.format(text=task["text"])
+    response = prompt_template.complete(formatted_prompt)
+
+    reward = 1.0 if response.lower() == task["expected"].lower() else 0.0


The example code references task["expected"] but based on the SentimentTask TypedDict definition in agent_lightning_showcase/datasets/sample_datasets.py, the correct key is "expected_sentiment". This will cause a KeyError if someone tries to run this exact code.

Suggested change

reward = 1.0 if response.lower() == task["expected"].lower() else 0.0

reward = 1.0 if response.lower() == task["expected_sentiment"].lower() else 0.0

Copilot · 2026-01-31T04:00:17Z

+trainer = agl.Trainer(
+    algorithm=agl.APO(AsyncOpenAI()),
+    n_runners=4,
+    initial_resources={"prompt_template": my_prompt_template},


The example code references an undefined variable my_prompt_template. This should either be defined in the example or replaced with a more descriptive placeholder like create_prompt_template() to match the pattern used in the actual example files.

Copilot · 2026-01-31T04:00:17Z

+                )
+            )
+
+        reward = grade_answer(response, task["expected"])


The example code uses task["expected"] but based on the QATask TypedDict definition in the codebase, the correct key is "expected_answer". Additionally, the function grade_answer is undefined - it should likely be qa_grader from agent_lightning_showcase.utils.graders to match the actual implementation.

Copilot · 2026-01-31T04:00:17Z

+    entry_points={
+        "console_scripts": [
+            "agl-showcase=agent_lightning_showcase.cli:main",
+        ],
+    },


The setup.py defines a console script entry point 'agent_lightning_showcase.cli:main', but there is no cli.py module in the agent_lightning_showcase package. This will cause an error when the package is installed and someone tries to run the 'agl-showcase' command.

Suggested change

entry_points={

"console_scripts": [

"agl-showcase=agent_lightning_showcase.cli:main",

],

},

Copilot · 2026-01-31T04:00:17Z

+
+```bash
+# Clone the repository
+git clone https://github.com/your-repo/agent-lightning-showcase.git


The repository URL contains a placeholder "your-repo" which should be replaced with the actual repository owner/organization name before release.

Suggested change

git clone https://github.com/your-repo/agent-lightning-showcase.git

git clone https://github.com/microsoft/agent-lightning-showcase.git

Copilot · 2026-01-31T04:00:18Z

+    author="Your Name",
+    author_email="your.email@example.com",


The author and author_email fields contain placeholder values ("Your Name" and "your.email@example.com"). These should be updated with actual information before release, or removed if not applicable.

Suggested change

author="Your Name",

author_email="your.email@example.com",

Copilot · 2026-01-31T04:00:18Z

+    for label in valid_labels:
+        if label in prediction_clean:
+            predicted_label = label
+            break


The sentiment extraction logic uses a simple substring search which can lead to false positives. For example, if a response contains "not positive" or "negative" appears first in text like "negative sentiment is incorrect, it's positive", the wrong label could be extracted. Consider using regex word boundaries or extracting only the final/first complete word match.

Suggested change

for label in valid_labels:

if label in prediction_clean:

predicted_label = label

break

# Match only complete sentiment words and pick the last occurrence

pattern = r'\b(' + '|'.join(valid_labels) + r')\b'

matches = list(re.finditer(pattern, prediction_clean))

if matches:

predicted_label = matches[-1].group(1)

Copilot · 2026-01-31T04:00:18Z

+    openai_client = AsyncOpenAI()
+


Variable openai_client is not used.

Suggested change

openai_client = AsyncOpenAI()

stephenspreetail requested a review from Copilot January 31, 2026 03:55

Copilot started reviewing on behalf of stephenspreetail January 31, 2026 03:55 View session

Copilot AI reviewed Jan 31, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Agent Lightning showcase application#1

Add Agent Lightning showcase application#1
stephenspreetail wants to merge 1 commit into
mainfrom
claude/agent-lightning-showcase-app-6Ltpm

stephenspreetail commented Jan 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Copilot AI Jan 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	reward = 1.0 if response.lower() == task["expected"].lower() else 0.0
	reward = 1.0 if response.lower() == task["expected_sentiment"].lower() else 0.0

	git clone https://github.com/your-repo/agent-lightning-showcase.git
	git clone https://github.com/microsoft/agent-lightning-showcase.git

-    for label in valid_labels:
-        if label in prediction_clean:
-            predicted_label = label
-            break
+    # Match only complete sentiment words and pick the last occurrence
+    pattern = r'\b(' + '|'.join(valid_labels) + r')\b'
+    matches = list(re.finditer(pattern, prediction_clean))
+    if matches:
+        predicted_label = matches[-1].group(1)

Conversation

stephenspreetail commented Jan 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants