Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
{
"name": "microsoft-365-agents-toolkit",
"description": "Toolkit for building and evaluating Microsoft 365 Copilot declarative agents \u2014 scaffolding, JSON manifest development, capability configuration, and eval workflows.",
"version": "1.3.1",
"version": "1.5.0",
"source": "./plugins/microsoft-365-agents-toolkit"
},
{
Expand Down
2 changes: 1 addition & 1 deletion marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"name": "microsoft-365-agents-toolkit",
"source": "./plugins/microsoft-365-agents-toolkit",
"description": "Toolkit for building and evaluating Microsoft 365 Copilot declarative agents \u2014 scaffolding, JSON manifest development, capability configuration, and eval workflows.",
"version": "1.3.1"
"version": "1.5.0"
},
{
"name": "workiq-productivity",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "microsoft-365-agents-toolkit",
"version": "1.3.1",
"version": "1.5.0",
"description": "Toolkit for building and evaluating Microsoft 365 Copilot declarative agents \u2014 scaffolding, JSON manifest development, capability configuration, and eval workflows.",
"skills": "./skills/"
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "microsoft-365-agents-toolkit",
"version": "1.3.1",
"version": "1.5.0",
"description": "Toolkit for building and evaluating Microsoft 365 Copilot declarative agents \u2014 scaffolding, JSON manifest development, capability configuration, and eval workflows.",
"skills": "./skills/"
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "microsoft-365-agents-toolkit",
"version": "1.3.1",
"version": "1.5.0",
"description": "Toolkit for building and evaluating Microsoft 365 Copilot declarative agents \u2014 scaffolding, JSON manifest development, capability configuration, and eval workflows.",
"skills": "./skills/"
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ description: >
"configure my agent", "deploy my agent", "fix my agent manifest", "edit my agent",
"localize my agent", "add localization", "translate my agent", "multi-language agent",
"add an API plugin", "add an MCP plugin", "add OAuth to my plugin",
"add email actions to agent", "add meeting actions to agent",
"add editorial answers to agent", "set agent response mode",
"review instructions", "improve instructions", "fix my instructions"
---

Expand Down Expand Up @@ -67,7 +69,7 @@ When you encounter ANY problem (missing files, malformed JSON, validation errors
- Missing `declarativeAgent.json` → Detect (file not found) → Inform ("no manifest found") → Ask ("would you like to create a new agent?")
- Malformed JSON → Detect (parse errors) → Inform (list specific syntax issues) → Ask ("should I fix these syntax errors?")
- Validation errors → Detect (parse and check manifest) → Inform (list all errors) → Ask ("how would you like to fix these?")
- Version incompatibility → Detect (feature requires newer version) → Inform ("this feature requires v1.6, your agent is v1.4") → Ask ("should I upgrade?")
- Version incompatibility → Detect (feature requires newer version) → Inform ("this feature requires v1.8, your agent is v1.7") → Ask ("should I upgrade?")

---

Expand All @@ -80,6 +82,10 @@ When you encounter ANY problem (missing files, malformed JSON, validation errors
| Adding an API plugin | [API Plugins](references/api-plugins.md) |
| Adding an MCP server | [MCP Plugin](references/mcp-plugin.md) |
| Adding OAuth to an MCP or API plugin | [Authentication](references/authentication.md) |
| Adding email write operations (`EmailActions`) | [Editing Workflow](references/editing-workflow.md) |
| Adding meeting or calendar write operations (`MeetingActions`) | [Editing Workflow](references/editing-workflow.md) |
| Adding predefined semantic Q&A (`editorial_answers`) | [Editing Workflow](references/editing-workflow.md) |
| Choosing the default response mode | [Conversation Design](references/conversation-design.md) |
| Reviewing or improving existing agent instructions | [Instruction Review](references/instruction-review.md) |
| User reports agent gives generic/wrong answers | [Instruction Review](references/instruction-review.md) |
| Localizing an agent into multiple languages | [Localization](references/localization.md) |
Expand All @@ -88,6 +94,21 @@ When you encounter ANY problem (missing files, malformed JSON, validation errors

---

## Build-Time Feature Discovery

When creating or designing an agent, actively evaluate these features instead of waiting for the user to name their manifest identifiers:

| Requirement signal | Feature to surface |
|--------------------|--------------------|
| Send, archive, flag, move, delete, or organize email; manage rules, auto-replies, or folders | `EmailActions` |
| Schedule meetings, create calendar events or time-finding polls, or provide time insights | `MeetingActions` |
| Return approved, stable answers for semantically similar FAQ-style questions | `editorial_answers` |
| Prefer fast replies or deeper reasoning by default | `behavior_overrides.default_response_mode` |

Surface only relevant options during requirements discovery. Do not add them without clear user intent. For write capabilities, distinguish them from the read-only `Email` and `Meetings` capabilities and include confirmation guidance for sensitive operations.

---

## ATK CLI Setup

Before running any ATK commands, check if the ATK CLI is available by running `npx -y --package @microsoft/m365agentstoolkit-cli atk --version`. If not found, **STOP and tell the user** — do NOT attempt to install it yourself.
Expand Down Expand Up @@ -133,7 +154,9 @@ Then read `M365_TITLE_ID` from `env/.env.local` and **ALWAYS** present the revie
Before adding ANY feature, read the `version` field in `declarativeAgent.json` and check the [Schema](references/schema.md) feature matrix. If the feature isn't supported in that version, **refuse** and offer to upgrade.

Key version gates:
- `sensitivity_label`, `worker_agents`, `EmbeddedKnowledge` → **v1.6 only**
- `EmailActions`, `MeetingActions` → **v1.8+**
- `editorial_answers`, `behavior_overrides.default_response_mode`, `conversation_starters[].depends_on` → **v1.7+**
- `sensitivity_label`, `worker_agents`, `EmbeddedKnowledge`, `user_overrides` → **v1.6+**
- `Meetings` → **v1.5+**
- `ScenarioModels`, `behavior_overrides`, `disclaimer` → **v1.4+**
- `Dataverse`, `TeamsMessages`, `Email`, `People` → **v1.3+**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -693,7 +693,7 @@ Reference the API plugin in your `declarativeAgent.json`:

```json
{
"version": "v1.6",
"version": "v1.8",
"name": "Repairs Agent",
"description": "An agent to manage repair tickets",
"instructions": "You help users manage repair tickets. Use the repairs API to search, create, update, and delete repairs.",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Follow these best practices for successful M365 Copilot agent development.
- **PII Handling:** Follow data protection regulations when handling personal information
- **Audit Logging:** Implement comprehensive audit trails for all agent actions
- **Secret Storage:** Never hardcode credentials; use Azure Key Vault or environment variables
- **Write Capability Confirmation:** Require clear user confirmation before sensitive `EmailActions` or `MeetingActions`, such as sending or deleting email and creating or changing calendar events

## Performance

Expand All @@ -18,6 +19,7 @@ Follow these best practices for successful M365 Copilot agent development.
- **Caching Strategy:** Implement appropriate caching for frequently accessed data
- **Response Time:** Keep operations under 30 seconds to avoid timeouts
- **Batch Operations:** Use batch APIs when processing multiple items
- **Response Mode Selection:** Use `Quick response` for latency-sensitive agents, `Think deeper` for complex analytical agents, and `Auto` for mixed workloads

## Error Handling

Expand All @@ -30,6 +32,8 @@ Follow these best practices for successful M365 Copilot agent development.
## Testing

- **Test All Conversation Starters:** Verify each starter works as intended
- **Test Write Actions:** Verify confirmation, permissions, cancellation, and failure behavior for `EmailActions` and `MeetingActions`
- **Test Editorial Matching:** Check close paraphrases and unrelated questions against every `editorial_answers` entry
- **Test Edge Cases:** Test with missing data, invalid inputs, and error conditions
- **Security Testing:** Verify scoping and permission controls
- **Cross-Environment Testing:** Test in dev, staging, and production environments
Expand All @@ -50,6 +54,7 @@ Follow these best practices for successful M365 Copilot agent development.
- **Code Organization:** Separate concerns (capabilities, API plugins, models)
- **Version Control:** Use semantic versioning for shared agents
- **Change Management:** Document changes and maintain changelog
- **Editorial Answer Governance:** Assign an owner and review cadence for approved answers so stale canonical content is removed promptly

## Conversation Design

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,12 +108,14 @@ Describe the purpose and context for each data source the agent can use. You don
- **Copilot connector knowledge**: Reference by connector name — "Use `ServiceNow KB` for help articles."
- **SharePoint/OneDrive**: Describe the data source — "Search internal HR policy documents" or "Reference company documents in SharePoint."
- **Email**: Describe the scenario — "Check user emails for relevant information."
- **Email actions**: Describe when to send, archive, flag, move, delete, or organize email, and require confirmation before sensitive writes.
- **Teams messages**: "Search Teams channels and chat messages." (messages only — NOT transcripts)
- **Meetings**: "Check calendar events, attendees, and meeting transcripts." (transcripts come from `Meetings`, not `TeamsMessages`)
- **Meeting actions**: Describe when to schedule or change calendar events, create time-finding polls, or surface time insights, and require confirmation before calendar writes.
- **Code interpreter**: "Use code interpreter to generate charts."
- **People knowledge**: "Look up people in the organization for contact info."

> **Common mistake:** Assuming meeting transcripts are part of `TeamsMessages`. Transcripts are retrieved through the `Meetings` capability. `TeamsMessages` covers channel posts, DMs, and meeting chat messages only. See the [Capability Reference](instruction-review.md#capability-reference-v16) for the full mapping.
> **Common mistake:** Assuming meeting transcripts are part of `TeamsMessages`. Transcripts are retrieved through the `Meetings` capability. `TeamsMessages` covers channel posts, DMs, and meeting chat messages only. See the [Capability Reference](instruction-review.md#capability-reference-v18) for the full mapping.

### 8. Provide Examples

Expand Down Expand Up @@ -163,6 +165,54 @@ Developing instructions is an iterative process:

---

## Manifest-Level Experience Controls

Evaluate these options while designing a new agent; they are easy to miss if discovery focuses only on knowledge sources and plugins.

### Editorial answers (v1.7+)

Use `editorial_answers` for approved, stable answers that should be selected through semantic similarity, such as official support contacts or policy FAQs.

- Use it for canonical content that changes infrequently.
- Do not use it for personalized, permission-sensitive, or rapidly changing answers.
- Keep each question and answer focused on one intent.
- Review inline answers as governed content because they reduce normal generative variation.

```json
{
"editorial_answers": {
"answers": [
{
"question": "How do I contact the help desk?",
"answer": "Email helpdesk@contoso.com or call extension 5555."
}
]
}
}
```

### Default response mode (v1.7+)

Choose `behavior_overrides.default_response_mode` from the dominant workload:

| Mode | Use when |
|------|----------|
| `Auto` | Requests vary between simple lookups and complex analysis. This is the default. |
| `Quick response` | Low latency matters more than deep reasoning, such as routing or short FAQ agents. |
| `Think deeper` | The agent primarily performs complex analysis, planning, or multi-step decisions. |

Users can override the default in the model selector. Do not rely on the setting as a hard enforcement mechanism.

```json
{
"behavior_overrides": {
"default_response_mode": "Think deeper"
}
}
```

---

## Example Instructions

The following example is for an IT help desk agent:
Expand Down Expand Up @@ -369,14 +419,29 @@ Identify the top 3-5 tasks users will perform:
}
```

#### 4. Use Natural Language
#### 4. Gate Capability-Specific Starters (v1.7+)

Use `depends_on` when a starter only works with specific configured capabilities. Every dependency must be present before the starter is shown.

```json
{
"title": "Project Update",
"text": "Summarize internal email and public web updates for this project.",
"depends_on": [
{ "name": "capabilities", "id": "Email" },
{ "name": "capabilities", "id": "WebSearch" }
]
}
```

#### 5. Use Natural Language

Write starters as users would naturally speak:

✅ Good: "What's the latest on Project Phoenix?"
❌ Bad: "Query project status for Phoenix"

#### 5. Provide 3-6 Starters (Not Too Many)
#### 6. Provide 3-6 Starters (Not Too Many)
- **Too few** (<3): Users don't see the full capability range
- **Just right** (3-6): Good variety without overwhelming
- **Too many** (>6): Clutters UI, users won't read them all
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ When you add, remove, or modify ANY capability or plugin, you MUST complete ALL
- Identify the agent's primary purpose and target users
- Determine required data sources (M365 services, external APIs)
- List necessary actions the agent must perform
- Identify email and calendar **write** operations separately from read/search requirements
- Identify approved, stable FAQ answers that should use canonical response content
- Decide whether the default experience should favor automatic selection, low latency, or deeper reasoning
- Identify security and compliance requirements

### Step 2: Design the Agent Architecture
Expand All @@ -76,6 +79,15 @@ When you add, remove, or modify ANY capability or plugin, you MUST complete ALL
- Plan authentication and authorization strategy
- Design conversation flow and instructions

**Build-time feature discovery:**

| User requirement | Manifest feature | Design rule |
|------------------|------------------|-------------|
| Email triage, supervised send, delete, rules, auto-reply, or folder management | `EmailActions` (v1.8+) | This is independent of read-only `Email`; its scopes do not carry over. Add confirmation guidance for sensitive writes. |
| Meeting scheduling, calendar events, time-finding polls, or time insights | `MeetingActions` (v1.8+) | This is separate from read-only `Meetings`. Add confirmation guidance before creating or changing events. |
| Canonical answers for semantically similar questions | `editorial_answers` (v1.7+) | Use for approved, stable Q&A. Do not use for dynamic or personalized data. |
| A consistent default speed/reasoning experience | `behavior_overrides.default_response_mode` (v1.7+) | Choose `Auto`, `Quick response`, or `Think deeper` based on the dominant workload. |

### Step 3: Edit JSON Manifest Files

**⚠️ PRE-EDIT CHECK — Before making ANY edits, do ALL of these:**
Expand All @@ -87,7 +99,7 @@ When you add, remove, or modify ANY capability or plugin, you MUST complete ALL
- Only after the user confirms, fix with surgical edits, then re-read the file
- Then continue with the user's original request as a separate step

2. **Check the schema version**: Read the `"version"` field in `declarativeAgent.json` (e.g., `"v1.4"`, `"v1.6"`). For EVERY feature you plan to add, verify it exists in that version using the [feature matrix](schema.md). If a requested feature requires a newer version → **STOP. Tell the user.** Offer to upgrade the version first.
2. **Check the schema version**: Read the `"version"` field in `declarativeAgent.json` (e.g., `"v1.4"`, `"v1.8"`). For EVERY feature you plan to add, verify it exists in that version using the [feature matrix](schema.md). If a requested feature requires a newer version → **STOP. Tell the user.** Offer to upgrade the version first.

3. **Run a proactive instruction review** (if the edit touches instructions or capabilities): Before modifying instructions or adding/removing capabilities, run [Instruction Review](instruction-review.md) **Phase 1 (Inventory)**, **Phase 2 (Comprehension Check)**, and **Phase 3 (Diagnose)** against the current instructions. This catches existing problems before you add to them. For Phase 2, use the brief confirmation shortcut ("I see this agent is designed to [purpose]…") since this is a proactive check. If the review finds high-severity issues (C1, C3, C11, D1-D8), inform the user and offer to fix them as part of the current edit.

Expand Down
Loading