Looking for 3 MCP/API providers for external measurement runs

[roy-tong]

I'm looking for 3 external MCP/API capability providers to test AgentMeasure against a real agent-facing integration.

The trial is intentionally small:

• ~15 minutes to instrument
• provider-side only
• no agent-side install required
• MCP supported but not required
• collector can remain local
• no production traffic required

What I want to learn:

1. Does the event model fit a real provider?
2. Where does operation vs attempt break?
3. Which fields are impossible or unsafe for providers to expose?
4. Does Reach → Choice → Use → Utility → Value match anything providers actually care about?

I am not asking for a partnership announcement. A failed integration or a strong “this abstraction is wrong” is a useful result.

If you want to try it, comment with:

• project/repo
• protocol (MCP / API / other)
• one agent-facing capability you want measured

I'll help with the first instrumentation pass.

[roy-tong]

External measurement outreach, 2026-08-25 — structured semantics questions filed upstream (Hypothesis-A channel, langfuse#16383 pattern):

• Semantics question: how does OpenLIT count requests and tokens when a retry emits two spans for one logical call? openlit/openlit#1488 — retry emits two spans for one logical call: what does the Requests view count?
• Semantics question: in AI Gateway analytics, is a retried provider call one request or two — and do cache hits count? cloudflare/cloudflare-docs#32983 — AI Gateway analytics: retried call = 1 request or 2; do cache hits count?
• Semantics question: what is a test case's canonical verdict when a judge-based metric is re-run and straddles the threshold? confident-ai/deepeval#3110 — one test case, three judge scores straddling the threshold: canonical verdict?
• Semantics question: how should tool-call telemetry count cancelled or retried tools/call requests? modelcontextprotocol/typescript-sdk#2715 — cancel/timeout/retry: what is the counting unit for tool-call telemetry?

(An axolotl draft on tokens-processed semantics across resume_from_checkpoint was held back: #3291 already covers adjacent ground — will engage there instead of filing a near-duplicate.)

These ask for counting-unit semantics only; they do not pitch the standard. Tracking replies in the evidence ledger.

[kbsoso9999-maker]

• @````
• ****

[roy-tong]

Update 2026-08-31 — the lowest-commitment way to start just got smaller.

Before any 7-day run, you can now send a trace, get a measurement check: 20–100 anonymized trace/log rows (or a public export), mapped locally, and you get back a short report — what safely counts as attempts vs operations in your data, where retries may inflate usage, and what the telemetry cannot prove.

• No SDK, no integration, no 7-day commitment.
• Raw data stays local by default; a sanitized sample is only shared if you explicitly authorize it.
• Production samples and synthetic fixtures enter through separate intakes — say which yours is.
• A sample check is not a full audit and claims no causal effect.

Email tongroy18@gmail.com with subject "measurement check" and three short fields: the data, its source + time window, and the decision it should inform. Full ladder and boundaries: AUDIT.md · Issue #2.