feat(ai): native Langfuse and Datadog usage sinks#554
Merged
Conversation
Adds two provider sinks behind the existing UsageSink seam, next to the
JSONL-file / webhook / ledger sinks:
- langfuse: posts a generation observation to {host}/api/public/ingestion
with public/secret-key basic auth.
- datadog: posts to https://http-intake.logs.{site}/api/v2/logs with the
DD-API-KEY header; site defaults to datadoghq.com, optional service tag.
The request-body shaping (langfuse_ingestion_body, datadog_log_body) is
pure and unit-tested; the sinks are fire-and-forget and swallow errors so
a broken sink can never fail the request it records. Config is
deserialize-only, so no schema change. Object-store (S3/GCS) and OTel
usage sinks are not yet included.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X19S6eQzKKExZ9RUPAHuGy
…-native-usage-sinks # Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two native usage sinks behind the existing
UsageSinkseam, so a completed-call event can be forwarded directly to Langfuse or Datadog instead of via the generic webhook:type: langfuse—host+public_key+secret_key. POSTs ageneration-createobservation to{host}/api/public/ingestionwith public/secret-key basic auth. Token counts go inusage; provider, cost, latency, status, and identifiers go inmetadata.type: datadog—api_key+ optionalsite(defaultdatadoghq.com) + optionalservice(defaultsbproxy). POSTs tohttps://http-intake.logs.{site}/api/v2/logswith theDD-API-KEYheader.Both are fire-and-forget on a spawned task and swallow errors, so a broken or misconfigured sink can never block or fail the request it records (same contract as the existing webhook/jsonl/ledger sinks).
Tests
langfuse_ingestion_body,datadog_log_body) and config parsing/build (8usage_sinktests pass).sbproxy-ailib suite (962) passes; clippy-D warningsand rustdoc-D warnings -D missing_docsclean; regenerated config schema is byte-identical (the sink config is deserialize-only).Caveat / scope
The payload shapes follow Langfuse's public ingestion API and Datadog's logs-intake API as documented. CI can unit-test the JSON I construct but cannot validate it against a live endpoint, so these should be confirmed against a real Langfuse/Datadog before relying on them in production. Object-store sinks (S3/GCS, which need cloud SDKs) and an OTel usage sink (overlaps the existing OTLP span pipeline) are intentionally not included here.