feat(enterprise-observability): Add bounded custom request dimensions for filtering, grouping and segmentation

[matdev83]

Assessment: 9.4/10

Expected Enterprise/observability value: exceptional
Proxy-level fit: exceptional
Implementation leverage vs complexity: high if custom metadata stays observational and strongly bounded

Summary

Add an Enterprise custom observability dimensions layer so integrations/operators can attach bounded metadata to requests/sessions and later filter, group, aggregate, evaluate and alert on those dimensions.

Helicone's Custom Properties let users attach request metadata such as environment, feature, application, workflow stage or release version, then use it throughout request filtering, cost analysis, alerts and dataset creation. It also supports post-hoc property updates after a request has completed.

Reference:

• https://github.com/Helicone/helicone/blob/main/docs/features/advanced-usage/custom-properties.mdx

AIProxer already has important trusted principal/project/policy metadata. This feature must not blur that security boundary.

Critical distinction from trusted policy/identity facts

AIProxer already carries authoritative facts such as:

principal
organization / tenant
workspace / project
department / cost center
roles / policy labels
request purpose where positively identified

Those may affect accounting, authorization and routing.

Custom observability dimensions are different:

feature=code-review
workflow=security-scan
environment=staging
release=2026.08.26
repo_class=monorepo
agent_version=1.2.3
experiment=router-shadow-v2

By default these are untrusted observational labels only.

A caller cannot gain model access, change billing identity, bypass #481 residency policy or alter #497 routing simply by inventing a custom property.

If an operator explicitly promotes one field into a trusted policy input through a separately authenticated mapping, that is a different authority and must be visible/audited.

1. Canonical property envelope

Each request/session may carry a bounded map of custom dimensions:

name -> typed/bounded value

Initial value types can remain deliberately small:

string
boolean
integer
bounded enum where definition exists

Avoid arbitrary nested JSON in the hot analytics path.

2. Hard cardinality and size limits

Helicone's flexibility is useful, but AIProxer must protect a high-concurrency gateway and analytics store.

Hard limits should cover:

max properties/request
max property-name bytes
max value bytes
max total custom-metadata bytes/request
max distinct indexed names/workspace
max distinct indexed values where appropriate

Over-limit input follows explicit reject | truncate/drop-observability-only semantics depending on ingestion surface.

No custom value becomes a Prometheus metric label automatically.

3. Ingestion sources and provenance

Support properties from several safe sources:

client/wrapper integration
server-side integration adapter
enterprise enrichment rule
post-hoc control API
CI/evaluation system

Each property projection can retain source/provenance where useful.

A server-derived property can be marked trusted_observation, but observation trust is still not policy authority.

4. Header/API surface

For HTTP integrations, offer a namespaced mechanism conceptually similar to Helicone's property headers, e.g.:

X-AIProxer-Property-Environment: staging
X-AIProxer-Property-Workflow: pr-review

Exact naming should follow final rebranding/API conventions.

Requirements:

• parse only the explicit prefix, not arbitrary headers;
• case/name normalization deterministic;
• header count/value limits applied before copying;
• strip AIProxer-owned observational headers before upstream forwarding unless a separate provider contract explicitly needs them.

A JSON management/SDK API should be available for clients that cannot safely add arbitrary headers.

5. Post-hoc annotations

Like Helicone, allow properties to be attached/updated after a request completes:

POST/PUT /.../requests/{id}/properties

Examples:

ci_build=passed
incident=INC-142
postprocessed=true
review_batch=2026-08-26

Post-hoc mutations require #506 authorization and #508 audit where operator-controlled.

Use revision/upsert semantics rather than mutating unrelated request evidence.

6. Indexed vs non-indexed properties

Do not create a relational/ClickHouse column for every name.

Use a storage shape such as:

bounded property map on request metadata
+ optional indexed projection for selected/high-value keys

Enterprise administrators may nominate keys for efficient filtering/grouping.

Index publication occurs control-plane-side; normal requests do not perform schema DDL.

7. Query/filter integration

Properties should become available to:

• feat(enterprise-analytics): Add governed ad-hoc observability query workbench with saved queries and exports #524 analytics workbench;
• feat(enterprise-logs): Add metadata-first request log store with object-storage payload offload #509 request/session search;
• feat(enterprise): Company-wide project catalog, session-project correlation #411 dashboards/report breakdowns;
• feat(enterprise-alerting): Add policy-aware alerts, incident channels and delivery history #516 scoped alerts;
• feat(enterprise-evals): Add versioned online evaluators for sampled production quality scoring #521 evaluator selectors;
• feat(enterprise-evalops): Add curated production datasets and safe model/route regression replay #523 dataset curation;
• feat(enterprise-export): Add governed asynchronous event sinks for Kafka and enterprise data platforms #518 event exports.

Example questions:

cost by workflow
p95 TTFT by agent_version
quality by release
error rate for feature=review
success/$ for experiment=router-shadow-v2

8. Session inheritance

A session may carry stable observational dimensions such as:

repo
workflow
client version
environment

Define explicit inheritance semantics:

session properties copied/projected onto subsequent requests
request-local override for observational view

Do not infer secure session ownership from a property.

#511 native harness IDs provide session correlation but not trusted metadata values.

9. Privacy and sensitive-data posture

Custom properties are an easy place to accidentally insert PII/secrets.

Requirements:

• documented safe-use policy;
• configurable name allow/deny patterns;
• secret-pattern detector/redaction hook where appropriate;
• feat(enterprise-logs): Add metadata-first request log store with object-storage payload offload #509 retention classification;
• feat(enterprise-rbac): Add control-plane RBAC and row-level data access scopes #506 row-level access;
• no raw values in broad metrics/log statements;
• optional hashing/pseudonymization for identifiers.

Operators can forbid arbitrary client properties entirely in strict deployments.

10. Property definitions / governance

For mature Enterprise workspaces, allow optional registered definitions:

name
type
description
allowed source(s)
indexable yes/no
sensitive yes/no
allowed values / validation
retention class

Unknown ad-hoc properties may be allowed with stricter generic bounds or denied according to workspace policy.

This prevents enterprise analytics from degrading into dozens of spelling variants such as env, environment, Environment.

11. No routing/auth side effects by default

Custom properties must remain outside authoritative request policy.

Conceptual separation:

Observation.Properties      -> dashboards/search/evals
TrustedPrincipal/Policy     -> auth/accounting/routing

A future policy bridge that permits selected server-authenticated property definitions to feed #497 must be explicit, compiled and audited; never generic property == value -> privileged route from caller metadata.

12. Performance

• header/property parsing is bounded O(property count);
• no DB read to validate every property on request path: definitions live in immutable runtime snapshot;
• no per-request dynamic schema operations;
• metadata copy size bounded;
• disabled/unconfigured mode has negligible overhead;
• analytics indexing is async/off hot path.

13. Open-Core boundary

This is primarily an Enterprise observability capability because its value comes from organization-wide querying, governance and analytics.

Potential boundary:

• OSS may expose a small generic request-metadata hook useful for integrations;
• Enterprise owns durable custom-property indexing, definitions, post-hoc annotation APIs, cross-org filters and UI.

Suggested V1

1. Bounded string/bool/int property map + explicit HTTP prefix.
2. Immutable property-definition snapshot.
3. Persist into feat(enterprise-logs): Add metadata-first request log store with object-storage payload offload #509 metadata projection.
4. Filter/group support for selected keys.
5. Post-hoc protected API.
6. Integrate feat(enterprise-quality): Add production feedback, quality scores and annotations for requests and sessions #519/feat(enterprise-evals): Add versioned online evaluators for sampled production quality scoring #521/feat(enterprise-evalops): Add curated production datasets and safe model/route regression replay #523/feat(enterprise-analytics): Add governed ad-hoc observability query workbench with saved queries and exports #524/feat(enterprise-alerting): Add policy-aware alerts, incident channels and delivery history #516.
7. Add session inheritance and stricter enterprise governance.

Acceptance criteria

• A supported client can attach a bounded observational property to a request and later filter/query by it.
• Custom properties cannot establish principal/project identity or bypass route/model/security policy.
• Property count/name/value/total-byte limits prevent unbounded per-request metadata growth.
• Selected properties can be indexed/grouped without adding dynamic columns on every request.
• A protected post-hoc API can annotate a completed request with audit/provenance.
• Strict deployments can disable unknown client properties or use registered definitions only.
• Raw property values never become Prometheus labels automatically.
• Property ingestion performs no per-request DB query.

Non-goals

• arbitrary caller-controlled policy claims;
• unbounded nested JSON metadata;
• one SQL column per property;
• trusting arbitrary X-* headers;
• automatic high-cardinality metric dimensions;
• replacing feat(enterprise): Company-wide project catalog, session-project correlation #411 authoritative project attribution or feat(enterprise-identity): Add SCIM/OIDC user lifecycle provisioning, group mapping and deprovisioning #505 identity facts.

Why 9.4/10

A serious observability product must let enterprises segment telemetry by the dimensions that matter to their workflows. Helicone demonstrates the leverage: one property system powers filtering, cost analysis, experiments, datasets and alerts. AIProxer can gain the same flexibility while preserving a stricter separation between observational labels and trusted routing/accounting authority.

[matdev83]

Cross-feature Enterprise observability rationale

This issue belongs to the same Helicone-inspired Enterprise observability architecture:

#502 / #509 / #522   telemetry
          ↓
#520                 observational dimensions
          ↓
#519                 outcome/quality evidence
          ↓
#521                 continuous sampled evaluators
          ↓
#524                 ad-hoc analytics
          ↓
#523                 regression datasets + safe replay
          ↓
#516 / #411          alerts/reports/dashboards

The strategic role of #520 is to make otherwise-generic telemetry meaningful to each organization. Enterprise engineers need to ask things such as:

quality by workflow
p95 TTFT by agent_version
cost by repository/project class
error rate by release
task-success/$ by experiment cohort

Those dimensions must remain observational by default, unlike trusted principal/project/policy facts. A caller-controlled workflow=security or environment=prod property must never grant model access, move billing scope, bypass residency policy, or select a privileged route.

This distinction is important to the product thesis: flexible analytics should not weaken AIProxer's unusually strong authority boundaries. #520 feeds #519/#521/#523/#524/#516/#411, but does not become an authorization/routing metadata escape hatch.