feat: Verify per-context private attributes are not applied to other contexts - #429
Conversation
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Validated against 4 real SDK contract-test servicesRan the locally built harness (this branch) against contract-test services built from source. Full Results — the 3 new subtests pass everywhere the SDK has the fix
¹ Negative control: the test really does catch the bug it was written forReverting the ruby-server-sdk fix locally (
Ruby file was restored afterwards; nothing committed in that repo. Regression: whole
|
| SDK | this branch | baseline origin/v3 |
|---|---|---|
| Go | 436 ran / 0 failed | 433 ran / 0 failed |
| Ruby | 320 ran / 6 failed | 317 ran / 6 failed |
Exactly +3 tests, and the Ruby failure lists are byte-identical (diff → no output). No existing tests broken and no event-sink state leaking between tests.
No changes suggested to the test expectations. Written by Devin
SDK validationRan the 3 new subtests ( No SDK fails the new tests, and no SDK gains a new failure. 18 services PASS, 2 N/A (no/incapable test service), 2 BLOCKED (need a macOS runner / a physical Roku), 1 pending. Negative control: reverting launchdarkly/ruby-server-sdk#416 ( Per-SDK results
Each PASS row was checked to have actually executed the new subtests ( Rollout note: several repos pin a released harness rather than a branch tip (dotnet downloads the latest v2 release; python/go/java/dotnet pin |
|
Addendum — the js-core client-side workspaces finished, all PASS with no new failures (js-core
That closes out the run: 24 services PASS, 2 N/A (standalone |
…ling (#404) ## Summary Adds contract tests exercising the RETRY specification's non-permanent-stop guarantee for the FDv1 streaming and polling data sources. Introduces two capability gates so SDKs can adopt independently: - `retry-conformance-fdv1-streaming` - `retry-conformance-fdv1-polling` The new tests exercise the SDK's real production 5-minute extended-regime timing — no test-harness bypass, no compression knobs on the servicedef, no test-only surface in the SDK's public API. They are marked as long-running (`t.LongRunning()`) and only run when the harness is invoked with `-enable-long-running-tests`. Legacy "do not retry after unrecoverable HTTP error" scenarios remain, gated behind the ABSENCE of the streaming capability so SDKs can migrate on their own schedule. Once every SDK reports the new capabilities, both the legacy tests and the gates can be removed. ## What's added **Streaming (guarded by `retry-conformance-fdv1-streaming` + `-enable-long-running-tests`):** - retry after unexpected HTTP error on initial connect (401/403/405) - retry after unexpected HTTP error on reconnect (401/403/405) - enters extended-regime backoff after unexpected HTTP error (paired-bound timing assertion) - does not permanently stop under sustained unexpected HTTP errors - returns to normal-regime backoff after healthy operation (streaming spec's activeSince reset: 75 s silent stream after an extended-regime reconnect, then next reconnect must be at normal-regime timing) **Polling (guarded by `retry-conformance-fdv1-polling` + `-enable-long-running-tests`):** - retry after unexpected HTTP error on initial poll (401/403/405) - returns to normal-regime cadence after two consecutive successful polls (RETRY §1.8.1 count-based reset) ## Design: long-running instead of compressed timings An earlier revision of this PR exposed extended-regime timing knobs on the servicedef (`ExtendedInitialDelayMS`, `ResetThresholdMS`) so tests could compress the 5-minute extended-initial delay into a ~30-second observable window. Reviewers pushed back because that shape forced SDKs to expose corresponding test-only knobs on their public API surface just to be testable by the harness. This revision drops all compression knobs and exercises real production timing instead. That makes each retry-conformance subtest take several minutes of wall clock, which is why the tests are marked `t.LongRunning()` and gated behind `-enable-long-running-tests`. Per-SDK CI schedules for the long-running suite are out of scope for this PR (planned follow-up work). Framework support for `t.LongRunning()` and the `-enable-long-running-tests` flag was ported from v3 in #428, a prerequisite that landed before this PR's refactor. ## Relationship to PR #401 Supersedes #401, which targeted `v3` and exercised FDv2 code paths — out of scope for the initial RETRY-conformance work, which is FDv1-only. This PR targets `v2` and drives the FDv1 streaming/polling paths in the SDK testservices. #401 will be closed with a pointer here. SDK-2808 ## Test plan - [x] Contract tests pass against go-server-sdk PR #429 (the reference server SDK implementation) at real production 5-minute extended-regime timing (~3-hour run with `-enable-long-running-tests`) - [x] Legacy "do not retry" tests still pass for SDKs without the new capability (verified by SDK capability check) - [x] Retry-conformance tests are correctly SKIPPED for SDKs without the capability - [x] Retry-conformance tests are correctly SKIPPED when `-enable-long-running-tests` is not set
…contexts (#430) Adds contract coverage for private attributes declared in a context's `_meta.privateAttributes` leaking into an SDK's *globally configured* private attribute list, so they get applied to unrelated contexts. - Port of #429 to the `v2` line. Nearly every SDK's CI pulls the harness from `v2` (ruby, php, erlang, rust, haskell, java, dotnet, ios, android, flutter, roku, node-client), so without this the regression below is untested for those SDKs. - 3 new subtests under `events/context properties`; runs for server-side, client-side and PHP suites. No existing test or expectation changed. - Reproduces [ruby-server-sdk#416](launchdarkly/ruby-server-sdk#416): `ContextFilter` did `@private_attributes.concat(context.private_attributes)`, mutating the configured list, so a context that declared nothing private still had the *previous* context's private attributes redacted. **Requirements** - [x] I have added test coverage for new or changed functionality - [x] I have followed the repository's [pull request submission guidelines](../blob/master/CONTRIBUTING.md#submitting-pull-requests) - [ ] I have validated my changes against all supported platform versions — validation across SDKs in progress, results posted as a comment **Related issues** - launchdarkly/ruby-server-sdk#416 — the bug this covers - #429 — the same change on the `v3` line <details> <summary>Implementation details</summary> **Why the existing tests could not catch it** `makeEventContextTestParams` creates a *new client* per parameter, so the `ContextFilter` is always fresh, and every context within one parameter comes from a single factory with identical `_meta.privateAttributes`. A leaked private attribute is therefore always an already-expected private attribute. The existing multi-kind fixtures don't catch it either: they either set no per-context privates, or set them only on the kind that is filtered last, and the leaked names don't exist as attributes on the other kind. **What the new tests do** `eventContextPrivateAttributeScoping` uses **one client** for multiple identify events, and each context carries the same three attributes (`selfPrivate`, `globallyPrivate`, `visible`) while differing only in what it declares private: 1. `private attributes of one context are not applied to later contexts` — identify a context declaring `selfPrivate` private, then identify a different context that declares nothing private, and assert `selfPrivate` is still visible on the second one. 2. + 3. `private attributes of one kind are not applied to other kinds of the same context (declared by org / by user)` — a multi-kind context where only one kind declares `selfPrivate` private; the other kind must keep it. Both orderings are covered because SDKs filter the individual contexts in an arbitrary order — with the Ruby bug present, only the `org` variant fails (Ruby filters `org` first), so a single-ordering test would catch this only half the time. `globallyPrivate` is configured via `GlobalPrivateAttributes` and asserted redacted in every expectation, so an SDK cannot pass by throwing away its configured private attributes along with the per-context ones. **Verification** Negative control against ruby-server-sdk with the #416 fix locally reverted to `.concat(...)`: subtests 1 and 2 fail with the second context wrongly reporting `redactedAttributes: ["selfPrivate","globallyPrivate"]`; all three pass with the fix in place. </details> Link to Devin session: https://app.devin.ai/sessions/6e3076285f2849919b966a4f801075ca Requested by: @kinyoklion <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds **three new subtests** under `events/context properties` via `eventContextPrivateAttributeScoping`, hooked from `EventContexts`. Existing expectations are unchanged. > > The harness reuses **one SDK client** for multiple identify events so failures like mutating the global private-attribute list (e.g. ruby-server-sdk#416) are detectable—prior cases mostly created a fresh client per scenario. > > **Coverage:** (1) a context that marks `selfPrivate` private must not cause a later context with no per-context privates to redact `selfPrivate`; (2) for multi-kind contexts, privates declared on `org` or `user` must not redact the same attribute on the other kind—both orderings are tested. > > Every assertion still expects **`globallyPrivate`** (from `GlobalPrivateAttributes`) to be redacted so SDKs cannot pass by dropping all configured privates. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 3fa29c4. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Devin AI <devin@launchdarkly.com>
🤖 I have created a release *beep* *boop* --- ## [3.2.0-alpha.8](v3.2.0-alpha.7...v3.2.0-alpha.8) (2026-08-19) ### Features * Verify per-context private attributes are not applied to other contexts ([#429](#429)) ([0a35294](0a35294)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Release 3.2.0-alpha.8** bumps the harness version in `main.go` and `.release-please-manifest.json` and records the release in `CHANGELOG.md`. > > The release notes highlight **new contract coverage** from [#429](#429): tests that **`_meta.privateAttributes` on one context must not redact attributes on other contexts** when the same client sends events for multiple contexts (including alongside globally configured private attributes). SDKs that merge per-context private lists into a global list would fail these tests. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit b19e419. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Adds
events/context propertiescoverage for private attributes declared in a context's_meta.privateAttributesleaking onto other contexts — the Ruby SDK bug fixed in launchdarkly/ruby-server-sdk#416, which the existing suite could not detect.selfPrivateprivate, the second not — the second must not have it redacted.selfPrivateprivate — the other kind must not have it redacted.Requirements
Implementation details
Why existing tests miss it
makeEventContextTestParamsbuilds one client per test param, and every context that param then generates comes from a singledata.ContextFactorywith identical_meta.privateAttributes. An SDK that merges per-context private attributes into its long-lived configured list (rather than a per-context copy) therefore leaks only attributes that are already expected to be redacted. The multi-kind factories are also blind to it: they either declare no per-context privates, or declare them only on the kind filtered last (data.NewContextFactoriesForExercisingAllAttributesgivesorgjustnamewhileothercarriesPrivate("a","c")), and the leaked names do not exist on the other kind.What the new tests do
CommonEventTests.eventContextPrivateAttributeScopingis called at the end ofEventContexts, so it runs for server-side, client-side, and PHP SDKs. Each context carriesselfPrivate,globallyPrivate, andvisible;globalPrivateAttributesis configured as["globallyPrivate"].selfPrivate-declaring context (expectselfPrivate+globallyPrivateredacted), then identify a context that declares nothing private (expect onlygloballyPrivateredacted). Pre-fix: Authenticate release asset downloads with GITHUB_TOKEN #416 Ruby redactsselfPrivatefrom the second context too.org+userin one context, only one kind declaringselfPrivateprivate; run once per kind so the leak is caught regardless of the order an SDK filters individual contexts in.Both cases still assert
globallyPrivateis redacted, which is what distinguishes a correct fix (copy the list) from a wrong one (stop applying configured privates).Testing
make build,make lint(0 issues), andmake testpass. SDK-level validation is in progress: the new subtests are being run against SDK contract test services, including the Ruby SDK before and after #416.Alternatives considered
Adding more
eventContextTestParamsentries — rejected, the leak is only observable across two contexts with different private attributes within one client, which that table cannot express.Link to Devin session: https://app.devin.ai/sessions/6e3076285f2849919b966a4f801075ca
Requested by: @kinyoklion
Note
Overview
Adds
eventContextPrivateAttributeScopingat the end ofEventContexts, so server-side, client-side, and PHP SDKs all run it.The new coverage catches SDKs that merge
_meta.privateAttributesinto a long-lived global list instead of a per-context copy (e.g. Ruby server SDK before #416). A sequential identify case sends one context that marksselfPrivateprivate, then another that does not—onlygloballyPrivate(fromglobalPrivateAttributes) must be redacted on the second event. Multi-kind subtests run for bothorganduseras the kind that declaresselfPrivate, so redaction cannot leak across kinds when filtering one kind at a time.Every assertion still expects
globallyPrivateredacted, so passing by dropping all configured private attributes along with per-context ones fails the suite.Reviewed by Cursor Bugbot for commit 62615b4. Bugbot is set up for automated code reviews on this repo. Configure here.