Skip to content

feat: Verify per-context private attributes are not applied to other contexts - #429

Merged
kinyoklion merged 1 commit into
v3from
devin/1787072111-private-attribute-scoping-tests
Aug 19, 2026
Merged

feat: Verify per-context private attributes are not applied to other contexts#429
kinyoklion merged 1 commit into
v3from
devin/1787072111-private-attribute-scoping-tests

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Aug 18, 2026

Copy link
Copy Markdown
Member

Adds events/context properties coverage for private attributes declared in a context's _meta.privateAttributes leaking onto other contexts — the Ruby SDK bug fixed in launchdarkly/ruby-server-sdk#416, which the existing suite could not detect.

  • New subtest: two identify events on one client, the first context declaring selfPrivate private, the second not — the second must not have it redacted.
  • New subtests (both kind orderings): one multi-kind context where only one kind declares selfPrivate private — the other kind must not have it redacted.
  • A globally configured private attribute is expected redacted in every assertion, so an SDK cannot pass by dropping its configured private attributes along with the per-context ones.
  • No new capability gate: this is required behavior for every SDK, so all SDKs run it.

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions
Implementation details

Why existing tests miss it

makeEventContextTestParams builds one client per test param, and every context that param then generates comes from a single data.ContextFactory with identical _meta.privateAttributes. An SDK that merges per-context private attributes into its long-lived configured list (rather than a per-context copy) therefore leaks only attributes that are already expected to be redacted. The multi-kind factories are also blind to it: they either declare no per-context privates, or declare them only on the kind filtered last (data.NewContextFactoriesForExercisingAllAttributes gives org just name while other carries Private("a","c")), and the leaked names do not exist on the other kind.

What the new tests do

CommonEventTests.eventContextPrivateAttributeScoping is called at the end of EventContexts, so it runs for server-side, client-side, and PHP SDKs. Each context carries selfPrivate, globallyPrivate, and visible; globalPrivateAttributes is configured as ["globallyPrivate"].

  • Sequential case: identify selfPrivate-declaring context (expect selfPrivate + globallyPrivate redacted), then identify a context that declares nothing private (expect only globallyPrivate redacted). Pre-fix: Authenticate release asset downloads with GITHUB_TOKEN #416 Ruby redacts selfPrivate from the second context too.
  • Multi-kind case: org + user in one context, only one kind declaring selfPrivate private; run once per kind so the leak is caught regardless of the order an SDK filters individual contexts in.

Both cases still assert globallyPrivate is redacted, which is what distinguishes a correct fix (copy the list) from a wrong one (stop applying configured privates).

Testing

make build, make lint (0 issues), and make test pass. SDK-level validation is in progress: the new subtests are being run against SDK contract test services, including the Ruby SDK before and after #416.

Alternatives considered

Adding more eventContextTestParams entries — rejected, the leak is only observable across two contexts with different private attributes within one client, which that table cannot express.

Link to Devin session: https://app.devin.ai/sessions/6e3076285f2849919b966a4f801075ca
Requested by: @kinyoklion


Note

Overview
Adds eventContextPrivateAttributeScoping at the end of EventContexts, so server-side, client-side, and PHP SDKs all run it.

The new coverage catches SDKs that merge _meta.privateAttributes into a long-lived global list instead of a per-context copy (e.g. Ruby server SDK before #416). A sequential identify case sends one context that marks selfPrivate private, then another that does not—only globallyPrivate (from globalPrivateAttributes) must be redacted on the second event. Multi-kind subtests run for both org and user as the kind that declares selfPrivate, so redaction cannot leak across kinds when filtering one kind at a time.

Every assertion still expects globallyPrivate redacted, so passing by dropping all configured private attributes along with per-context ones fails the suite.

Reviewed by Cursor Bugbot for commit 62615b4. Bugbot is set up for automated code reviews on this repo. Configure here.

@kinyoklion kinyoklion self-assigned this Aug 18, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@kinyoklion
kinyoklion marked this pull request as ready for review August 18, 2026 16:58
@kinyoklion
kinyoklion requested a review from a team as a code owner August 18, 2026 16:58
@devin-ai-integration

Copy link
Copy Markdown
Contributor

Validated against 4 real SDK contract-test services

Ran the locally built harness (this branch) against contract-test services built from source. Full events/context properties path each time.

Results — the 3 new subtests pass everywhere the SDK has the fix
SDK 3 new subtests full events/context properties
Go server 7.15.6 ✅ pass ✅ 110 ran / 0 failed
Node server (js-core server-node) ✅ pass ✅ 182 ran / 0 failed
Python server ✅ pass 110 ran / 4 failed — all pre-existing¹
Ruby server main (has ruby-server-sdk#416) ✅ pass 110 ran / 4 failed — all pre-existing¹

¹ single-kind, allAttributesPrivate, slash-prefixed attribute name/* fails identically with a baseline harness built from origin/v3, so it is unrelated to this PR.

Negative control: the test really does catch the bug it was written for

Reverting the ruby-server-sdk fix locally (@private_attributes + context.private_attributes.concat(...)) and restarting the service:

  • private attributes of one context are not applied to later contexts — FAILS. The second identify event, whose context declared nothing private, wrongly redacts selfPrivate:
    item[0] JSON property "context" JSON property "selfPrivate" not found,
    JSON property "_meta" JSON property "redactedAttributes" got more items than expected: [0]: "selfPrivate"
    full value was: [{"context":{"_meta":{"redactedAttributes":["selfPrivate","globallyPrivate"]},
      "key":"EventContextPrivateScopingNone...","kind":"user","visible":"3"},"kind":"identify"}]
    
  • ...other kinds of the same context (declared by org) — FAILS (user kind also loses selfPrivate).
  • ...(declared by user) — still passes, because Ruby filters org first. Good argument for keeping both orderings in the test; a single-ordering version would only catch this 50% of the time.

Ruby file was restored afterwards; nothing committed in that repo.

Regression: whole events section, new harness vs origin/v3 baseline
SDK this branch baseline origin/v3
Go 436 ran / 0 failed 433 ran / 0 failed
Ruby 320 ran / 6 failed 317 ran / 6 failed

Exactly +3 tests, and the Ruby failure lists are byte-identical (diff → no output). No existing tests broken and no event-sink state leaking between tests.

No changes suggested to the test expectations. Written by Devin

@devin-ai-integration

Copy link
Copy Markdown
Contributor

SDK validation

Ran the 3 new subtests (events/context properties/private attributes of one ...) 3x against each SDK's own contract test service, plus the full events/context properties group with both the PR binary and a baseline binary built from the base branch, to separate new failures from pre-existing ones. Every SDK was run on the harness line(s) its CI actually pulls.

No SDK fails the new tests, and no SDK gains a new failure. 18 services PASS, 2 N/A (no/incapable test service), 2 BLOCKED (need a macOS runner / a physical Roku), 1 pending.

Negative control: reverting launchdarkly/ruby-server-sdk#416 (+ back to .concat) makes the sequential subtest and the multi-kind "declared by org" subtest fail, while "declared by user" still passes — which is why both kind orderings are included.

Per-SDK results
SDK Verdict Line(s) run Version Commit Exit codes Notes
go-server-sdk PASS v2 + v3 7.15.6 88bd0dc 0,0,0 / 0,0,0
python-server-sdk PASS v2 + v3 9.16.1 5da1515 0,0,0 / 0,0,0 v3-only /~1ssn failures pre-existing, and absent from the v3.0.0-alpha.6 tag CI pins
ruby-server-sdk PASS v2 8.14.0 2fd26ea 0,0,0 includes #416
php-server-sdk PASS v2 6.8.2 80f0d88 0,0,0
erlang-server-sdk PASS v2 3.11.2 4bb97ec 0,0,0 OTP 24 only
rust-server-sdk PASS v2 3.2.0 e26d224 0,0,0
haskell-server-sdk PASS v2 4.6.0 aab4645 0,0,0 built with system GHC 8.8.4
java-server-sdk PASS v2 + v3 7.15.0 2a98583 0,0,0 / 0,0,0
dotnet-server-sdk PASS v2 + v3 8.16.0 7a9529c 0,0,0 / 0,0,0
dotnet-client-sdk PASS v2 5.9.5 7a9529c 0,0,0 run as net8.0 on Linux, not CI's macOS target
node-server-sdk (js-core) PASS v2 + v3 9.13.0 965008f 0,0,0 / 0,0,0
shopify-oxygen (js-core) N/A v2 + v3 0.1.19 965008f 1,1,1 HTTP 400 on client creation for the whole events group, identical on baseline; its CI skips event.*
cpp-server-sdk PASS v2 + v3 3.13.1 120b79a 0,0,0 / 0,0,0 Beast backend only, not the -curl jobs
cpp-client-sdk PASS v2 3.11.4 120b79a 0,0,0
node-client-sdk PASS v2 3.4.3 91b619c 0,0,0
electron-client-sdk N/A 5ed0bb6 no contract test service in the repo
flutter-client-sdk PASS v2 + v3 4.20.2 a11df74 0,0,0 / 0,0,0 Dart implementation via flutter test, headless Linux
android-client-sdk PASS v2 + v3 5.15.0 347ca2c 0,0,0 / 0,0,0 emulator API 25; v3-head-only /~1ssn failures are newer than its v3.1.0-alpha.6 pin
ios-client-sdk BLOCKED 5bfd8f8 ContractTests is Apple-platform only; needs a macOS + Xcode runner
roku-client-sdk BLOCKED fb5c24b needs sideloading to a physical Roku; its CI runs lint + package only
js-core browser/react/react-native/vue pending v3 965008f still running

Each PASS row was checked to have actually executed the new subtests (Test Summary: ... 5 ran, subtests listed as executed rather than SKIPPED) and to be unaffected by that repo's suppressions file / TEST_HARNESS_PARAMS. Full logs (3 runs + PR/baseline group runs per SDK) were kept per session.

Rollout note: several repos pin a released harness rather than a branch tip (dotnet downloads the latest v2 release; python/go/java/dotnet pin v3.0.0-alpha.6; android pins v3.1.0-alpha.6), so these subtests only reach those CI jobs once a new v2 release / v3 alpha is cut.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Addendum — the js-core client-side workspaces finished, all PASS with no new failures (js-core 965008f):

Workspace Verdict Line(s) run Version Exit codes
node-client PASS v2 + v3 4.0.2 0,0,0 / 0,0,0
browser PASS v2 + v3 4.9.4 0,0,0 / 0,0,0
vue PASS v2 + v3 0.2.0 0,0,0 / 0,0,0
react-native PASS v2 + v3 10.19.4 0,0,0 / 0,0,0
react PASS v2 4.1.12 0,0,0
electron PASS v2 0.0.1 0,0,0

Test Summary: 65 total, 60 skipped, 5 ran on v2 and 66 total, 61 skipped, 5 ran on v3 for each; browser/vue/react ran under Playwright chromium on Xvfb, react-native on a headless API-31 emulator. The group run against the baseline binary is failure-free for every workspace, so no suppression entries are needed.

That closes out the run: 24 services PASS, 2 N/A (standalone electron-client-sdk repo has no contract test service; js-core shopify-oxygen's service 400s on the whole events group, pre-existing and skipped in its CI), 2 BLOCKED (ios needs a macOS + Xcode runner, roku a physical device).

tanderson-ld added a commit that referenced this pull request Aug 19, 2026
…ling (#404)

## Summary

Adds contract tests exercising the RETRY specification's
non-permanent-stop guarantee for the FDv1 streaming and polling data
sources. Introduces two capability gates so SDKs can adopt
independently:

- `retry-conformance-fdv1-streaming`
- `retry-conformance-fdv1-polling`

The new tests exercise the SDK's real production 5-minute
extended-regime timing — no test-harness bypass, no compression knobs on
the servicedef, no test-only surface in the SDK's public API. They are
marked as long-running (`t.LongRunning()`) and only run when the harness
is invoked with `-enable-long-running-tests`.

Legacy "do not retry after unrecoverable HTTP error" scenarios remain,
gated behind the ABSENCE of the streaming capability so SDKs can migrate
on their own schedule. Once every SDK reports the new capabilities, both
the legacy tests and the gates can be removed.

## What's added

**Streaming (guarded by `retry-conformance-fdv1-streaming` +
`-enable-long-running-tests`):**
- retry after unexpected HTTP error on initial connect (401/403/405)
- retry after unexpected HTTP error on reconnect (401/403/405)
- enters extended-regime backoff after unexpected HTTP error
(paired-bound timing assertion)
- does not permanently stop under sustained unexpected HTTP errors
- returns to normal-regime backoff after healthy operation (streaming
spec's activeSince reset: 75 s silent stream after an extended-regime
reconnect, then next reconnect must be at normal-regime timing)

**Polling (guarded by `retry-conformance-fdv1-polling` +
`-enable-long-running-tests`):**
- retry after unexpected HTTP error on initial poll (401/403/405)
- returns to normal-regime cadence after two consecutive successful
polls (RETRY §1.8.1 count-based reset)

## Design: long-running instead of compressed timings

An earlier revision of this PR exposed extended-regime timing knobs on
the servicedef (`ExtendedInitialDelayMS`, `ResetThresholdMS`) so tests
could compress the 5-minute extended-initial delay into a ~30-second
observable window. Reviewers pushed back because that shape forced SDKs
to expose corresponding test-only knobs on their public API surface just
to be testable by the harness.

This revision drops all compression knobs and exercises real production
timing instead. That makes each retry-conformance subtest take several
minutes of wall clock, which is why the tests are marked
`t.LongRunning()` and gated behind `-enable-long-running-tests`. Per-SDK
CI schedules for the long-running suite are out of scope for this PR
(planned follow-up work).

Framework support for `t.LongRunning()` and the
`-enable-long-running-tests` flag was ported from v3 in #428, a
prerequisite that landed before this PR's refactor.

## Relationship to PR #401

Supersedes #401, which targeted `v3` and exercised FDv2 code paths — out
of scope for the initial RETRY-conformance work, which is FDv1-only.
This PR targets `v2` and drives the FDv1 streaming/polling paths in the
SDK testservices. #401 will be closed with a pointer here.

SDK-2808

## Test plan
- [x] Contract tests pass against go-server-sdk PR #429 (the reference
server SDK implementation) at real production 5-minute extended-regime
timing (~3-hour run with `-enable-long-running-tests`)
- [x] Legacy "do not retry" tests still pass for SDKs without the new
capability (verified by SDK capability check)
- [x] Retry-conformance tests are correctly SKIPPED for SDKs without the
capability
- [x] Retry-conformance tests are correctly SKIPPED when
`-enable-long-running-tests` is not set
kinyoklion added a commit that referenced this pull request Aug 19, 2026
…contexts (#430)

Adds contract coverage for private attributes declared in a context's
`_meta.privateAttributes` leaking into an SDK's *globally configured*
private attribute list, so they get applied to unrelated contexts.

- Port of #429 to the `v2` line. Nearly every SDK's CI pulls the harness
from `v2` (ruby, php, erlang, rust, haskell, java, dotnet, ios, android,
flutter, roku, node-client), so without this the regression below is
untested for those SDKs.
- 3 new subtests under `events/context properties`; runs for
server-side, client-side and PHP suites. No existing test or expectation
changed.
- Reproduces
[ruby-server-sdk#416](launchdarkly/ruby-server-sdk#416):
`ContextFilter` did
`@private_attributes.concat(context.private_attributes)`, mutating the
configured list, so a context that declared nothing private still had
the *previous* context's private attributes redacted.

**Requirements**

- [x] I have added test coverage for new or changed functionality
- [x] I have followed the repository's [pull request submission
guidelines](../blob/master/CONTRIBUTING.md#submitting-pull-requests)
- [ ] I have validated my changes against all supported platform
versions — validation across SDKs in progress, results posted as a
comment

**Related issues**

- launchdarkly/ruby-server-sdk#416 — the bug this covers
- #429 — the same change on the `v3` line

<details>
<summary>Implementation details</summary>

**Why the existing tests could not catch it**

`makeEventContextTestParams` creates a *new client* per parameter, so
the `ContextFilter` is always fresh, and every context within one
parameter comes from a single factory with identical
`_meta.privateAttributes`. A leaked private attribute is therefore
always an already-expected private attribute. The existing multi-kind
fixtures don't catch it either: they either set no per-context privates,
or set them only on the kind that is filtered last, and the leaked names
don't exist as attributes on the other kind.

**What the new tests do**

`eventContextPrivateAttributeScoping` uses **one client** for multiple
identify events, and each context carries the same three attributes
(`selfPrivate`, `globallyPrivate`, `visible`) while differing only in
what it declares private:

1. `private attributes of one context are not applied to later contexts`
— identify a context declaring `selfPrivate` private, then identify a
different context that declares nothing private, and assert
`selfPrivate` is still visible on the second one.
2. + 3. `private attributes of one kind are not applied to other kinds
of the same context (declared by org / by user)` — a multi-kind context
where only one kind declares `selfPrivate` private; the other kind must
keep it. Both orderings are covered because SDKs filter the individual
contexts in an arbitrary order — with the Ruby bug present, only the
`org` variant fails (Ruby filters `org` first), so a single-ordering
test would catch this only half the time.

`globallyPrivate` is configured via `GlobalPrivateAttributes` and
asserted redacted in every expectation, so an SDK cannot pass by
throwing away its configured private attributes along with the
per-context ones.

**Verification**

Negative control against ruby-server-sdk with the #416 fix locally
reverted to `.concat(...)`: subtests 1 and 2 fail with the second
context wrongly reporting `redactedAttributes:
["selfPrivate","globallyPrivate"]`; all three pass with the fix in
place.

</details>


Link to Devin session:
https://app.devin.ai/sessions/6e3076285f2849919b966a4f801075ca
Requested by: @kinyoklion

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Adds **three new subtests** under `events/context properties` via
`eventContextPrivateAttributeScoping`, hooked from `EventContexts`.
Existing expectations are unchanged.
> 
> The harness reuses **one SDK client** for multiple identify events so
failures like mutating the global private-attribute list (e.g.
ruby-server-sdk#416) are detectable—prior cases mostly created a fresh
client per scenario.
> 
> **Coverage:** (1) a context that marks `selfPrivate` private must not
cause a later context with no per-context privates to redact
`selfPrivate`; (2) for multi-kind contexts, privates declared on `org`
or `user` must not redact the same attribute on the other kind—both
orderings are tested.
> 
> Every assertion still expects **`globallyPrivate`** (from
`GlobalPrivateAttributes`) to be redacted so SDKs cannot pass by
dropping all configured privates.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
3fa29c4. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Devin AI <devin@launchdarkly.com>
@kinyoklion
kinyoklion merged commit 0a35294 into v3 Aug 19, 2026
7 checks passed
@kinyoklion
kinyoklion deleted the devin/1787072111-private-attribute-scoping-tests branch August 19, 2026 20:43
kinyoklion pushed a commit that referenced this pull request Aug 19, 2026
🤖 I have created a release *beep* *boop*
---


##
[3.2.0-alpha.8](v3.2.0-alpha.7...v3.2.0-alpha.8)
(2026-08-19)


### Features

* Verify per-context private attributes are not applied to other
contexts
([#429](#429))
([0a35294](0a35294))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> **Release 3.2.0-alpha.8** bumps the harness version in `main.go` and
`.release-please-manifest.json` and records the release in
`CHANGELOG.md`.
> 
> The release notes highlight **new contract coverage** from
[#429](#429):
tests that **`_meta.privateAttributes` on one context must not redact
attributes on other contexts** when the same client sends events for
multiple contexts (including alongside globally configured private
attributes). SDKs that merge per-context private lists into a global
list would fail these tests.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
b19e419. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devin-pr PRs created by Devin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants