Skip to content

feat: Add contract test for undecodable data-source payload - #419

Open
jsonbailey wants to merge 2 commits into
v2from
jb/decode-contract-test
Open

feat: Add contract test for undecodable data-source payload#419
jsonbailey wants to merge 2 commits into
v2from
jb/decode-contract-test

Conversation

@jsonbailey

@jsonbailey jsonbailey commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What

Adds a server-side polling contract test that verifies an SDK does not silently corrupt flag data when a data-source response body cannot be decoded as valid UTF-8.

The mock polling service serves a body that is valid JSON except for a single invalid UTF-8 byte (0xFF) spliced into the middle of a flag's string value. Because the bad byte sits inside a JSON string, the body is structurally still JSON:

  • An SDK that decodes the body strictly raises on the invalid byte, rejects the whole payload, and never initializes from it — so an evaluation returns the caller's default.
  • An SDK that decodes with byte replacement (e.g. Python str.decode(errors='replace')) turns the 0xFF into U+FFFD, parses the now-valid JSON, and applies a corrupted flag value. This is the latent bug the test catches.

The test evaluates the flag (off-variation whose value is the corrupted string) and asserts the SDK returns the evaluation default, i.e. it did not apply data from an undecodable payload. If the SDK returns the U+FFFD value "AB<U+FFFD>CD", the test fails with an explicit message.

Gating

Baked into the existing server-side polling suite (doServerSidePollTests) — no new capability. It runs by default for every polling-capable server-side SDK; the polling group already requires the server-side-polling capability.

If it turns out some SDKs legitimately fail (i.e. we decide lenient decoding is acceptable for them), a dedicated opt-in capability can be added later to gate it.

How it works

  • Builds a valid flag payload via the mockld data builder, serializes it, then splices 0xFF into the flag's string value.
  • Serves the raw bytes through a custom http.Handler on a NewMockEndpoint (no new mock-service plumbing needed).
  • Configures the client with initCanFail: true and a short start-wait so the client is created in both SDK states (strict SDK stays uninitialized and answers with the default; lenient SDK initializes and answers with the corrupted value).

Verification (Python async SDK, before/after)

Demonstrated against the LaunchDarkly Python SDK's async data source, which had this exact bug (fixed in python-server-sdk#489, which switched the async transport from response.text(errors='replace') to strict decoding):

  • Before the fix (errors='replace'): the test fails — the SDK initialized from the corrupt body and served the U+FFFD-corrupted value {"value": "AB<U+FFFD>CD", "variationIndex": 0, "reason": {"kind": "OFF"}}.
  • After the fix (strict decode): the test passes — the SDK rejects the undecodable payload and returns the default.

Confirmed the test runs (not skipped) under the stock server-side capability set.


Note

Overview
Adds a server-side polling contract test that checks SDKs reject poll responses whose body is not valid UTF-8, instead of applying flag data after lenient byte replacement (e.g. 0xFF → U+FFFD).

The harness builds a normal flag payload, splices an invalid byte into a string variation, and serves it from a mock HTTP endpoint. It then evaluates the flag with initCanFail and a short start-wait, and expects the evaluation default—not the corrupted "ABCD" value that a replace-decoding SDK would use.

The new undecodable payload subtest is wired into doServerSidePollTests alongside existing request and payload tests; it uses the existing server-side-polling capability gate.

Reviewed by Cursor Bugbot for commit 73cd524. Bugbot is set up for automated code reviews on this repo. Configure here.

@jsonbailey
jsonbailey requested a review from a team as a code owner August 13, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants