feat: Add contract test for undecodable data-source payload - #419
Open
jsonbailey wants to merge 2 commits into
Open
feat: Add contract test for undecodable data-source payload#419jsonbailey wants to merge 2 commits into
jsonbailey wants to merge 2 commits into
Conversation
keelerm84
approved these changes
Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a server-side polling contract test that verifies an SDK does not silently corrupt flag data when a data-source response body cannot be decoded as valid UTF-8.
The mock polling service serves a body that is valid JSON except for a single invalid UTF-8 byte (
0xFF) spliced into the middle of a flag's string value. Because the bad byte sits inside a JSON string, the body is structurally still JSON:str.decode(errors='replace')) turns the0xFFinto U+FFFD, parses the now-valid JSON, and applies a corrupted flag value. This is the latent bug the test catches.The test evaluates the flag (off-variation whose value is the corrupted string) and asserts the SDK returns the evaluation default, i.e. it did not apply data from an undecodable payload. If the SDK returns the U+FFFD value
"AB<U+FFFD>CD", the test fails with an explicit message.Gating
Baked into the existing server-side polling suite (
doServerSidePollTests) — no new capability. It runs by default for every polling-capable server-side SDK; the polling group already requires theserver-side-pollingcapability.If it turns out some SDKs legitimately fail (i.e. we decide lenient decoding is acceptable for them), a dedicated opt-in capability can be added later to gate it.
How it works
0xFFinto the flag's string value.http.Handleron aNewMockEndpoint(no new mock-service plumbing needed).initCanFail: trueand a short start-wait so the client is created in both SDK states (strict SDK stays uninitialized and answers with the default; lenient SDK initializes and answers with the corrupted value).Verification (Python async SDK, before/after)
Demonstrated against the LaunchDarkly Python SDK's async data source, which had this exact bug (fixed in python-server-sdk#489, which switched the async transport from
response.text(errors='replace')to strict decoding):errors='replace'): the test fails — the SDK initialized from the corrupt body and served the U+FFFD-corrupted value{"value": "AB<U+FFFD>CD", "variationIndex": 0, "reason": {"kind": "OFF"}}.Confirmed the test runs (not skipped) under the stock server-side capability set.
Note
Overview
Adds a server-side polling contract test that checks SDKs reject poll responses whose body is not valid UTF-8, instead of applying flag data after lenient byte replacement (e.g.
0xFF→ U+FFFD).The harness builds a normal flag payload, splices an invalid byte into a string variation, and serves it from a mock HTTP endpoint. It then evaluates the flag with
initCanFailand a short start-wait, and expects the evaluation default—not the corrupted"ABCD"value that a replace-decoding SDK would use.The new
undecodable payloadsubtest is wired intodoServerSidePollTestsalongside existing request and payload tests; it uses the existing server-side-polling capability gate.Reviewed by Cursor Bugbot for commit 73cd524. Bugbot is set up for automated code reviews on this repo. Configure here.