Skip to content

feat: Verify SDKs read all three persistent-store tombstone shapes - #438

Open
jsonbailey wants to merge 1 commit into
v2from
jb/sdk-2959/tombstone-contract-tests
Open

feat: Verify SDKs read all three persistent-store tombstone shapes#438
jsonbailey wants to merge 1 commit into
v2from
jb/sdk-2959/tombstone-contract-tests

Conversation

@jsonbailey

@jsonbailey jsonbailey commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

SDK-2959

What

Adds contract tests that plant tombstone records directly in a persistent store and check that the SDK reading them behaves correctly, for each of the three tombstone shapes our SDKs write.

A tombstone is the record an SDK writes to mark an item deleted, so that version-based conflict resolution can reject a stale out-of-order update. The store already addresses the record by key (Redis hash field, Consul key, DynamoDB sort key), so the key inside the JSON body is redundant. Three incompatible shapes are in circulation:

Shape Written by
{"version":N,"deleted":true} .NET, Java, Node (Redis upsert path), Haskell
{"key":"<real-key>","version":N,"deleted":true} Python, Ruby, C++, Erlang, Node (init path)
full flag/segment object with "key":"$deleted" Go v5+, Relay Proxy, Rust

Two confirmed bugs motivated this: Python failed its entire all-flags read on the keyless shape (SDK-2941 / FROPS-511), and Rust parses the Go/Relay shape as a live flag, resurrecting deleted flags. These tests prove no SDK depends on the inner key, which is the prerequisite for eventually dropping it.

Tests

New file sdktests/server_side_persistence_tombstones.go, registered with one line from ServerSidePersistentTests.Run inside the existing daemon mode sub-tree (no data source, CacheModeOff). No new capability: it runs automatically for every store an SDK declares (persistent-data-store-redis / -consul / -dynamodb).

tombstones/flags/<shape> — plants a valid flag plus a tombstone in one features write, then asserts:

  1. the valid flag still evaluates to its expected value (catches a reader that fails the whole collection on one unrecognized record);
  2. the tombstoned key is absent from all-flags state, both as a value and in $flagsState (catches a reader that resurrects a tombstone as a live flag);
  3. evaluating the tombstoned key returns the caller's default with reason FLAG_NOT_FOUND, matching what the neighbouring daemon-mode tests assert.

tombstones/segments/<shape> — plants a tombstone in the segments namespace plus a flag whose rule matches that segment key, then asserts the valid flag is still readable and the segment-matching flag serves its no-match variation. The $deleted full-object segment includes the evaluation context in its included list, so an SDK that reads it as a live segment would serve matched instead.

All assertions are behavioral, so they read identically across Redis, Consul, and DynamoDB. Every planted shape carries a parseable version, which DynamoDB's WriteMap requires.

Out of scope

No changes to basicDeletedFlagValidationMatcher, no v3 port (the file is self-contained so the port is a copy plus one line), no capability changes, no CHANGELOG/version edits.

Verification

make lint0 issues. make build, go vet ./..., and make test all pass. The tests themselves need a live SDK test service plus Redis/Consul/DynamoDB, so they were not executed end to end here.

To run against an SDK, start Redis / Consul / DynamoDB-local and the SDK's test service, then:

./sdk-test-harness -url http://localhost:8000 \
  -enable-persistence-tests \
  -run 'persistent data store/./daemon mode/tombstones'

The full test path is persistent data store/<store>/daemon mode/tombstones/{flags,segments}/<shape>. Drop the -run filter to exercise the whole persistence suite.


Note

Overview
Adds daemon-mode contract tests that plant deleted-item tombstones in Redis/Consul/DynamoDB and assert SDKs treat them as deleted, regardless of which of the three JSON shapes other SDKs write.

For flags, each shape is stored next to a live flag. The suite checks that the live flag still evaluates, the tombstoned key is absent from all-flags (value and $flagsState), and a direct eval returns the default with FLAG_NOT_FOUND. For segments, a tombstone sits in the segments collection while a segment-match flag must serve the no-match variation (the $deleted full-object shape includes the context so a false resurrection would match).

Tests run under existing persistent data store/*/daemon mode with cache off and no new capability.

Reviewed by Cursor Bugbot for commit c95ea45. Bugbot is set up for automated code reviews on this repo. Configure here.

@jsonbailey
jsonbailey requested a review from a team as a code owner August 21, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants