fix: deprovision deactivated directory users in WorkOS backfill - #5052
fix: deprovision deactivated directory users in WorkOS backfill#5052adaam2 wants to merge 3 commits into
Conversation
|
There was a problem hiding this comment.
All reported issues were addressed across 6 files
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
…stamp deprovision with snapshot time
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
tgmendes
left a comment
There was a problem hiding this comment.
We never ran or tested this backfill before, and it's a file that needs to be deleted.
We opted at the time to do this backfill via a one off script: https://github.com/speakeasy-api/gram/pull/2894/changes. It was never merged because it wasn't prod ready, but should be easy to revive and adapt.
I think the right path forward is:
- Create a script similar to that one
- Run the backfill
- Update the sync job to properly handle dir sync deactivations (if not done so already)
Why
Production orgs retain members who have left the company. The
dsync.user.created/dsync.user.updatedhandlers ignored thestateproperty until #4283, so SCIM deactivations (which WorkOS emits as updates withstate=inactive, not deletes) were consumed as plain upserts. The per-org event cursor advanced past those events, and the WorkOS Events API only retains ~30 days, so the fixed handlers can never replay them. The result is orphaned org relationships and role assignments for deactivated directory users.What
Extends the existing snapshot-driven
BackfillWorkOSOrganizationactivity (run via the manually triggeredBackfillWorkOSWorkflow) to reconcile directory users:ListDirectoryUserswrapper on the WorkOS client (paginated, boundary type, no vendor leakage), plusListDirectories/ListDirectoryUserson theWorkOSClientinterface and stub.state=active→ upsert the directory user row (restoring soft-deleted rows), linking by normalized email.deprovisionOrganizationAccess(relationship tombstone, role-assignment revocation, user-info cache invalidation), mirroring what the fixed event handler does.runPostCommitEffectsconverted from a method to a shared free function so the backfill reuses the same post-commit cache invalidation as event processing.Guard semantics (the important bit)
Directory-user reconciliation guards are timestamp-only and deliberately ignore
workos_last_event_id. The pre-#4283 handlers advanced the event cursor on exactly the rows this backfill exists to repair, soShouldProcessEvent's cursor branch would skip all of them. A fresh snapshot cannot be stale, so timestamps arbitrate instead. Deprovision writes are stamped with the reconcile time so older queued events cannot resurrect access while genuinely newer events still apply.Rollout
Trigger
BackfillWorkOSWorkflowfrom Temporal (scoped to the affected org first viaworkos_organization_id, then org-wide). Idempotent and safe to run alongside the live event stream.Tests
Summary by cubic
Deprovisions deactivated WorkOS Directory Sync users during backfill to remove lingering org access. Also skips deprovision when a relationship was updated after the snapshot to avoid rolling back newer changes.
BackfillWorkOSOrganizationto reconcile directory users from WorkOS snapshots.deprovisionOrganizationAccess, invalidate the user-info cache, and stamp deprovision with the snapshotupdated_at. Recover the linked Gram user from storeduser_idwhen the directory email changed. Skip deprovision if the membership’sworkos_updated_atis newer than the snapshot.workos_last_event_idto repair cases where cursors advanced past deactivations.runPostCommitEffectsfunction;NewBackfillWorkOSOrganizationnow takes a cache adapter.ListDirectoriesandListDirectoryUsers(plus stub support) to fetch directory snapshots.Written for commit ad09d74. Summary will update on new commits.