Skip to content

fix: deprovision deactivated directory users in WorkOS backfill - #5052

Open
adaam2 wants to merge 3 commits into
mainfrom
chore/dsync-bug
Open

fix: deprovision deactivated directory users in WorkOS backfill#5052
adaam2 wants to merge 3 commits into
mainfrom
chore/dsync-bug

Conversation

@adaam2

@adaam2 adaam2 commented Aug 7, 2026

Copy link
Copy Markdown
Member

Why

Production orgs retain members who have left the company. The dsync.user.created / dsync.user.updated handlers ignored the state property until #4283, so SCIM deactivations (which WorkOS emits as updates with state=inactive, not deletes) were consumed as plain upserts. The per-org event cursor advanced past those events, and the WorkOS Events API only retains ~30 days, so the fixed handlers can never replay them. The result is orphaned org relationships and role assignments for deactivated directory users.

What

Extends the existing snapshot-driven BackfillWorkOSOrganization activity (run via the manually triggered BackfillWorkOSWorkflow) to reconcile directory users:

  • New ListDirectoryUsers wrapper on the WorkOS client (paginated, boundary type, no vendor leakage), plus ListDirectories/ListDirectoryUsers on the WorkOSClient interface and stub.
  • For each directory user in the snapshot:
    • state=active → upsert the directory user row (restoring soft-deleted rows), linking by normalized email.
    • non-active state → soft-delete the row and run deprovisionOrganizationAccess (relationship tombstone, role-assignment revocation, user-info cache invalidation), mirroring what the fixed event handler does.
  • runPostCommitEffects converted from a method to a shared free function so the backfill reuses the same post-commit cache invalidation as event processing.

Guard semantics (the important bit)

Directory-user reconciliation guards are timestamp-only and deliberately ignore workos_last_event_id. The pre-#4283 handlers advanced the event cursor on exactly the rows this backfill exists to repair, so ShouldProcessEvent's cursor branch would skip all of them. A fresh snapshot cannot be stale, so timestamps arbitrate instead. Deprovision writes are stamped with the reconcile time so older queued events cannot resurrect access while genuinely newer events still apply.

Rollout

Trigger BackfillWorkOSWorkflow from Temporal (scoped to the affected org first via workos_organization_id, then org-wide). Idempotent and safe to run alongside the live event stream.

Tests

  • Inactive snapshot user deprovisions access despite an advanced event cursor (directory row soft-deleted, relationship + assignments tombstoned, user-info cache invalidated).
  • Active snapshot user upserts and links to the Gram user by email.
  • Snapshot older than local row is skipped.

Summary by cubic

Deprovisions deactivated WorkOS Directory Sync users during backfill to remove lingering org access. Also skips deprovision when a relationship was updated after the snapshot to avoid rolling back newer changes.

  • Bug Fixes
    • Extend BackfillWorkOSOrganization to reconcile directory users from WorkOS snapshots.
    • Inactive users: soft-delete directory user, run deprovisionOrganizationAccess, invalidate the user-info cache, and stamp deprovision with the snapshot updated_at. Recover the linked Gram user from stored user_id when the directory email changed. Skip deprovision if the membership’s workos_updated_at is newer than the snapshot.
    • Active users: upsert/restore directory user and link to the Gram user by normalized email.
    • Guards compare timestamps only and ignore workos_last_event_id to repair cases where cursors advanced past deactivations.
    • Share post-commit effects via a new runPostCommitEffects function; NewBackfillWorkOSOrganization now takes a cache adapter.
    • Add WorkOS client methods ListDirectories and ListDirectoryUsers (plus stub support) to fetch directory snapshots.

Written for commit ad09d74. Summary will update on new commits.

Review in cubic

@adaam2
adaam2 requested a review from a team as a code owner August 7, 2026 13:23
@adaam2 adaam2 added bug Something isn't working go Pull requests that update go code labels Aug 7, 2026
@changeset-bot

changeset-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: ad09d74

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/background/activities/backfill_workos_organization.go Outdated
Comment thread server/internal/background/activities/backfill_workos_organization.go Outdated
Comment thread server/internal/background/activities/backfill_workos_organization.go Outdated
@adaam2
adaam2 requested a review from tgmendes August 7, 2026 13:45

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/background/activities/backfill_workos_test.go

@tgmendes tgmendes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We never ran or tested this backfill before, and it's a file that needs to be deleted.

We opted at the time to do this backfill via a one off script: https://github.com/speakeasy-api/gram/pull/2894/changes. It was never merged because it wasn't prod ready, but should be easy to revive and adapt.

I think the right path forward is:

  1. Create a script similar to that one
  2. Run the backfill
  3. Update the sync job to properly handle dir sync deactivations (if not done so already)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants