Add rewind support#296
Merged
Merged
Conversation
Implement full orchestration rewind in the core SDK so a failed
orchestration can be replayed from the point of failure. The client-side
`rewindInstance` already existed but was a no-op without the worker and
backend pieces; this wires up the whole path.
Backends do not compute the rewound history themselves: they append an
`ExecutionRewoundEvent`, flip the instance to RUNNING, and re-dispatch.
The worker detects the rewind, rewrites the history (dropping failed
task/sub-orchestration results and the terminal `executionCompleted`),
and returns a `RewindOrchestrationAction { newHistory }` that the
backend applies.
Changes:
- proto: re-sync orchestrator_service.proto from durabletask-protobuf main,
adding `RewindOrchestrationAction` and `OrchestratorAction.rewindOrchestration = 9`;
regenerate the JS/TS stubs.
- worker: new `buildRewindResult` (worker/rewind.ts) plus an executor
trigger before replay and an informational `EXECUTIONREWOUND` no-op case.
- in-memory backend: `rewindInstance` / `prepareRewind` /
`processRewindOrchestrationAction` (incl. recursive sub-orchestration
rewind and purged-sub re-creation), an `executionCompleted` bookend on
terminal completion, an explicit failureDetails presence guard, and a
`rewindOrchestration` test-client method.
- helpers: `newExecutionCompletedEvent` and `newExecutionRewoundEvent`.
- tests: 14 buildRewindResult unit cases and 8 in-memory e2e cases.
Known limitation (shared with the .NET/Python/Go SDKs): timer events between
RetryPolicy retry attempts are not removed, so rewinding a retry-policy-wrapped
activity is unsupported.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds end-to-end rewind support to the Durable Task JS SDK (worker + in-memory backend) so failed orchestrations can be reset to runnable state while preserving prior successful results, matching the durabletask-python reference behavior.
Changes:
- Added worker-side history rewrite logic (
buildRewindResult) and executor short-circuiting when a rewind is detected. - Implemented in-memory backend rewind handling, including terminal
executionCompletedbookend and processing ofRewindOrchestrationAction(with recursive sub-orchestration rewind). - Updated protocol/proto stubs for
RewindOrchestrationActionand related upstream additive proto changes; added unit + e2e tests for rewind.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/durabletask-js/src/worker/rewind.ts | Implements history rewrite to drop failed activity/sub-orchestration results and produce a RewindOrchestrationAction. |
| packages/durabletask-js/src/worker/orchestration-executor.ts | Detects rewind work items and short-circuits to buildRewindResult; adds no-op handling for executionRewound events. |
| packages/durabletask-js/src/testing/in-memory-backend.ts | Adds in-memory backend rewind entrypoint, applies rewind action, appends terminal executionCompleted bookend, and supports recursive sub-orchestration rewind. |
| packages/durabletask-js/src/testing/test-client.ts | Exposes rewind operation from the test client to the in-memory backend. |
| packages/durabletask-js/src/utils/pb-helper.util.ts | Adds helpers for creating executionCompleted and executionRewound history events. |
| packages/durabletask-js/test/build-rewind-result.spec.ts | Unit tests for the worker-side history rewrite behavior. |
| packages/durabletask-js/test/rewind-e2e.spec.ts | In-memory backend e2e tests covering rewind scenarios, including sub-orchestrations and purged sub-orchestrations. |
| internal/protocol/protos/orchestrator_service.proto | Proto updates including RewindOrchestrationAction and other additive upstream drift. |
| packages/durabletask-js/src/proto/orchestrator_service_pb.js | Regenerated JS proto stubs reflecting proto updates. |
| packages/durabletask-js/src/proto/orchestrator_service_pb.d.ts | Regenerated TS declarations reflecting proto updates. |
| internal/protocol/SOURCE_COMMIT | Updates the recorded upstream protocol source commit. |
YunchuWang
commented
Jul 13, 2026
The rewind e2e cases that exercise real backend rewind (fail -> rewind -> succeed, plus the running/terminated negative cases) were it.skip'd because earlier DTS emulator builds returned UNIMPLEMENTED. The current emulator (mcr.microsoft.com/dts/dts-emulator:latest) supports rewind and delegates the history rewrite to the SDK worker, so they now pass end-to-end against buildRewindResult. - Un-skip the 4 backend-support cases and add per-test timeouts (the two positive cases run two 30s completion waits each, exceeding jest's 5s default). - Fix waitingOrchestrator to the generator style (async function* + yield ctx.waitForExternalEvent), matching every other e2e test. The plain async/await form does not suspend in durabletask-js, so the orchestrator completed immediately instead of staying Running, preventing the terminated case from reaching TERMINATED. All 7 rewind e2e cases pass against the emulator. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Mirror processCreateSubOrchestrationAction: the purged-sub re-creation path in processRewindOrchestrationAction now passes parentInstance metadata (name, instanceId, taskScheduledId) so the re-created sub keeps its parent link and can route its completion back to the parent orchestration. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
kaibocai
approved these changes
Jul 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds rewind support to the durabletask-js SDK, reaching parity with the Python reference implementation in microsoft/durabletask-python#121. Rewind resets a failed orchestration to a runnable state so that failed activities re-run while previously successful results are preserved.
Before this change, durabletask-js had only the client entry point (
client.rewindInstance(instanceId, reason)); the worker-side history rewrite and the in-memory backend support were missing, so rewind was a no-op end-to-end.Architecture
Rewind spans three layers, split exactly as in the Python reference:
rewindInstance(instanceId, reason)sends aRewindInstanceRequestover gRPC.FAILED, appends anExecutionRewoundEvent, flips status toRUNNING, and re-dispatches the work item. On terminal completion it now appends anexecutionCompleted"bookend" event — this is the precondition the worker uses to detect a rewind.executionRewoundin the new events andexecutionCompletedin the committed history, it short-circuits normal replay and callsbuildRewindResult(), which rewrites history — droppingtaskFailed, the matching failedtaskScheduled,subOrchestrationInstanceFailed, andexecutionCompleted; assigning a fresh execution id toexecutionStarted; keeping successful results and theexecutionRewoundevent — wrapped in aRewindOrchestrationAction. The backend then applies the new history and recursively rewinds failed sub-orchestrations.Changes
orchestrator_service.protofromdurabletask-protobufmain and regenerated the JS/TS stubs. AddsRewindOrchestrationActionandOrchestratorAction.rewindOrchestration = 9. The re-sync also pulls in benign, purely-additive upstream drift:ActivityRequest.tags,PurgeInstanceFilter.timeout, and[deprecated=true]annotations onOrchestratorResponse.isPartial/chunkIndex. No fields were renumbered or removed.worker/rewind.ts(buildRewindResult);orchestration-executor.tsgains the rewind short-circuit and an informationalEXECUTIONREWOUNDno-op case.rewindInstance,prepareRewind, andprocessRewindOrchestrationAction(including recursive and purged sub-orchestration handling), the terminalexecutionCompletedbookend, and afailureDetailspresence guard. New helpersnewExecutionCompletedEvent/newExecutionRewoundEvent; newrewindOrchestration()test-client method.Tests
buildRewindResult(test/build-rewind-result.spec.ts).test/rewind-e2e.spec.ts): failed activity, preserves successful results, not-found, non-failed instance, sub-orchestration, purged sub-orchestration, no reason, and rewind-twice.build:coreandlintare green.Known limitation
Consistent with the Core/.NET/Python implementations: timer events emitted between retry attempts of an activity scheduled with a
RetryPolicyare not removed during the history rewrite, so rewinding a retry-policy activity is not currently supported (documented inworker/rewind.ts).Reference
Ports andystaples/durabletask-python@d5052d7 (
worker._build_rewind_result,testing/in_memory_backend.py,internal/helpers.new_execution_completed_event, and the corresponding unit + e2e tests).