Skip to content

feat(Odin): Integrate actions in checkin worker#535

Open
vikramlc-cognite wants to merge 12 commits into
masterfrom
EDGE-182-actions-integrate-checkin-worker
Open

feat(Odin): Integrate actions in checkin worker#535
vikramlc-cognite wants to merge 12 commits into
masterfrom
EDGE-182-actions-integrate-checkin-worker

Conversation

@vikramlc-cognite
Copy link
Copy Markdown
Contributor

@vikramlc-cognite vikramlc-cognite commented May 27, 2026

PR Short Summary

  • Wires up action dispatch in CheckinWorker: every checkin and startup response can now carry pending_actions, which are handed off to a configurable dispatcher callback, and any action outcome reports (action_updates) are carried back in the next outgoing checkin request.

Type of change

  • Bug fix
  • New feature (non-breaking change that adds functionality)
  • Breaking change
  • Refactor
  • Documentation update
  • Chore / tooling / CI

What changed

cognite/extractorutils/unstable/core/checkin_worker.py

  • New state: _action_updates: list[ActionUpdate], _action_dispatcher: Callable | None
  • set_action_dispatcher(dispatcher) — registers the callback; calling it twice replaces the previous one
  • queue_action_update(update) — thread-safe append under _lock; intended to be called from action handler threads
  • _handle_checkin_response — fires _action_dispatcher on a named daemon Thread("ActionDispatcher") when pending_actions is present, so action
    execution never blocks the checkin interval
  • try_write_checkin — drains _action_updates under _lock before each POST, passes them as action_updates (omitted when empty via exclude_none)
  • _requeue_checkin — gained action_updates parameter; prepends (not appends) failed-batch updates back to _action_updates to preserve send order
    across retries

tests/test_unstable/test_checkin_worker_actions.py (new file)

  • 8 tests covering all acceptance criteria, following the same requests_mock + gzip-decoded body pattern as test_checkin_worker.py

Why it changed

  • Implements EDGE-182: actions integration — checkin worker needs to parse pending_actions from Odin responses, dispatch them to extractor logic, and carry back outcome reports.
  • Odin spec: every checkin/startup response can include pending actions; the worker is the sole interface to Odin, so dispatch must live here

What to focus on during review

  • _requeue_checkin uses prepend, not append ((action_updates or []) + self._action_updates). This preserves ordering across retries: updates from a failed batch must precede updates that arrived after the failure. The test test_action_updates_requeue_order_preserved_across_retry validates this.

  • Daemon thread for dispatch — the ActionDispatcher thread is daemon so it doesn't block extractor shutdown. The trade-off is that in-flight action handling is dropped on process exit without a clean shutdown window.

  • Lock scope in try_write_checkin — the drain (_action_updates[:] + .clear()) is a single critical section. A queue_action_update racing the drain either lands in the current batch (wins the lock first) or the next one (loses it) — never lost.

  • extra="ignore" on Action and CheckinResponse — deliberate: Odin may add fields before the SDK is updated.


Test evidence

  • All tests pass.

Risks and unknowns

  • If an action handler calls queue_action_update after the drain in try_write_checkin but before the POST succeeds or fails, that update is not part
    of the current batch and will go out on the next checkin — this is correct but means outcome reporting has at most one checkin interval of latency.
  • No backpressure on _action_updates: if action handlers produce updates faster than checkins can flush them, the list grows unbounded. Not a
    concern at current action volumes but worth noting.

Rollout and rollback

  • No migration, no feature flag. _action_dispatcher defaults to None, so all existing extractors that don't call set_action_dispatcher are completely unaffected — the new code paths are dead without a dispatcher registered.

Checklist

  • Self-reviewed the diff
  • Tests added (test_checkin_worker_actions.py — 8 tests)
  • Docs: N/A (internal API, no public docs affected)
  • No secrets, credentials, or PII committed
  • No breaking changes — all new fields on existing models are | None with defaults

@vikramlc-cognite vikramlc-cognite self-assigned this May 27, 2026
@vikramlc-cognite
Copy link
Copy Markdown
Contributor Author

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Extraction Pipelines 2.0 Actions by adding new DTOs (such as Action, ActionUpdate, and ActionStatus) and integrating action dispatching and queuing within the check-in worker. It also includes comprehensive unit tests for the new action dispatching mechanics and DTO validation. The review feedback highlights a critical issue where spawning a daemon thread for the action dispatcher without a try-except block can cause unhandled exceptions to silently terminate the thread, making troubleshooting difficult. A code suggestion is provided to safely wrap the dispatcher call and log any exceptions.

Comment thread cognite/extractorutils/unstable/core/checkin_worker.py
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for "Actions" in the checkin worker. It adds new action-related DTOs (such as ActionType, ActionStatus, Action, and ActionUpdate), updates checkin requests and responses to include actions, and implements action dispatching and queuing within the CheckinWorker. Additionally, comprehensive unit tests have been added. The reviewer recommended wrapping the spawned action dispatcher thread's target in a try-except block to prevent silent failures and ensure exceptions are properly logged.

Comment thread cognite/extractorutils/unstable/core/checkin_worker.py
@vikramlc-cognite vikramlc-cognite changed the base branch from master to EDGE-178-add-DTOs-actions May 28, 2026 02:16
@vikramlc-cognite
Copy link
Copy Markdown
Contributor Author

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces action dispatching capabilities to the CheckinWorker class, allowing users to set an action dispatcher callback and queue action updates thread-safely to be included in outgoing check-in requests. It also spawns a daemon thread to execute the dispatcher when pending actions are received in a check-in response, and includes a comprehensive test suite to verify this functionality. The reviewer recommends wrapping the action dispatcher execution in a try-except block inside the spawned thread to prevent silent failures and ensure any unhandled exceptions are properly logged.

Comment thread cognite/extractorutils/unstable/core/checkin_worker.py
@vikramlc-cognite vikramlc-cognite changed the title Edge 182 actions integrate checkin worker feat(Odin): Integrate actions in checkin worker May 28, 2026
@vikramlc-cognite vikramlc-cognite marked this pull request as ready for review May 28, 2026 05:13
@vikramlc-cognite vikramlc-cognite requested a review from a team as a code owner May 28, 2026 05:13
Base automatically changed from EDGE-178-add-DTOs-actions to master May 28, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants