fix: optimize unsubmitted forms cleanup logic and performance

[Michellebuchiokonicha]

Task-3

Summary:
Refactored the cleanup_unsubmitted_forms job to address correctness, performance, and safety issues identified in the legacy code.

Key Changes:

1. Correctness: Fixed date logic from a rigid 24h window to a catch-all (lt 7 days) to prevent orphaned data.
2. Performance: Resolved N+1 query issue by implementing batch processing (Bulk Read + Bulk Delete). Reduced DB load from O(N) to O(1).
3. Safety: Added strict ID filtering and a take: 1000 limit to prevent memory exhaustion (OOM) on large backlogs.

Walkthrough Video:
https://www.loom.com/share/da0c1f0b3b754003b405eb434cdd8e97

Summary by CodeRabbit

• Chores
  • Optimized the unsubmitted forms cleanup process by consolidating database operations into efficient bulk transactions, improving overall system performance and reducing execution time.

[coderabbitai]

Walkthrough

The cleanup logic for unsubmitted forms was refactored to replace per-token processing and multiple individual transactions with a bulk, date-filtered fetch and a single consolidated transaction. This optimization reduces database queries and improves efficiency while preserving error handling and job status management.

Changes

Cohort / File(s)	Summary
Unsubmitted Forms Cleanup Optimization tests/unsubmitted_forms/cleanup_unsubmitted_forms.ts	Refactored from per-token queries and transactions to bulk date-filtered fetch with single transaction; added early exit for empty results and bulk pre-check to gather unsubmitted relationships by productId, avoiding N+1 queries; consolidated deletion operations (corpus items, relationships, tokens, entities) into one transactional block.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 With eager paws and whiskers bright,
This rabbit bundled queries tight—
No more one-by-one, row after row,
Bulk operations steal the show!
From scattered tokens, cleanup flows so clean. ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: it addresses both the optimization of the cleanup logic and performance improvements through bulk processing and elimination of N+1 queries.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}