feat(jobs): queue topology, DLQs, and cron discovery (AP-21)#98
Conversation
AP-21 Phase 4: Implement jobs queue topology and DLQs
Context
Source docs
ScopeImplement cron discovery and Cloudflare Queue consumers for Out of scopeDo not move all lifecycle purge ownership or implement bundle generation in this ticket. DependenciesNone. Implementation notesQueue handlers are idempotent by target identity, not queue message identity. Add Worker bindings/types when Wrangler config changes. AcceptanceJobs Worker has queue consumers, DLQ behavior, cron discovery, tests, and documented retry/terminal-failure behavior. VerificationRun jobs tests, typegen/checks for Worker bindings, and Remote Cursor handoffStart by reading |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
WalkthroughThe pull request implements the Phase 4 jobs worker queue topology by adding cron-driven discovery and message handlers for the job queue system. It defines Zod-based schemas for three queue message types (byte-purge, safety-scan, bundle-generate) with a shared discriminated-union contract and exports queue-target skip-reason logic to determine when revision work should be skipped based on status fields. The cron scheduler routes events by schedule constant to discovery routines: upload-cleanup queries expired sessions and enqueues byte-purge messages with deduplicated R2 prefixes; auto-deletion and maintenance-gc discover expirable artifacts and age-limited records; retention is a no-op placeholder. Queue handlers parse incoming messages and decide between skip/defer/process paths with conditional acking and retry behavior. The worker integrates cron and queue handlers, disables all processing when JOBS_ENABLED is false, and includes complete Wrangler configuration with scheduled triggers, queue/DLQ bindings, and environment-specific resource identifiers for preview and production. Tests cover routing, handler failure isolation, and message contract validation. Documentation clarifies retry/DLQ semantics and marks the queue topology feature complete. Possibly related issues
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Comment |
|
agent-paste PR preview is ready. API: https://agent-paste-api-pr-98.isaac-a46.workers.dev |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/ops/status/phase-backlog.md (1)
115-117:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRemove the already-completed DLQ-consumer scope from Phase 4 item 6.
Line 117 still lists “bundle-generate DLQ consumer” as pending, but item 4 and this PR mark that work complete. Keep item 6 focused on actual remaining bundle generation/download work.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/ops/status/phase-backlog.md` around lines 115 - 117, Update Phase 4 item 6 in docs/ops/status/phase-backlog.md by removing the completed scope "bundle-generate DLQ consumer" from the bullet so the item only lists remaining bundle generation/download work (e.g., keep "Add bundle status fields, deterministic R2 bundle keys, bundle size caps, Agent View bundle state" and delete the "bundle-generate DLQ consumer" phrase), ensuring the item reflects current pending work.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/jobs/src/discovery/maintenance-gc.ts`:
- Around line 10-27: The current delete queries executed via executor.query in
maintenance-gc.ts (the idempotency and operation_events deletes using
idempotencyCutoff and auditCutoff) are unbounded and can remove all matching
rows in one call; instead enforce MAINTENANCE_GC_SWEEP_CAP by performing bounded
deletes: compute remaining = MAINTENANCE_GC_SWEEP_CAP, run the idempotency
delete with a LIMIT of remaining (or with a CTE deleting and returning up to
remaining rows), decrement remaining by idempotency.rows.length, then if
remaining > 0 run the audit delete similarly limited, and set cap_hit based on
whether discovered >= MAINTENANCE_GC_SWEEP_CAP; update the queries/variables
idempotency, audit, discovered, cap_hit accordingly so each executor.query
deletes at most the remaining cap.
In `@apps/jobs/src/discovery/upload-cleanup.ts`:
- Around line 40-45: The code currently flips upload_sessions to
status='expired' via executor.query before ensuring queue.send succeeded, which
can make sessions unrecoverable; change the flow in upload-cleanup.ts so you
iterate per sessionId (the same sessionIds used now), call queue.send for each
and only update that session row to 'expired' (or leave as-is) after a
successful enqueue or when a no-op is appropriate; update both places where
executor.query sets status='expired' (the block around executor.query([...]) and
the similar block at lines 59-69) to perform per-session updates after
queue.send succeeds rather than batching the status change up-front.
In `@apps/jobs/src/handlers/bundle-generate.ts`:
- Around line 19-22: The BundleGenerateMessage.parse and loadRevisionState calls
are happening outside the per-message try/catch so parse/query failures can
abort the whole batch; wrap each message's work in its own try/catch (for both
handlers/loops using BundleGenerateMessage.parse and loadRevisionState) so that
parsing and state loading occur inside the try, and on error call
message.retry() (or the existing retry flow) and continue to the next message;
ensure you reference the same control flow used elsewhere in this file so only
the failing message is retried and other messages keep processing.
In `@apps/jobs/src/handlers/byte-purge.ts`:
- Around line 11-13: The current loop calls BytePurgeMessage.parse(message.body)
outside the per-message try/catch which allows parse errors to escape and abort
batch processing; move the call to BytePurgeMessage.parse(...) inside the try
block that processes each message so parsing errors are caught locally and the
loop can continue handling subsequent messages (update the for (const message of
messages) { ... } block to call BytePurgeMessage.parse within the try
surrounding processing logic).
In `@apps/jobs/src/handlers/safety-scan.ts`:
- Around line 18-21: Move SafetyScanMessage.parse(message.body) and the
loadRevisionState(executor, payload.workspace_id, payload.revision_id) call
inside the per-message try/catch block so any parse errors or DB read failures
are handled per message; specifically, in the messages loop wrap parsing and
state loading in the same try that currently surrounds processing (referencing
SafetyScanMessage.parse and loadRevisionState) so exceptions from those calls
trigger the per-message retry/ack logic rather than aborting the entire batch.
In `@apps/jobs/src/op-log.ts`:
- Around line 13-23: logOpError currently only writes JSON to console.error so
errors never reach Sentry; import and use `@sentry/cloudflare` inside logOpError
(or call Sentry if initialized elsewhere) to send the error to Sentry in
addition to console output: call Sentry.captureException or
Sentry.captureMessage with the event name and attach the fields as extra/context
(include component:"jobs" and at timestamp as tags/extra), leaving the
console.error call intact for local logs and ensuring the Sentry SDK
(`@sentry/cloudflare`) is referenced where appropriate.
- Around line 1-23: Both logOp and logOpError allow user-supplied keys to
overwrite fixed metadata and can throw if fields are non-serializable; to fix,
ensure your fixed metadata (level, component, event, at) take precedence by
placing them after spreading user fields or, better, namespace user data under a
single key (e.g., "fields") so collisions cannot occur, and wrap the
JSON.stringify call in a try/catch to handle circular/non-serializable values—on
error, fall back to a safe representation (e.g., util.inspect or String()) and
include an error note so the logger never throws; apply these changes to both
logOp and logOpError.
---
Outside diff comments:
In `@docs/ops/status/phase-backlog.md`:
- Around line 115-117: Update Phase 4 item 6 in docs/ops/status/phase-backlog.md
by removing the completed scope "bundle-generate DLQ consumer" from the bullet
so the item only lists remaining bundle generation/download work (e.g., keep
"Add bundle status fields, deterministic R2 bundle keys, bundle size caps, Agent
View bundle state" and delete the "bundle-generate DLQ consumer" phrase),
ensuring the item reflects current pending work.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 57a12fff-c7e4-49ac-9531-fe6cd40e6824
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (33)
apps/jobs/.gitignoreapps/jobs/README.mdapps/jobs/package.jsonapps/jobs/src/constants.tsapps/jobs/src/cron.tsapps/jobs/src/db.tsapps/jobs/src/discovery/auto-deletion.tsapps/jobs/src/discovery/maintenance-gc.tsapps/jobs/src/discovery/retention.tsapps/jobs/src/discovery/types.tsapps/jobs/src/discovery/upload-cleanup.tsapps/jobs/src/env.tsapps/jobs/src/handlers/bundle-generate.tsapps/jobs/src/handlers/byte-purge.tsapps/jobs/src/handlers/safety-scan.tsapps/jobs/src/index.test.tsapps/jobs/src/index.tsapps/jobs/src/op-log.tsapps/jobs/src/queue.tsapps/jobs/src/r2-purge.tsapps/jobs/src/worker-configuration.d.tsapps/jobs/tsconfig.jsonapps/jobs/wrangler.jsoncdocs/ops/project-status.mddocs/ops/status/implementation.mddocs/ops/status/phase-backlog.mdpackages/commands/src/index.tspackages/commands/src/queue-target.test.tspackages/commands/src/queue-target.tspackages/contracts/package.jsonpackages/contracts/src/index.tspackages/contracts/src/jobs.test.tspackages/contracts/src/jobs.ts
Implement Cloudflare Queue consumers for byte-purge, safety-scan, and bundle-generate with asymmetric DLQ handling per ADR 0050. Wire cron discovery for upload cleanup and maintenance GC, add job message contracts, and document retry/terminal-failure behavior. Verification: pnpm --filter @agent-paste/jobs test, pnpm --filter @agent-paste/jobs typegen, pnpm verify
- Cap maintenance GC deletes across idempotency and audit phases - Expire upload sessions only after successful enqueue or no-op - Isolate per-message parse/handler errors in queue consumers - Harden op-log against throws and metadata override; report to Sentry - Drop completed DLQ consumer line from phase-backlog item 6
cursor
Bot
force-pushed
the
cursor/jobs-queue-topology-dlqs-126a
branch
from
9cc1652 to
567b868
Compare
|
agent-paste PR preview is ready. API: https://agent-paste-api-pr-98.isaac-a46.workers.dev |
There was a problem hiding this comment.
Actionable comments posted: 7
♻️ Duplicate comments (1)
apps/jobs/src/op-log.ts (1)
24-32:⚠️ Potential issue | 🟠 Major | ⚡ Quick winForward normal
logOpErrorevents to Sentry, not only stringify failures.At Line 24 and Line 52, error-level logs only go to
console.errorunless structured logging itself fails. That leaves most operator errors out of Sentry alerting.💡 Suggested patch
function emitLog(level: "info" | "error", event: string, fields: Record<string, unknown>): void { try { + const safeFields = sanitizeFields(fields); const line = JSON.stringify({ level, component: "jobs", event, at: new Date().toISOString(), - ...sanitizeFields(fields), + ...safeFields, }); if (level === "error") { console.error(line); + try { + Sentry.captureMessage(`[jobs] ${event}`, { + level: "error", + extra: safeFields, + }); + } catch { + // Never throw from operator logging. + } } else { console.log(line); }#!/bin/bash # Verify current Sentry capture paths in jobs logger. rg -n "captureException|captureMessage|logOpError|emitLog" apps/jobs/src/op-log.ts -C 3Also applies to: 52-53
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/jobs/src/op-log.ts` around lines 24 - 32, In logOpError/emitLog, error-level events are only sent to console.error and Sentry.captureException is only invoked inside the stringify catch path; update logOpError (and any emitLog caller) to forward normal error-level logs to Sentry as well (e.g., call Sentry.captureException or Sentry.captureMessage with the structured log/event payload) in addition to console.error, ensuring you pass the original error/object or a meaningful message and metadata so Sentry receives operator errors even when JSON.stringify succeeds.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/jobs/src/cron.ts`:
- Around line 38-42: The hourly branch currently calls runAutoDeletionDiscovery,
runRetentionDiscovery, and runMaintenanceGc sequentially so an exception in
runAutoDeletionDiscovery prevents the other tasks from running; wrap each call
(runAutoDeletionDiscovery(executor, now), runRetentionDiscovery(executor),
runMaintenanceGc(executor, now)) in its own try/catch, log errors (including the
exception) on failure, and allow execution to continue to the next task so one
failure won't block the rest; keep the existing return after the sequence to
preserve flow.
In `@apps/jobs/src/db.ts`:
- Around line 9-12: The current type-guard only checks for "query" and can
misclassify a binding missing transaction; update the guard in the block that
returns a SqlExecutor to validate the full executor shape (e.g., ensure binding
is an object, not null, has "query" and also has "transaction" and that
transaction is a function) before returning binding as SqlExecutor; otherwise
call createHyperdriveExecutor(binding as HyperdriveBinding).
In `@apps/jobs/src/handlers/bundle-generate.ts`:
- Around line 64-66: BundleGenerateMessage.parse failures are deterministic and
should not trigger retries: split the current try/catch so you first call
BundleGenerateMessage.parse(message.body) and if it throws log the parse error
and ack the message (do not call message.retry()), then return; only after a
successful parse call runCommand and in the subsequent error handling use
message.retry() for transient errors—ensure any references to message.retry()
remain only in the post-parse error path and malformed payloads are acknowledged
and dropped.
In `@apps/jobs/src/index.test.ts`:
- Around line 45-54: Add a new test alongside the scheduled-gating test that
covers the queue-consumer gating when JOBS_ENABLED === "false": create a
fake/spy queue message and a mocked queue handler (or the worker.queue consumer
function) using the same test setup (e.g., createExecutor or the test harness
used in this file), call the queue-consumer entrypoint/path (worker.queue
consumer function) with JOBS_ENABLED: "false", then assert the message's ack()
was called and assert the actual queue handler/work function (the consumer
callback or job processor) was not invoked
(expect(handler).not.toHaveBeenCalled()). Ensure you reference the same symbols
used in tests: worker.queue (or the queue consumer entrypoint), the message.ack
method, and the queue handler mock so the new test mirrors the scheduled-job
gating test style.
In `@apps/jobs/src/op-log.test.ts`:
- Around line 5-23: The tests create console spies with vi.spyOn(console, "...")
but call mockRestore() only after assertions, which can leak patched console
between tests; update each test that uses a spy (the "does not let caller fields
override fixed metadata" test which spies console.log around logOp(...) and the
"does not throw when JSON serialization fails" test which spies console.error
around logOpError(...)) to ensure the spy is always restored by wrapping the spy
and assertions in a try/finally that calls mockRestore(), or alternatively add a
global afterEach(vi.restoreAllMocks) to the test file so spies are reliably
cleaned up even when assertions fail.
In `@apps/jobs/src/queue.ts`:
- Around line 33-36: The code currently logs unknown queues via
logOpError("queue.unknown", { queue: batch.queue }) then calls message.ack() for
every message in batch.messages which permanently drops misrouted messages;
replace the ack behavior so unknown-queue messages are not acknowledged: locate
the batch handling where logOpError is called and remove or replace
message.ack() with a safe alternative such as message.nack() (or call the
requeue/reject API your message type exposes) for each message in batch.messages
— or simply leave them unacked/throw to let the broker retry — so messages can
be recovered if the queue was renamed or misrouted.
In `@apps/jobs/wrangler.jsonc`:
- Around line 43-66: Add final-handling consumer entries for the dead-letter
queues so DLQ messages are processed instead of expiring: add consumer objects
for "byte-purge-dlq" and "safety-scan-dlq" (and their environment variants like
"*-preview-dlq" and "*-production-dlq" if present) to the "consumers" array
similar to the existing "bundle-generate-dlq" entry — e.g., include the queue
name as "byte-purge-dlq"/"safety-scan-dlq" and set sensible fields such as
"max_batch_size" (e.g., 10) and any needed "max_retries" or handler wiring so
DLQ messages are routed to your final-failure handling/logging logic instead of
being left to expire.
---
Duplicate comments:
In `@apps/jobs/src/op-log.ts`:
- Around line 24-32: In logOpError/emitLog, error-level events are only sent to
console.error and Sentry.captureException is only invoked inside the stringify
catch path; update logOpError (and any emitLog caller) to forward normal
error-level logs to Sentry as well (e.g., call Sentry.captureException or
Sentry.captureMessage with the structured log/event payload) in addition to
console.error, ensuring you pass the original error/object or a meaningful
message and metadata so Sentry receives operator errors even when JSON.stringify
succeeds.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6eefe42d-5880-47b7-92c6-d1619ad713f3
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (34)
apps/jobs/.gitignoreapps/jobs/README.mdapps/jobs/package.jsonapps/jobs/src/constants.tsapps/jobs/src/cron.tsapps/jobs/src/db.tsapps/jobs/src/discovery/auto-deletion.tsapps/jobs/src/discovery/maintenance-gc.tsapps/jobs/src/discovery/retention.tsapps/jobs/src/discovery/types.tsapps/jobs/src/discovery/upload-cleanup.tsapps/jobs/src/env.tsapps/jobs/src/handlers/bundle-generate.tsapps/jobs/src/handlers/byte-purge.tsapps/jobs/src/handlers/safety-scan.tsapps/jobs/src/index.test.tsapps/jobs/src/index.tsapps/jobs/src/op-log.test.tsapps/jobs/src/op-log.tsapps/jobs/src/queue.tsapps/jobs/src/r2-purge.tsapps/jobs/src/worker-configuration.d.tsapps/jobs/tsconfig.jsonapps/jobs/wrangler.jsoncdocs/ops/project-status.mddocs/ops/status/implementation.mddocs/ops/status/phase-backlog.mdpackages/commands/src/index.tspackages/commands/src/queue-target.test.tspackages/commands/src/queue-target.tspackages/contracts/package.jsonpackages/contracts/src/index.tspackages/contracts/src/jobs.test.tspackages/contracts/src/jobs.ts
| "consumers": [ | ||
| { | ||
| "queue": "byte-purge", | ||
| "max_batch_size": 50, | ||
| "max_retries": 3, | ||
| "dead_letter_queue": "byte-purge-dlq" | ||
| }, | ||
| { | ||
| "queue": "safety-scan", | ||
| "max_batch_size": 1, | ||
| "max_retries": 3, | ||
| "dead_letter_queue": "safety-scan-dlq" | ||
| }, | ||
| { | ||
| "queue": "bundle-generate", | ||
| "max_batch_size": 1, | ||
| "max_retries": 5, | ||
| "dead_letter_queue": "bundle-generate-dlq" | ||
| }, | ||
| { | ||
| "queue": "bundle-generate-dlq", | ||
| "max_batch_size": 10 | ||
| } | ||
| ] |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
For Cloudflare Queues, what is the recommended terminal-failure pattern when using dead_letter_queue: should each DLQ have an explicit consumer for logging/finalization, and what happens operationally if a DLQ has no consumer?
💡 Result:
Recommended terminal-failure pattern for Cloudflare Queues DLQs: 1) For each DLQ (e.g., my-queue-dlq), configure its own consumer that does “final handling” such as logging/alerting and any cleanup/redrive logic. Cloudflare’s DLQ “queue” is just another queue, and you must configure a consumer for it as you would for any other queue; it can be processed independently from the primary queue.[1] 2) Use the primary consumer’s dead_letter_queue setting so messages that fail after max_retries are routed to the DLQ instead of being permanently discarded.[1] What happens operationally if a DLQ has no consumer: Messages delivered to a DLQ without an active consumer will persist for four (4) days and then be deleted from the queue.[1] Notes: - If you don’t configure a DLQ for the primary consumer at all, messages that repeatedly fail processing are eventually discarded permanently.[1][2]
Citations:
- 1: https://developers.cloudflare.com/queues/configuration/dead-letter-queues/index.md
- 2: https://developers.cloudflare.com/queues/configuration/configure-queues/index.md
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="apps/jobs/wrangler.jsonc"
echo "== File sanity (first/last lines) =="
wc -l "$FILE"
sed -n '1,120p' "$FILE" | nl -ba | sed -n '1,120p'
echo "..."
sed -n '120,220p' "$FILE" | nl -ba | sed -n '1,120p'
echo "..."
sed -n '220,340p' "$FILE" | nl -ba | sed -n '1,140p'
echo
echo "== Ripgrep queue name occurrences =="
rg -n "\"(byte-purge|safety-scan|bundle-generate)(-dlq)?\"" "$FILE" || true
echo
rg -n "byte-purge-dlq|safety-scan-dlq" "$FILE" || true
echo
echo "== Context around DLQ names (if present) =="
for q in "byte-purge-dlq" "safety-scan-dlq" "bundle-generate-dlq"; do
echo "--- $q ---"
rg -n --context 5 "\"queue\"[[:space:]]*:[[:space:]]*\"$q\"" "$FILE" || true
rg -n --context 5 "\"dead_letter_queue\"[[:space:]]*:[[:space:]]*\"$q\"" "$FILE" || true
doneRepository: zaks-io/agent-paste
Length of output: 190
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="apps/jobs/wrangler.jsonc"
echo "== Queues/DLQs occurrences =="
rg -n "byte-purge-dlq|safety-scan-dlq|bundle-generate-dlq|byte-purge\"|safety-scan\"|bundle-generate\"" "$FILE" || true
echo
echo "== Consumer blocks context (cat with line numbers) =="
# print the whole file in numbered form in a controlled way (184 lines only)
cat -n "$FILE" | sed -n '1,220p'
echo
echo "== Context around DLQ names (if present) =="
for q in "byte-purge-dlq" "safety-scan-dlq" "bundle-generate-dlq"; do
echo "--- $q ---"
rg -n --context 8 "\"queue\"[[:space:]]*:[[:space:]]*\"$q\"" "$FILE" || true
rg -n --context 8 "\"dead_letter_queue\"[[:space:]]*:[[:space:]]*\"$q\"" "$FILE" || true
doneRepository: zaks-io/agent-paste
Length of output: 9310
Add DLQ “final handling” consumers for byte-purge and safety-scan.
apps/jobs/wrangler.jsonc sets dead_letter_queue for byte-purge/safety-scan (and corresponding *-preview/*-production DLQs), but only bundle-generate-dlq has a consumer block. With no consumer for byte-purge-dlq / safety-scan-dlq, DLQ messages will sit briefly (~4 days) and then be deleted, leaving purge/scan terminal-failure handling/logging missing. (Applies to lines 43-66, 100-123, 157-180.)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/jobs/wrangler.jsonc` around lines 43 - 66, Add final-handling consumer
entries for the dead-letter queues so DLQ messages are processed instead of
expiring: add consumer objects for "byte-purge-dlq" and "safety-scan-dlq" (and
their environment variants like "*-preview-dlq" and "*-production-dlq" if
present) to the "consumers" array similar to the existing "bundle-generate-dlq"
entry — e.g., include the queue name as "byte-purge-dlq"/"safety-scan-dlq" and
set sensible fields such as "max_batch_size" (e.g., 10) and any needed
"max_retries" or handler wiring so DLQ messages are routed to your final-failure
handling/logging logic instead of being left to expire.
Add focused tests for maintenance GC caps, upload cleanup no-op and cap_hit, cron/queue edges, per-message handler retries, R2 pagination, and op-log Sentry fallback.
|
agent-paste PR preview is ready. API: https://agent-paste-api-pr-98.isaac-a46.workers.dev |
- Isolate hourly cron sweeps with per-task try/catch - Require full SqlExecutor shape before using local DB binding - Ack invalid bundle DLQ payloads; retry unknown queues - Export runQueueConsumer for disabled-queue gating tests - Harden op-log with Sentry error forwarding and spy cleanup
|
agent-paste PR preview is ready. API: https://agent-paste-api-pr-98.isaac-a46.workers.dev |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/jobs/src/op-log-sentry.test.ts`:
- Around line 15-17: Replace the teardown that currently calls
vi.clearAllMocks() inside the afterEach hook with vi.restoreAllMocks() (or call
vi.restoreAllMocks() in addition to vi.clearAllMocks()) so mocked
implementations like console spies are fully restored between tests; update the
afterEach block that references vi.clearAllMocks() to use vi.restoreAllMocks()
to prevent leaked console spies in subsequent tests.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 369be999-81d1-4037-834d-f268957f8683
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (12)
apps/jobs/package.jsonapps/jobs/src/cron.tsapps/jobs/src/db.test.tsapps/jobs/src/db.tsapps/jobs/src/handlers/bundle-generate.tsapps/jobs/src/index.test.tsapps/jobs/src/index.tsapps/jobs/src/jobs-coverage.test.tsapps/jobs/src/op-log-sentry.test.tsapps/jobs/src/op-log.test.tsapps/jobs/src/op-log.tsapps/jobs/src/queue.ts
|
agent-paste PR preview is ready. API: https://agent-paste-api-pr-98.isaac-a46.workers.dev |
isuttell
dismissed stale reviews from coderabbitai[bot], coderabbitai[bot], and coderabbitai[bot]
Dismissed as stale: latest head 3f5d9b2 has green Validate, green PR Preview, CodeRabbit status success, and the referenced findings have been fixed or were out of AP-21 scope.
|
agent-paste PR preview resources were cleaned up. The pr-preview-${context.issue.number} environment is left in place; remove it from the GitHub UI if desired. |
2 participants
Summary
Implements Phase 4 jobs worker queue topology per ADR 0032, 0049, and 0050 (rebased on
main@ 420ce90, including AP-76).Review follow-up: maintenance GC sweep cap enforcement, upload session expiry only after successful enqueue/no-op, per-message queue handler isolation, hardened op-log with Sentry capture, phase-backlog item 6 cleanup.
Out of scope
api(AP-22)Verification
pnpm --filter @agent-paste/jobs testpnpm --filter @agent-paste/jobs typecheckpnpm verifyLinear: AP-21
Linear Issue: AP-21
Summary by CodeRabbit
Release Notes
New Features
Infrastructure
Documentation