Is your feature request related to a problem? Please describe.
When a job with a given dedupe_key is already queued, there isn't a clean way to update its priority.
A concrete example from our use case: we enqueue a job at a low priority, and later some trigger decides the same work is now urgent and tries to enqueue the same dedupe_key at a higher priority to move it toward the top of the queue. Today that raises DuplicateJobError (the partial unique index on dedupe_key rejects the second insert), and the already-queued job keeps its original low priority. As far as we can tell there isn't a supported way to promote it.
This is also pretty closely related to #678 (configurable duplicate handling on enqueue). Probably the same interface, just another option for how it handles duplicates.
Describe the solution you'd like
It would be great to have a configurable option in enqueue that updates the priority of the already-queued job instead of raising, taking the max of the existing priority and the new priority from the enqueue attempt:
- If no queued/picked job with that
dedupe_key exists → insert normally.
- If one exists →
UPDATE its priority to GREATEST(existing_priority, new_priority) and leave it in the queue, so priority is only ever raised, never lowered.
Under the hood this could map to INSERT ... ON CONFLICT (dedupe_key) WHERE (...) DO UPDATE SET priority = GREATEST(<queue_table>.priority, EXCLUDED.priority).
# promote an already-queued job; keep the higher of the two priorities
await queries.enqueue(
"process_report",
payload,
priority=100,
dedupe_key="report:42",
on_conflict="update_priority",
)
If we implement #678 as well, the full on_conflict surface would then be raise | skip | update_priority.
Describe alternatives you've considered
- Dequeuing/cancelling the existing job and re-enqueuing at the higher priority — this has a race condition (the job may already be picked), loses ordering/history, and needs extra round-trips.
- Manually
UPDATE-ing the queue table's priority column out of band — this works, but it sidesteps the public API and is fragile against schema changes. I'd prefer not to have to do this.
- Not using dedupe keys and inserting a new task with higher priority — possible, but then we lose ability to avoid extra work, causes a lot of churn in idempotent tasks where it gets executed twice.
- More of an alternative implementation, but always taking the new priority rather than the max — this could accidentally lower priority;
GREATEST seems like it would be safer and solves the use case of bumping up a tasks priority.
Additional context
Same stuff from other issue probably also applies
Is your feature request related to a problem? Please describe.
When a job with a given
dedupe_keyis already queued, there isn't a clean way to update its priority.A concrete example from our use case: we enqueue a job at a low priority, and later some trigger decides the same work is now urgent and tries to enqueue the same
dedupe_keyat a higher priority to move it toward the top of the queue. Today that raisesDuplicateJobError(the partial unique index ondedupe_keyrejects the second insert), and the already-queued job keeps its original low priority. As far as we can tell there isn't a supported way to promote it.This is also pretty closely related to #678 (configurable duplicate handling on
enqueue). Probably the same interface, just another option for how it handles duplicates.Describe the solution you'd like
It would be great to have a configurable option in
enqueuethat updates the priority of the already-queued job instead of raising, taking the max of the existing priority and the new priority from the enqueue attempt:dedupe_keyexists → insert normally.UPDATEits priority toGREATEST(existing_priority, new_priority)and leave it in the queue, so priority is only ever raised, never lowered.Under the hood this could map to
INSERT ... ON CONFLICT (dedupe_key) WHERE (...) DO UPDATE SET priority = GREATEST(<queue_table>.priority, EXCLUDED.priority).If we implement #678 as well, the full
on_conflictsurface would then beraise|skip|update_priority.Describe alternatives you've considered
UPDATE-ing the queue table'sprioritycolumn out of band — this works, but it sidesteps the public API and is fragile against schema changes. I'd prefer not to have to do this.GREATESTseems like it would be safer and solves the use case of bumping up a tasks priority.Additional context
Same stuff from other issue probably also applies