fix(bullmq): apply configurable job retention to prevent unbounded Redis growth

[apdelsm]

Problem

The BullMQ backend never sets removeOnComplete/removeOnFail, and BullMQ keeps completed and failed jobs in Redis indefinitely by default. For recurring jobs this is a slow memory leak: every run leaves a job record behind forever.

We hit this in a real deployment — a Redis instance sitting at ~94 MB "with no traffic", of which 31,315 keys were bull:default:repeat:hermes:<job>:<ts> hashes (completed recurring-job runs, ttl=-1) from a job running every: '30s'. The bull:default:completed set and the events stream were also growing unbounded.

Fix

• Add an optional defaultJobOptions?: DefaultJobOptions to BullMQBackendOptions so callers can control retention (and attempts/backoff).
• Apply bounded retention by default (removeOnComplete: { count: 1000 }, removeOnFail: { count: 5000 }) so Redis no longer grows without limit out of the box; anything the caller passes is merged on top and takes precedence.
• Wire the options into both the queue (defaultJobOptions) and the job scheduler's template (opts). This second part matters: jobs produced by a scheduler do not inherit the queue's defaultJobOptions, so without it recurring jobs would still pile up.

Testing

• deno task check, deno lint, deno fmt clean.
• deno task test — all BullMQ integration tests pass against a local Redis (7 passed), including a new regression test that asserts no completed jobs accumulate when retention is set.

Defaults are conservative and easy to tune; happy to adjust the retention values or make them opt-in instead of on-by-default if you prefer.

🤖 Generated with Claude Code