[Daemon] Add `scope: :cluster` setting

[rsamoilov]

Description

Rage::Daemon currently supports two scope levels:

• :node (default) — one daemon instance per server. Rage uses IPC to coordinate which worker runs it; if that worker dies, the daemon restarts in another worker on the same node.
• :worker — one daemon instance per worker process.

A missing piece is :cluster scope — ensuring exactly one daemon instance runs across all deployed servers. This is essential for tasks that must be globally unique, such as a single consumer for an ordered message queue, a global scheduler that must not produce duplicate work, or a leader-election process.

class GlobalScheduler < Rage::Daemon
  scope :cluster

  def perform
    loop do
      schedule_pending_jobs
      sleep 60
    end
  end
end

With :cluster scope, only one instance of GlobalScheduler would run across the entire deployment. If the server running it goes down, another server would take over.

Implementation

The cluster scope should build on the existing config/pubsub.yml mechanism backed by the Redis adapter. Rage already uses this infrastructure for cross-server communication (e.g. WebSocket broadcasting via Rage::Cable), so the Redis dependency is not new for applications that need cluster-level coordination.

The coordination mechanism would work roughly as follows:

1. On boot, each node attempts to acquire a distributed lock in Redis for each :cluster-scoped daemon.
2. The node that acquires the lock starts the daemon; the others monitor.
3. The lock holder periodically renews the lock (heartbeat). If the lock expires (because the node crashed or became unresponsive), another node acquires it and starts the daemon.
4. On graceful shutdown, the node releases the lock, allowing another node to take over immediately.

Design considerations

• Lock granularity. Should there be one lock per daemon class, or a single lock for all cluster-scoped daemons? Per-daemon locks are more flexible (different daemons can run on different nodes) but add more Redis keys and heartbeat traffic.
• Heartbeat interval and expiry. The heartbeat must be frequent enough to detect failures quickly but not so frequent that it creates significant Redis traffic.
• Failover latency. There will be a gap between a node dying and another node acquiring the lock. The failover time equals the lock TTL minus the time of the last heartbeat. Document this behavior so users understand the tradeoff.
• Redis unavailability. If Redis becomes unreachable, the lock holder can't renew and competitors can't acquire. Define the behavior: should the current holder continue running (risking split-brain if Redis partitions), or should it stop (risking downtime)?
• No Redis configured. If config/pubsub.yml is not set up, using scope :cluster should raise a clear error at boot time rather than failing silently at runtime.

Tips

• Review the Daemon API docs to understand the existing scope mechanism and daemon lifecycle.
• Examine how config/pubsub.yml and the Redis adapter are used for Rage::Cable broadcasting to understand the existing Redis integration.
• Check the architecture doc to see how Rage's core components interact and to understand the design principles.
• Read the contributing guide for coding conventions and design principles used across the codebase.
• Before starting the implementation, please share your proposed design approach. This feature involves distributed coordination with several design tradeoffs, so discussing the approach early will drastically increase the chances of acceptance and help avoid rework.
• Feel free to ask any questions or request help in the comments below!