Allow staggering of restore_delay for WOL

[Chewyyy]

Allow a delay per client, rather than a stagnant delay

Reasons: If Proxmox boots at the same time as a NAS and the drives don't connect, then LXCs that rely on the NAS will error out and not boot up.

Essentially, what I want to be able to do is say 30sec delay on NAS after power restores, and 180sec on ProxMox

[bpleonardo]

Ideally we could deal with dependencies. For example, have the NAS be a dependency of Proxmox, and only try to wake a machine if all of its dependencies are online.

Or we could just have a priority system, specifying which clients should wake up first.

[bpleonardo]

Also, while I have no experience running selfhosted servers, I'm wondering: shouldn't your container wait for its dependencies to come up? Or at least have an exponential backoff for retrying. Sorry if that's already the case, it's just odd that the solution to this problem is having delays...

[Chewyyy]

Proxmox doesn’t have a built in dependency before launching. I think they have built in time delays but I’d only want the time delay to occur on a power outage restore not on every system start up

[Pwhitcomb]

Could this maybe be implemented by making a dependency check?

Example clients:
NAS
Proxmox1
Proxmox2
Desktop

Example Order:
Desktop - No dependencies - Can start whenever
NAS - No dependencies - Can start whenever
Proxmox1 - Depends on NAS - Has to start after NAS
Proxmox2 - Depends on NAS and Proxmox1 - Has to start after NAS and Proxmox1

Possibly solved by having pings be sent to dependencies and when they are successful after a time, then start the next host?

Thoughts?

[bpleonardo]

That's what I thought too. But would probably require different health checks strategies (similar to Docker Compose's). A machine can be online but not ready to serve requests, in this case it would still break.

And we should decide what to do when a dependency fails. Should we just not wake up the rest? Maybe that's correct.

[qm3ster]

I get the idea, and having another condition in addition to NUT to wait for before some actions does make sense, but application dependencies should probably be waited on at the last possible moment, so you probably just want something else to do pct|qm start rather than having automatic start checked?

[djgamerr]

+1 for this
Having dependencies is quite useful, eg:

1. start pfsense
2. after pfsense is online - start proxmox
   (2 different interfaces, wolnut wouldnt be able to reach proxmox without pfsense running)

If dependencies are unmet - entry should just be skipped (no deps - no wol)
I reckon that dependencies could be an array of strings, and the use-case would be as simple as this
"Send WOL only if all deps in the array are online"

[qm3ster]

@djgamerr You'll have less startup delay if you do something like

# /etc/systemd/system/wait-for-internet.service
[Unit]
Description=Wait for internet reachability
After=network-online.target
Wants=network-online.target
Before=pve-guests.service # or pve-cluster, pveproxy, pvedaemon, whatever is the earliest thing you can't start offline.
StartLimitIntervalSec=0

[Service]
Type=oneshot
ExecStart=/usr/bin/ping -c1 -W2 1.1.1.1
Restart=on-failure
RestartSec=3
RemainAfterExit=yes

directly on the proxmox host.