fix(jobs): nrdFeedJob has no retry and no WhoisXML 429/503 backoff — transient error loses entire day's NRD data

[walidboulanouar]

Problem

packages/jobs/src/nrd-feed.ts WhoisXML fetch:

const res = await fetch(url, { signal: AbortSignal.timeout(120_000) });
if (!res.ok) {
  throw new Error(`WhoisXML NRD feed failed (${res.status}): ${text}`);
}

The job config has retry: { maxAttempts: 1 } (explicitly disabled). A transient 429 or 503 from WhoisXML:

1. Throws immediately with no backoff
2. Is not retried (maxAttempts: 1)
3. Silently kills the entire day's NRD piracy domain discovery
4. No Knock/alert fires to notify the team

The next opportunity is 24 hours later. Any piracy domains registered that day go undetected until the following NRD snapshot.

Fix

Wrap the WhoisXML fetch in a retry loop with exponential backoff, and add a Knock alert on permanent failure:

// Or use Trigger.dev's built-in retry: { maxAttempts: 3, factor: 2, minTimeoutInMs: 30_000 }
// and remove the explicit maxAttempts: 1

For the 429 case specifically: read the Retry-After header and sleep before retry.

Also: send a Knock system-alert notification to admins when the NRD feed job permanently fails.

Severity

MEDIUM — 24-hour gap in piracy domain discovery on NRD provider outages.

[walidboulanouar]

Verified fixed — nrdFeedJob has retry maxAttempts:3 with backoff plus an ops alert on failure (nrd-feed.ts:107).