investigate: disk preclear/burn-in feature

[danneu]

Overview

Research competitor features (TrueNAS, Unraid) to inform a disk preclear/burn-in system for braid. Pre-clear is the single most-cited feature from Unraid users, and TrueNAS explicitly punts this to community scripts despite a feature request. Opportunity to make this a first-class, well-integrated part of braid.

Why preclear matters

• Catches DOA and infant-mortality drives within the return window
• Catches cabling/backplane/PSU failures before blaming the disk
• Establishes a SMART baseline for future delta detection
• Warms up the disk thermally to validate case airflow under load

Current state: TrueNAS

• No built-in feature — users follow forum stickies and run Spearfoot's disk-burnin.sh
• Canonical sequence: smartctl short → conveyance → long (baseline) → badblocks 4-pass → smartctl long (post) → manual SMART diffing
• Wall clock: ~7–10 days per 12TB CMR drive (badblocks dominates)
• UX problems: no progress reporting, no SMART diffing built in, no integration with replace workflow, new users don't know it's expected

References

• Hard Drive Burn-In Testing (TrueNAS community sticky)
• Spearfoot disk-burnin-and-testing (GitHub)
• Feature request: Include disk burn-in in TrueNAS GUI

Current state: Unraid

• Unassigned Devices Preclear plugin (successor to gfjardim's original)
• UX: disks appear in the Unassigned Devices panel; each row has a Preclear button
• Runs as a background daemon (survives SSH disconnects)
• Live progress reporting: current step/pass/%, MB/s, ETA, elapsed time in the same UI row
• SMART pre/post reports saved and viewable
• Notifications fire on completion/failure
• Key win: preclear is a normal disk-prep step in the same panel where you add the disk, not a separate shell exercise

Proposed design for braid

Command surface

braid preclear <disk>                # start with defaults
braid preclear status                # all in-progress + history
braid preclear status <disk>         # detail for one disk
braid preclear cancel <disk>         # stop early
braid preclear <disk> --quick        # ~hours (1-pass write/verify)
braid preclear <disk> --standard     # ~days (default: 2-pass)
braid preclear <disk> --thorough     # ~weeks (4-pass + 2 cycles)
braid preclear <disk> --cycles N     # explicit override

Default: --standard. Show ETA before starting so users aren't surprised by multi-day runtimes.

Recipe (leverage Spearfoot's proven sequence)

• smartctl -t short → -t conveyance (if supported) → -t long (baseline)
• badblocks 4-pass with auto-sized block-count (8192 for >8TB, 4096 for ≤8TB)
• smartctl -t long (post)
• Automatic SMART before/after diffing (only show attributes that moved)
• Verdict: pass unless any of these moved: Reallocated_Sector_Ct, Current_Pending_Sector, Offline_Uncorrectable, UDMA_CRC_Error_Count, or SMART self-test reported failure

Future: optional --engine=fio as an alternative to badblocks for faster/cleaner progress reporting.

Process lifetime: systemd-run (not a braid daemon)

Spawn each preclear as a transient unit:

systemd-run --unit=braid-preclear-<serial> --remain-after-exit \
  /run/wrappers/bin/braid preclear-runner <disk> <opts>

Benefits:

• Survives SSH disconnects
• Logs go to journald (journalctl -u braid-preclear-<serial>)
• braid queries systemctl is-active for status without owning a daemon
• ExecStopPost hook fires alerts on completion/failure
• Aligns with braid's existing systemd-lifecycle design (per docs/decisions/018-systemd-lifecycle.md)

Progress tracking

• Per-disk state file: /var/lib/braid/preclear/<serial>.json (phase, pass, bytes, MB/s, ETA, SMART snapshot)
• CLI: braid preclear status shows table (active disk rows: phase/pass/% /MB/s/ETA/elapsed)
• TUI: live-updating panel via existing progress.rs infra
• History: one record per run in state_paths.rs (started, finished, engine, SMART before/after, deltas, verdict)

Integration with braid add and braid replace

Require a successful preclear on record by serial:

$ braid add /dev/sdc
error: /dev/sdc has no successful preclear on record (serial 8KGAB12N).
       Run `braid preclear /dev/sdc` first, or pass --skip-preclear to override.

Override exists for replacement-of-replacement and user's who pre-cleared elsewhere.

Parallelism

Spawn multiple transient units, one per disk:

for d in /dev/sd{b,c,d,e}; do braid preclear $d; done
braid preclear status

Live 4-row table. No tmux required.

Out of scope (don't add these)

• Resume across reboots — badblocks doesn't resume cleanly; state marks run "interrupted, no verdict"; user re-runs
• Non-destructive mode on existing pool members — conflicts with the write/verify that's the whole point
• Historical SMART graphs — belongs in broader SMART monitoring story, not here

Advantages over TrueNAS

• Built-in command vs forum stickies + shell scripts
• systemd-run vs manual tmux for persistence across disconnects
• Structured progress reporting vs tail -f logs
• Automatic SMART diffing vs manual inspection
• Alert/notification integration via existing system
• Blocks unburned disks from being added (with override)
• Parallel multi-disk via one command per disk vs manual tmux

Next steps

1. Review docs/decisions/018-systemd-lifecycle.md to confirm transient-unit approach aligns with braid patterns
2. Draft plan file under plans/wip/
3. Validate badblocks behavior at 8–20TB scale (block-count overflow issues)
4. Prototype state JSON schema and verdict logic
5. Estimate scope: CLI command surface + progress tracking + state management + systemd wiring + braid add / braid replace gating