Skip to content

tune --backend model leaves an empty run directory that report cannot read #22

Description

@vyncint

Reproduce

$ launchbound tune reduce-flip --backend model --cc 8.6
reduce-flip — ESTIMATED tuning (...)
  estimated c1-9b9fe33407467e5e  block_x=32 tile=128  cost 76.800
  ...

$ find runs/reduce-flip-model -type f | wc -l
0

An empty directory, every time, in the user's repository. --out behaves the same way:

$ launchbound tune reduce-flip --backend model --out /tmp/run
$ ls -A /tmp/run     # nothing
$ launchbound report /tmp/run
error: run dir: /tmp/run/verdicts.json: No such file or directory (os error 2) (stage writes it)

Where

crates/launchbound-cli/src/main.rs:372-373, before the backend match:

let out = out.unwrap_or_else(|| PathBuf::from("runs").join(format!("{}-{backend}", spec.name)));
std::fs::create_dir_all(&out)?;

match backend {
    "metal" => { ... .checkpoint(&out.join("results.json")) ... }
    "cuda"  => { ... }
    "model" => { /* prints to stdout; never touches `out` */ }
}

The directory is created for every backend; only the measured ones write into it.

Why it is worth fixing

  • --out is offered on a command where it does nothing, and the help (Run directory (default runs/<kernel>-<backend>)) gives no hint that the backend decides whether it is used.
  • report is documented as "Render the report for a staged (and possibly measured) run directory", and pointing it at what tune just produced fails. Following the two commands in the order --help lists them does not work.
  • Every model run leaves an untracked empty directory behind. runs/ is in the repository (runs/reduce-stable-metal is committed), so this is litter where someone will see it.

Fix — either

  1. Create it where it is used — move create_dir_all into the backends that write, and have the model branch reject or ignore --out explicitly. Cheapest, and makes the flag honest.
  2. Have the model backend write a run directory tooverdicts.json it already has, plus estimates instead of measurements, clearly labelled. That makes tune → report work uniformly across backends, which is the shape the help already implies, and gives model runs something to diff.

(2) is more work and more useful; (1) is correct on its own and can land first.

Done when

tune --backend model either produces a run directory report can read, or does not create one and says so if --out was given.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions