Skip to content

Fail the structure gate when a stage cannot enumerate its inputs #54

Description

@ckrough

scripts/gates/structure.sh exits 0 under a write-restricted sandbox while silently skipping Stage C (the tracked Python suites) and Stage D (the gate self-tests). The exit code reports success, so a caller cannot distinguish a real pass from a run where most of the gate never executed.

The mechanism is the here-strings that feed the per-file loops. Stage C iterates with done <<< "$py_tracked" (line 151) and Stage D with done <<< "$st_tracked" (line 183); Stage A uses the same construct at line 89. Bash implements a here-string by writing a temporary file. When the sandbox denies that write, the redirect fails, the loop body never runs, and the stage completes with no failures to report — a pass by vacancy. The run prints cannot create temp file for here document to stderr, but nothing inspects it and the exit status stays 0.

Setting TMPDIR to a writable path does not fix it: the denial tracks the process's working directory, not TMPDIR.

This matters because Daedalus runs this script as the repository's bound test gate (.daedalus/config.json). Any sandboxed pipeline run gates on Stages A, B, and E only, and reports green.

Observed 2026-08-09 while working issue #53. Run with dangerouslyDisableSandbox, the same script exits 0 having actually run 52 tests in scriptorium/skills/docs/scripts/test_check_markdown.py and 58 in scripts/evals/test_run_trigger_evals.py.

Steps to Reproduce

  1. Run bash scripts/gates/structure.sh from a Claude Code Bash call with the default sandbox active, where the repository path is outside the sandbox write allowlist.
  2. Observe cannot create temp file for here document on stderr, four times.
  3. Observe that no PASS python suite: lines appear for Stage C and no self-test output for Stage D.
  4. Observe echo $? is 0.
  5. Re-run the same command with the sandbox disabled and observe both stages executing and reporting suite results.

Acceptance Criteria

  • A stage that cannot enumerate its inputs fails the gate rather than passing, so a here-string or temp-file failure produces a non-zero exit.
  • Stage C and Stage D each assert that they iterated at least once, matching the existing pattern where the tracked-shell-test stage fails when its set is empty.
  • The gate distinguishes "no matching files exist" from "the file list could not be read", and the failure message names which.
  • Reproducing the sandboxed conditions above yields a non-zero exit.
  • A green run still reports the per-suite PASS lines it reports today.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions