Skip to content

Timing-result mismatches print FAIL but may still exit successfully #30

Description

@memotype

Severity: Low
Expected real-world likelihood: Low

Problem

The optional -t block compares two known citylots.json results. On mismatch it prints a line beginning with FAIL, but it does not call the harness's fail helper or otherwise preserve a failing status. Execution continues to later timing commands, so the script's final status can still be 0.

The timing mode is optional and depends on an external fixture, which keeps likelihood low. If used as a correctness/performance check, however, visible failure text can be missed by automation.

Relevant behavior

The two checks have this shape:

if [[ $r = 37.805335380794915 ]]; then
  echo pass
else
  echo "FAIL: ..."
fi

The block then continues to later bj and jq timing commands.

Possible fixes (suggestions only)

These are possible approaches, not prescribed implementations:

  • Call the existing fail helper when a known-result assertion is wrong.
  • Accumulate a local timing-check status and return it at the end.
  • Explicitly document -t as informational-only and avoid presenting mismatches as harness failures.

No global pipefail is needed or suggested.

Suggested coverage

  • Exercise the result-check helper with matching and deliberately mismatching expected values without requiring the large fixture.
  • Assert the harness's final exit status in both cases.
  • Keep actual timing tests optional and fixture-dependent.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinglowLow prioritytest-harnessTest runner and assertion behavior

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions