test(maestro-case): harden checkers and a few tests

[charlesliu9]

test(maestro-case): harden checkers and close the operate-mode coverage gap

Checker robustness:

• case_check.py routes every CLI call through a _run helper that turns a
  TimeoutExpired into a FAIL with the partial output, instead of a raw
  traceback the eval harness reports as a checker crash. TimeoutExpired
  carries bytes even under text=True, so the handler decodes defensively.
• The athena_cm_event plan checker now accepts task entry fields written
  without a leading "- " bullet; agents render the contract both ways and
  the line-anchored regex was rejecting the plain form.

Both are covered by new unit tests.

Task fixes:

• single_node/process gains the missing mode:build tag and a prompt hint
  that ProcurementProcess resolves from processOrchestration-index.json,
  the recurring cause of a placeholder taskTypeId.

[github-actions]

Claude finished @charlesliu9's task in 2m 16s —— View job

---

Coder-eval task lint (advisory)

• Read lint rubric
• Identify changed task YAMLs
• Check PR body for evidence of passing run
• Apply rubric to each changed file
• Within-PR duplicate check
• Post final comment

---

Coder-eval task lint (advisory)

1 task YAML changed; 0 Critical, 0 High, 0 Medium, 0 Low, 1 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

❌ High — PR body does not claim the modified task has been run and still passes. Please edit the PR description to add a line like: Ran skill-case-single-process locally and it passed.

Per-task lint

tests/tasks/uipath-maestro-case/single_node/process/process.yaml — verdict: OK

Changes add the missing mode:build tag (line 9) and a prompt hint directing registry resolution to processOrchestration-index.json (lines 99-102). Both changes are clean:

• Self-report anti-pattern: No self-report files. Criteria use run_command with real CLI validation and a Python checker.
• Prompt over-specification: The new hint names the registry file and says not to write a placeholder taskTypeId. This is a ground-truth anchor for a feature:registry task — it tells the agent which resource exists, not the procedure to follow. The skill still teaches how to resolve. Comparable to sibling case_management.yaml which similarly names its registry file.
• Meaningful coverage: Two run_command criteria — uip maestro case validate (weight 3.0) and a Python checker that validates caseplan structure + runs debug (weight 5.0). Strong.
• Could pass for the wrong reason: Both criteria require real CLI execution; can't be faked.
• Near-duplicate: Siblings (agent, api_workflow, case_management, rpa, run_once_envelope) share the single-node scaffold but each exercises a materially distinct task-type plugin. Scaffold reuse, not duplication.
• Validate-only flow tests: N/A — uipath-maestro-case, not a flow skill.
• CLI verb reachability: No command_executed criteria to check.
• Redundant uip CLI / Run-limit fields: No sandbox block; run_limits correctly at top level.

Within-PR duplicates

• No duplicate clusters detected (only 1 task YAML changed).

Conclusion

• ⚠ 1 task passes the rubric (OK), but the PR body is missing a passing-run claim (High, advisory). Consider editing the description to confirm the modified task still passes.

---

[github-actions]

Claude finished @charlesliu9's task in 2m 36s —— View job

---

PR Review: test(maestro-case): harden checkers and a few tests

• Read review guidelines and project conventions
• Analyze full diff
• Read changed files in full for context
• Check for correctness, security, and best practices
• Post review findings

---

Summary

Hardens the case_check.py checker infrastructure by routing all subprocess.run calls through a _run helper that catches TimeoutExpired and reports it as a clean FAIL (with partial output) instead of a raw traceback. Loosens the athena plan checker regex to accept task-entry fields without a leading - bullet. Adds the missing mode:build tag and a prompt hint to single_node/process/process.yaml. Both checker changes are covered by new unit tests.

Change-by-Change Review

1. tests/tasks/uipath-maestro-case/_shared/case_check.py — OK

• All three subprocess.run(…, timeout=…) call sites (validate, resources-refresh, case-debug) now route through _run. No direct subprocess.run calls with timeout remain outside the helper (verified via grep).
• _run catches TimeoutExpired and decodes partial stdout/stderr defensively — TimeoutExpired carries bytes even under text=True on POSIX (CPython behavior), so isinstance(s, bytes) + decode("utf-8", "replace") is the right approach; the (s or "") fallback covers None stderr.
• _fail return type upgraded from implicit None to NoReturn — correct, since sys.exit() always raises SystemExit. This also satisfies mypy: _run's except-branch calls _fail which never returns, so the function's declared -> CompletedProcess[str] is sound.
• check=False is explicitly passed — good defensive coding (prevents CalledProcessError from shadowing the TimeoutExpired path).

2. tests/tasks/uipath-maestro-case/_shared/test_case_check.py — OK

• New test_timeout_reports_fail_not_traceback monkeypatches subprocess.run to raise TimeoutExpired with output=b"half a payload\xff". The \xff byte exercises the "replace" decode path (not valid UTF-8). stderr is implicitly None, confirming the (s or "") fallback.
• Asserts the exit message contains both the timeout description and the partial output — exercises the full message formatting chain.

3. tests/tasks/uipath-maestro-case/athena_cm_event/check_athena_cm_event_plan.py — OK

• Regex ^-\s* (required bullet) → ^(?:-\s*)? (optional bullet). Agents render task-entry fields both as - entry-rule: … and entry-rule: …; the old regex rejected the plain form. The ^ anchor and $ anchor still enforce line-level matching, so there's no risk of over-matching mid-line content.

4. tests/tasks/uipath-maestro-case/athena_cm_event/test_checkers.py — OK

• write_tasks_md gains a bullet_prefix: bool = True parameter — default preserves existing test behavior, False generates the plain (no-bullet) form.
• New test_plan_checker_accepts_plain_task_entry_fields exercises the regex fix end-to-end by running the actual checker script against a fixture without bullets.

5. tests/tasks/uipath-maestro-case/single_node/process/process.yaml — OK

• Adds missing mode:build tag — consistent with every other generate-lifecycle task in the suite.
• Prompt hint about resolving ProcurementProcess from processOrchestration-index.json is the recurring cause of placeholder taskTypeId failures; the hint is scoped and actionable.

What's Missing

Nothing identified.

Area Ratings

Area	Status	Notes
Frontmatter	N/A	No skill SKILL.md changed
E2E Tests	OK	Both checker changes covered by unit tests
Skill Body	N/A	No skill body changed
References & Assets	N/A	No references or assets changed
Repo Hygiene	OK	No secrets, no cross-skill deps, changes scoped to maestro-case test infra

Issues for Manual Review

None found.

Conclusion

Clean PR. The timeout handler is well-structured — catches the right exception, decodes bytes defensively, and surfaces partial output for debugging. The regex loosening is minimal and precisely targeted. The new tests exercise the exact edge cases motivating the changes. No issues to flag — approve.
|