Extend uipath-tasks skill: catalogs, comments, labels, metadata, data

[de-si]

Extends the uipath-tasks skill and evals for the new uip tasks commands: catalogs, comments, labels, metadata, and data. Adds 3 references, 3 smoke evals, and 1 e2e eval.

Verification: every command these evals exercise was run live against the test tenant (procodeapps/integrationtest) and returned Success — catalogs create/get/update/list, data get/save, comments list/add, labels set/clear, metadata note/catalog-id/unset. Smoke evals run through coder-eval in CI. A full coder-eval pass of the e2e eval is gated on the CLI release, since the sandbox installs @uipath/cli@latest, which does not yet include these commands.

Jira: https://uipath.atlassian.net/browse/ACTN-11600

🤖 Generated with Claude Code

[github-actions]

Claude finished @de-si's task in 5m 0s —— View job

---

Coder-eval task lint (advisory)

4 task YAMLs changed; verdicts: 0 Critical, 1 High, 2 Medium, 1 Low, 0 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

❌ High — PR body says "Every command verified live against the test tenant" but does not claim the coder-eval tasks themselves were run and passed. Please edit the PR description to add a line like:
Ran skill-tasks-smoke-catalogs, skill-tasks-smoke-attributes, skill-tasks-smoke-data, skill-tasks-e2e-catalogs-annotations locally and they passed.

Per-task lint

tests/tasks/uipath-tasks/e2e_catalogs_and_annotations.yaml — verdict: High

Issues:

• [High] Prompt over-specification: lines 24–49 provide verbatim CLI commands for every step (uip tasks catalogs create --name … --retention-action Delete …, uip tasks data TASK …, etc.); an agent can copy-paste these without invoking the skill. Fix this →
• [Medium] Meaningful coverage: all criteria after step 2 (lines 79–152) use min_count: 0, so the test passes with only login status + tasks list if the tenant has no tasks. Consider adding a run_command or json_check criterion that verifies at least one catalog was created (e.g. checking tasks catalogs list output length).

Suggested fixes:

• Rewrite steps 3–4 as goal statements ("Create a task catalog with a Delete retention policy and 30-day retention period") instead of copy-paste CLI invocations.
• Set min_count: 1 on the catalog-lifecycle criteria (lines 79–106) — those steps are not conditional on discovering a task.

tests/tasks/uipath-tasks/smoke_catalogs.yaml — verdict: Medium (theme-captured; see Theme 1)

tests/tasks/uipath-tasks/smoke_task_attributes.yaml — verdict: Medium (theme-captured; see Theme 1)

tests/tasks/uipath-tasks/smoke_task_data.yaml — verdict: Low

Issues:

• [Low] Prompt over-specification: line 26 "save-data resolves the task type itself. Do NOT pass a --type flag" leaks procedural knowledge the skill should teach. The command_not_executed guard (line 46) already catches --type misuse, so the hint makes the test easier than intended.

Suggested fixes:

• Remove the --type hint from the prompt; let the skill teach this constraint. The command_not_executed criterion at line 46 will still catch misuse.

Within-PR duplicates

No duplicate clusters detected. The 3 smoke tests partition the new commands into distinct groups (catalogs, attributes, data), and the e2e test covers all at a different tier against a live tenant.

Themes

• Theme 1 (High): CLI verb reachability — 12 new verbs absent from catalog (all 4 tasks). Command patterns reference tasks catalogs list/get/create/update, tasks data, tasks save-data, tasks comments, tasks add-comment, tasks labels, tasks metadata — none present in assets/uip-catalog-snapshot.json (last generated 2026-08-10, CLI v1.200.0-dev.8173). The PR claims these were verified live, so the catalog likely just needs a refresh. Consider running python3 scripts/build-uip-catalog.py to update the snapshot, or wait for the nightly refresh-uip-catalog.yml workflow.

Conclusion

⚠ 4 task(s) have issues, max severity High. Advisory only — not blocking merge. The High on the e2e test's prompt over-specification is the most impactful finding; the CLI-verb-reachability theme will self-resolve when the catalog refreshes. Please also add a passing-run claim to the PR description.

[github-actions]

Claude finished @de-si's task in 2m 39s —— View job

---

PR Review: Extend uipath-tasks skill

• Read review criteria and project conventions
• Read full diff and all changed files
• Check skill structure and frontmatter
• Check test tasks (tags, IDs, criteria)
• Check cross-references, links, and secrets
• Post review findings

---

Summary

Extends the uipath-tasks skill with five new command areas (catalogs, comments, labels, metadata, data). Adds 3 reference files, 4 new Critical Rules, a navigation table expansion, 3 smoke tests, and 1 e2e test. Well-structured PR with clean separation between SKILL.md and references.

Change-by-Change Review

1. skills/uipath-tasks/SKILL.md — SKILL.md updates

Severity: OK

Clean additions throughout:

• Description updated to cover new command areas (323 chars, well under 1024 cap).
• "When to Use" expanded with 5 new bullet points for the new capabilities.
• Critical Rules 9–12 are well-written, prescriptive, and cover the important gotchas (folder scoping, retention actions, label replacement semantics, save-data type routing).
• Task Navigation table expanded with 10 new rows — all commands include --folder-id and match the reference docs.
• References section updated with pointers to the 3 new reference files.
• Frontmatter name matches folder. All relative links resolve. No cross-skill references.

2. skills/uipath-tasks/references/task-catalogs.md — New reference

Severity: OK

Covers the full catalog CRUD lifecycle with correct CLI examples. Retention table is clear. The "Folder scoping" section at the top is nearly verbatim with Critical Rule #9 in SKILL.md — see note in §Duplication below, but it's minor given each reference is designed to be self-contained.

3. skills/uipath-tasks/references/task-metadata.md — New reference

Severity: OK

Combines metadata, comments, and labels in one file. Good flag table for metadata. Labels replacement semantics clearly documented. Grouping these three lightweight operations in one file is a reasonable choice.

4. skills/uipath-tasks/references/task-data.md — New reference

Severity: OK

Concise, focused. Type-routing explanation is clear. The warning about saving to completed tasks or unknown fields is useful agent guidance.

5. tests/tasks/uipath-tasks/e2e_catalogs_and_annotations.yaml — New e2e test

Severity: Medium

expected_turns: 34 exceeds max_turns: 32 (e2e_catalogs_and_annotations.yaml:13-14). The agent will be cut off before reaching the expected number of turns. Either raise max_turns to at least 34, or lower expected_turns to fit within 32.

Fix this →

Note: the existing e2e_fetch_tasks.yaml has the same pattern (31 vs 30) — this may be an intentional convention in this skill, but it's worth aligning.

The 13 success criteria are well-designed: conditional steps use min_count: 0 so the test passes cleanly when the tenant has no tasks. The command_not_executed guard preventing complete/assign/reassign is important for safety.

6. tests/tasks/uipath-tasks/smoke_catalogs.yaml — New smoke test

Severity: OK

Clean. Tests all four catalog CRUD operations plus a negative guard for invalid retention actions. Weights are reasonable (2.0–2.5 for commands, 1.0 for the guard).

7. tests/tasks/uipath-tasks/smoke_task_attributes.yaml — New smoke test

Severity: OK

Comprehensive — covers comments, labels (set + clear), metadata (title + priority), and catalog association/disassociation. 7 criteria with good coverage of the new command surface.

8. tests/tasks/uipath-tasks/smoke_task_data.yaml — New smoke test

Severity: OK

Focused on get/save data with a command_not_executed guard ensuring --type is not passed to save-data. Minimal and correct.

What's Missing

• mode:* and lifecycle:* tags — Required on every task per tests/README.md line 114. None of the 4 new tasks include them. The existing tests in this skill also lack them, so this is a pre-existing gap — but new tasks are a good opportunity to fix it. Suggested values:

  • smoke_catalogs.yaml: mode:operate, lifecycle:setup
  • smoke_task_attributes.yaml: mode:operate, lifecycle:setup
  • smoke_task_data.yaml: mode:operate, lifecycle:setup
  • e2e_catalogs_and_annotations.yaml: mode:operate, lifecycle:setup

  Fix this →

• No passing-run claim in PR body — Per .claude/rules/test-writing.md step 5: "Run the task with coder-eval and add a passing-run claim to the PR." The PR description mentions "Every command verified live against the test tenant" but doesn't include a coder-eval run ID or link.

Area Ratings

Area	Status	Notes
Frontmatter	OK	Valid YAML, name matches folder, description 323 chars
E2E Tests	Issue	expected_turns > max_turns in e2e; missing mode:*/lifecycle:* tags on all 4 tasks
Skill Body	OK	Critical Rules, Navigation, When to Use — all properly extended
References & Assets	OK	3 new references, kebab-case, all links resolve, no cross-skill deps
Repo Hygiene	OK	CODEOWNERS already covers both paths; no secrets; skill-status.json has entry

Issues for Manual Review

• CLI command accuracy — The new uip tasks subcommands (catalogs list/get/create/update, comments, add-comment, labels, metadata, data, save-data) and their flags should be verified against the current CLI version. The PR claims live verification — domain owners should confirm flag names and response codes match the actual API.
• save-data type routing — The claim that save-data resolves task type automatically (no --type flag needed) is a key behavioral assertion. If the CLI changes this, both the reference and the smoke test's command_not_executed guard would need updating.

Conclusion

Clean, well-structured PR. The skill additions are consistent, prescriptive, and follow repo conventions. Two actionable items:

1. Medium — Fix expected_turns/max_turns mismatch in e2e_catalogs_and_annotations.yaml
2. Medium — Add required mode:* and lifecycle:* tags to all 4 new test tasks (pre-existing gap in this skill, but new tasks should set the standard)

Neither blocks merge — both are quality improvements that can land in a follow-up if preferred.

---