fix(process-mining): grade the real add-table verb, and make the verb linter see leaf verbs

[Cellcote]

Why

The process-mining eval set looked flaky across the two most recent nightly runs
(2026-08-05,
2026-08-10).
It wasn't flake — it was a grader bug that the CLI-verb linter could not see.

Only 2 of 6 tasks moved between the runs, each by exactly one criterion:

Task	2026-08-05	2026-08-10
skill-pm-add-table-queryable	1.0	0.778 FAILURE
skill-pm-custom-app-pipeline-e2e	0.833 FAILURE	1.0

(The runs are not otherwise comparable — different model, different uip build,
and the task set itself changed between them. Details at the bottom.)

1. skill-pm-add-table-queryable graded a verb that doesn't exist

The 3.0-weight criterion required uip pm apps model add-table. The catalog has
uip pm apps data-model add-table; apps model is the semantic model
(fields / get / update) and has no add-table child. The skill teaches the
correct verb in both SKILL.md (Rule 1) and references/data-model.md.

So the eval rewarded a hallucinated verb and penalised agents that followed the
skill:

• 08-05 — the agent typed apps model add-table, it errored, it retried with
  apps data-model add-table. The regex matched the failed attempt → passed.
• 08-10 — the agent went straight to the correct verb → matched nothing →
  0.778, FAILURE.

The green run was the wrong result. Fixed to apps\s+data-model\s+add-table.

2. Why check-cli-verbs.py reported the set clean

Two defects, each now covered by a regression test:

• \b discarded the leaf verb. On AT_BOUNDARY the walker bailed and
  called _trim_to_word_boundary, dropping the token it had just accumulated.
  But \b after a word character confirms the token ended — the opposite of a
  partial match. Since uip <group> <leaf>\b is this repo's standard pattern
  shape, the checker was only ever validating the command group.
• Longest-prefix matching swallowed bogus subcommands. best_match then fell
  back to pm apps model, a real group, and reported reachable.

Third gap, also fixed: iter_command_patterns only yielded command_executed,
so negative criteria were never verb-checked. A negative whose verb does not
exist can never fire, so it awards its weight on every run. command_executed
with max_count: 0 is the other negative idiom here and gets the same handling.

Negatives report Medium, not High — they inflate scores rather than breaking
the run, and the guard is sometimes deliberate.

False positives are guarded three ways, all tested: positional arguments after a
leaf verb (is resources run list <connector> <activity>), partial kebab tokens
(create- from a pattern matching create-raw / create-resource), and
anything under an unwalkable_groups entry.

Also pinned encoding="utf-8" on the file reads — the script crashed on Windows
under cp1252.

3. Tag hygiene

Five of six tasks were missing required tags or using values outside the closed
vocabularies in tests/README.md §Tag Taxonomy (mode:fix, lifecycle:extend,
lifecycle:maintain are not valid; tier was absent on four). Those tasks were
invisible to make targets, coverage reports and evalboard drilldown.

Task	Before	After
add_table_queryable	mode:build, lifecycle:extend	integration, mode:build, lifecycle:setup
app_types_discovery_smoke	smoke, mode:diagnose, discover	smoke, mode:diagnose, lifecycle:discover
mapping_fix_in_place	mode:fix, lifecycle:maintain	integration, mode:diagnose, lifecycle:setup
query_group_by_sugar	mode:diagnose	smoke, mode:diagnose, lifecycle:discover
transform_fix_apply_not_reingest	mode:fix, lifecycle:maintain	integration, mode:diagnose, lifecycle:setup

The two repair tasks follow the precedent set by
uipath-api-workflow/diagnose/runtime_fix (mode:diagnose for "here is an
error, fix it"), with lifecycle:setup because they mutate tenant state. The
bare discover tag duplicating lifecycle:discover was dropped.
custom_app_pipeline_e2e was already correct.

Verification

• pytest tests/scripts/test_verb_checkers.py — 21 passed (14 existing + 7 new).
• check-cli-verbs.py tests/tasks/uipath-process-mining/*.yaml — 0 High, 0 Medium.
• Repo-wide across all task YAMLs — 0 High, 2 Medium, 124 Info.
• No CI gate changes behaviour: nothing in .github/ invokes this script;
  /audit-verbs uses --report (exit 0 on Medium) and /lint-task merges
  --json findings.

The 2 Medium findings are both in
tests/tasks/uipath-governance/aops-policy/deployment_configure_smoke.yaml —
negatives against deployment user add / add-user, which don't exist (the real
verb is configure, which auto-registers). Plausibly intentional guards; left
alone as out of scope, but worth a look from that skill's owner.

Passing-run claim: the eval tasks themselves were not re-run against a tenant
for this PR — the change is to grader regexes and tags, verified statically
against the uip catalog (1.200.0-dev.8173) and the linter's own test suite.
Re-running skill-pm-add-table-queryable should be part of the next nightly.

Not addressed here

Deliberately out of scope, from the same investigation:

• The runs aren't comparable. 08-05 ran gemini-3.5-flash on uip 8093;
  08-10 ran gpt-5.6-terra on 8173, and mapping_fix_in_place gained two
  criteria in between. With one replicate per task, a single criterion flips
  the whole task. Pin the model or run ≥3 replicates before calling anything flaky.
• custom_app_pipeline_e2e's "inspect the transformations" criterion is
  incidental to the task goal and cost the 08-05 run a FAILURE for no behavioural
  reason — drop it or make it advisory (pass_threshold: 0).
• Coverage: 20/44 uip pm verbs are referenced. Critical Rule 10
  (apps model fields, the DNA-46960 duration lockout) has zero tasks despite
  being called out in the frontmatter, and the whole query family beyond run
  is untested. Separate PR.

🤖 Generated with Claude Code

[github-actions]

Claude finished @Cellcote's task in 3m 18s —— View job

---

Coder-eval task lint (advisory)

• Read lint rubric
• Identify changed task YAMLs (5 files)
• Check PR body for passing-run evidence
• Read and lint each changed task
• Within-PR duplicate check
• Post results

---

Coder-eval task lint (advisory)

5 task YAMLs changed; verdict counts: 0 Critical, 0 High, 0 Medium, 0 Low, 5 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

❌ High — PR body explicitly states "the eval tasks themselves were not re-run against a tenant for this PR." The author provides a rationale (changes are to grader regexes and tags only, verified statically against the uip catalog and the linter's test suite) and defers live validation to the next nightly run. This is reasonable context, but per convention please consider adding a passing-run claim after the next nightly confirms the fix, e.g.:

Ran skill-pm-add-table-queryable in the <date> nightly and it passed.

Per-task lint

tests/tasks/uipath-process-mining/add_table_queryable.yaml — verdict: OK

The core fix in this PR: command_pattern corrected from apps\s+model\s+add-table to apps\s+data-model\s+add-table (line 52), matching the real catalog verb. Tags fixed to include required integration tier and valid lifecycle:setup. Good mix of positive criteria (command_executed for add-table and ingestions create, file_contains for foreignKeys/Cases) and negative guards (Tags.sql and Due_dates.sql smuggling). No issues.

tests/tasks/uipath-process-mining/app_types_discovery_smoke.yaml — verdict: OK

Tag fix only: bare discover → lifecycle:discover (line 11). Task structure unchanged. Lightweight smoke test with skill_triggered + command_executed for app-types list — appropriate depth for a command-construction smoke without a live tenant.

tests/tasks/uipath-process-mining/mapping_fix_in_place.yaml — verdict: OK

Tag fix only: mode:fix, lifecycle:maintain → mode:diagnose, lifecycle:setup (line 19). Task structure unchanged. Strong criteria mix: 4 positive command checks (get, update --file, update --etag, ingestions create), 1 file content check (date format correction with includes/excludes), and 4 negative guards (no delete, no recreate, no re-upload, no published-stage target).

tests/tasks/uipath-process-mining/query_group_by_sugar.yaml — verdict: OK

Tag fix only: added missing smoke tier and lifecycle:discover (line 11). Task structure unchanged. Criteria validate both the correct approach (--group-by/--metric sugar with valid aggregation function) and the anti-pattern (no raw --body-json with bare column names, no invented function names).

tests/tasks/uipath-process-mining/transform_fix_apply_not_reingest.yaml — verdict: OK

Tag fix only: mode:fix, lifecycle:maintain → mode:diagnose, lifecycle:setup (line 16). Task structure unchanged. Well-designed mirror of mapping_fix_in_place — same repair shape but opposite re-ingest direction (this one asserts ingestions create must NOT run, the mapping fix requires it). 3 positive + 3 negative criteria.

Within-PR duplicates

No duplicate clusters detected. All 5 changes modify existing tasks (no new task_ids added). The repair-pair tasks (mapping_fix_in_place / transform_fix_apply_not_reingest) exercise different subsystems with opposite re-ingest polarity — complementary, not duplicative.

Conclusion

⚠ Per-task lint is clean (5/5 OK), but the PR body does not claim a passing run for the modified tasks. Advisory only — not blocking merge. The tag and verb-pattern fixes are well-motivated and the static verification (catalog check + linter test suite) provides confidence; a nightly confirmation would close the loop.

Note: Could not execute check-cli-verbs.py --json due to CI runner permissions — CLI verb reachability was assessed manually against the PR description's catalog verification claim. The author's own PR states the verbs were verified against catalog 1.200.0-dev.8173 and check-cli-verbs.py reports 0 High, 0 Medium for this skill's tasks.

[github-actions]

Claude finished @Cellcote's task in 4m 26s —— View job

---

PR Review: fix(process-mining) — grader verb fix & verb linter improvements

• Read review criteria and project conventions
• Read full diff
• Read each changed file in full context
• Check script correctness (check-cli-verbs.py)
• Check test correctness (test_verb_checkers.py)
• Check YAML task file changes
• Post review findings

---

Summary

Fixes a grader bug where add_table_queryable.yaml matched the nonexistent verb pm apps model add-table instead of the real pm apps data-model add-table, then fixes two defects in check-cli-verbs.py that masked it: (1) \b after a complete token discarded the leaf verb, so only the command group was ever verified; (2) command_not_executed criteria were never scanned. Also corrects tags on 5 of 6 process-mining tasks to align with the tag taxonomy.

Change-by-Change Review

1. scripts/check-cli-verbs.py — verb linter fixes

Severity: OK

Three well-targeted fixes:

• \b handling (line 193–201): The new branch correctly distinguishes \b after a word character (confirms token ended → keep the accumulated literal) from \b after a non-word character (ambiguous → trim). The all(p and p[-1].isalnum() for p in paths) guard is correct for multi-path scenarios — if any branch doesn't end cleanly, fall back to trimming. This is compatible with the existing Issue 9 test (uip\s+\bfoo\b) since that pattern has \b after a space, not after alnum.

• best_match strict mode (line 277–293): The logic correctly prevents longest-prefix fallback from swallowing bogus subcommands. When a matched group has known children and the next token looks like a subcommand (lowercase kebab-case) that isn't one of them, it returns None. The three guard rails are sound: (a) under_unwalkable — skip the check when children are unknown; (b) partial kebab tokens ending in - are not flagged; (c) leaf verbs without children pass through (so positional arguments aren't mistaken for subcommands). Verified against catalog: has_children("pm apps model", catalog) → True (fields, get, update exist), while has_children("is resources run list", catalog) → False (no children), matching the test expectations.

• Negative criteria scanning (line 326–338, 385–401): Both command_not_executed and command_executed with max_count: 0 are now scanned. The Medium severity (vs High for positive criteria) is the right call — a dead negative inflates scores but doesn't break runs.

• encoding="utf-8" pins (lines 69, 82, 94, 346): Correct fix for Windows cp1252 crashes.

• load_unwalkable (line 73–82): Clean extraction, reads the catalog once. Note: load_catalog also reads the same file — the two reads could be combined, but since this runs at script startup and not in a hot loop, it's not worth optimizing.

2. tests/scripts/test_verb_checkers.py — 7 new regression tests

Severity: OK

All 7 new tests are well-structured, with clear docstrings explaining the bug they prevent:

• test_word_boundary_after_complete_token_keeps_leaf — directly reproduces the grader bug
• test_bogus_leaf_under_real_group_is_unknown — covers the strict-mode best_match fix
• test_positional_args_after_leaf_verb_stay_reachable — false-positive guard
• test_partial_kebab_token_is_not_reported_missing — false-positive guard
• test_unwalkable_group_never_reports_missing_leaf — unwalkable guard
• test_negative_criteria_are_scanned_and_reported_medium — negative criterion scanning
• test_zero_bounded_command_executed_counts_as_negative — max_count: 0 idiom

Minor nit (Low): In test_negative_criteria_are_scanned_and_reported_medium (line 342–344), the monkeypatch.setattr calls on load_catalog, load_renames, and load_unwalkable are dead code — lint_file receives catalog/renames/unwalkable as explicit parameters and never calls those functions. Harmless but slightly misleading.

3. tests/tasks/uipath-process-mining/add_table_queryable.yaml — verb + tag fix

Severity: OK

• Verb fix (line 52): apps\s+model\s+add-table → apps\s+data-model\s+add-table. Verified against catalog: pm apps data-model add-table exists at line 1156 of uip-catalog-snapshot.json; pm apps model has no add-table child.
• Description (line 4): Updated to match the corrected verb.
• Tags (line 16): mode:build, lifecycle:extend → integration, mode:build, lifecycle:setup. Adds required tier tag (integration) and replaces invalid lifecycle:extend with valid lifecycle:setup. Per taxonomy, lifecycle values are discover, generate, setup — extend is not in the vocabulary.

4. tests/tasks/uipath-process-mining/app_types_discovery_smoke.yaml — tag fix

Severity: OK

Tags: discover (bare) → lifecycle:discover. Drops the bare duplicate of the namespaced tag. Correct per taxonomy — lifecycle:discover is the proper form.

5. tests/tasks/uipath-process-mining/mapping_fix_in_place.yaml — tag fix

Severity: OK

Tags: mode:fix, lifecycle:maintain → integration, mode:diagnose, lifecycle:setup. Adds missing tier tag. Replaces invalid mode:fix and lifecycle:maintain with valid values. The mode:diagnose choice follows the precedent noted in the PR body for "here is an error, fix it" tasks.

6. tests/tasks/uipath-process-mining/query_group_by_sugar.yaml — tag fix

Severity: OK

Tags: adds smoke tier and lifecycle:discover. Both required and previously missing.

7. tests/tasks/uipath-process-mining/transform_fix_apply_not_reingest.yaml — tag fix

Severity: OK

Same pattern as mapping_fix_in_place: adds integration tier, replaces invalid mode:fix/lifecycle:maintain with mode:diagnose/lifecycle:setup.

What's Missing

Nothing identified. The scope is tight — grader fix, linter fix, tag hygiene — and each change is complete. The PR description explicitly defers coverage gaps and the custom_app_pipeline_e2e criterion question to separate work.

Area Ratings

Area	Status	Notes
Frontmatter	N/A	No SKILL.md changes
E2E Tests	OK	Task YAML fixes are well-scoped; linter tests cover regressions
Skill Body	N/A	No skill content changes
References & Assets	N/A	No reference changes
Repo Hygiene	OK	No new paths, no secrets, scoped changes

Issues for Manual Review

• Verb correctness is catalog-verified but not tenant-verified. The PR correctly notes the eval tasks were not re-run. The next nightly run on add_table_queryable will be the real validation that the corrected regex matches what the agent actually types.
• Tag reclassification (mode:fix → mode:diagnose). The reasoning (follows uipath-api-workflow/diagnose/runtime_fix precedent) is sound, but the skill owner should confirm these two repair tasks (mapping_fix_in_place, transform_fix_apply_not_reingest) are correctly diagnose rather than build.

Conclusion

Clean PR. The grader bug is real, the fix is correct (verified against catalog), the linter improvements are well-reasoned with good false-positive guards, and the tag changes align with the documented taxonomy. All 7 new tests target specific regressions. No issues found that would block merge.

[Cellcote]

Superseded by the split requested in review:

• fix(process-mining): grade the real add-table verb and fix required tags #2545 — the process-mining eval-set fix (add-table verb + required tags)
• fix(tests): make the CLI-verb linter see leaf verbs and negative criteria #2546 — the check-cli-verbs.py fix that catches this class of bug

Same commit content, separated so the grader fix and the tooling fix can be
reviewed and reverted independently. Merge #2545 first — #2546 reports the
process-mining task as High until it lands.