docs(guardrails): document bring-your-own (BYOG) guardrail configurations [AL-513]

[apetraru-uipath]

What changed?

Documents the bring-your-own guardrail (BYOG) feature on the agent-authoring side across every place it touches — discovery, low-code/coded authoring, review, and troubleshooting. Ticket: AL-513.

• uipath-agents (low-code + coded guardrails, guardrails-recommend — 4 files): the BYOG concept, the --byo filter on uip agent guardrails list, the IsByo/Byo* fields, and that Validator is not unique once a BYOG config exists for that type (a tenant can have both a built-in and a BYO entry sharing the same name — disambiguate via IsByo). A guardrail pins to a specific BYO configuration via ByoValidatorName — unique per tenant, the same value for both low-code (byoValidatorName field) and coded (ByoValidator(<ValidatorName>)) — with no connection_id passed by the agent; the platform resolves the connection server-side.
• uipath-review (2 files): the same disambiguation note for audit/correctness checks, plus a callout that a BYO entry's Status: Disabled is a tenant config switch, not a schema defect.
• uipath-troubleshoot: BYOG as a possible cause of a guardrail violation, a step to cross-check BYOG health via the admin CLI, and a resolution path — including re-enabling a disabled configuration — for a dead connection or disabled config.
• New test: tests/tasks/uipath-agents/lowcode/guardrails/byog_pinning/ — a coder-eval task covering the full agent-authoring thread. See Testing below.

The admin-side reference (uip guardrails byo-configurations CRUD, registering a BYOG configuration) is a separate PR — #2550 — so this one doesn't carry uipath-platform CODEOWNERS approval. Every reference to that content here goes through the skill-level /uipath:uipath-platform link, not a literal path, so there's no merge-order dependency.

Review fix (valentinabojan): an earlier draft used a field named byoConfigurationId to pin a low-code guardrail to a BYO configuration — that field doesn't exist. Fixed everywhere in this PR (mapping table, authoring guidance, anti-pattern rule, both disambiguation notes, and the byog_pinning test's actual grading key) to use ByoValidatorName, which is what both low-code and coded agents really reference a configuration by.

How has this been tested?

• scripts/check-skill-verbs.py and scripts/check-task-driver.py are clean on every touched file.
• byog_pinning task: a pre_run script (seed_byog.py) provisions the tenant's BYOG configuration directly via the CLI — the admin prerequisite is never the agent's job. The user prompt asks for an agent guarded by that configuration; the check (check_byog_pinning.py) re-confirms the seed landed, then requires the agent.json guardrail to carry byoValidatorName matching it — verified against both a positive control (correct pin passes) and two negative controls (wrong name fails; the old byoConfigurationId-only shape correctly fails too, since that field isn't checked). seed_byog.py registers the configuration against an inert placeholder connection GUID, never a real Integration Service connection — this task only pins the configuration, never executes the guardrail, so no live credentialed connection needs to be involved.
• Attempted a full coder-eval run of skill-agent-guardrail-byog-pinning against the only tenant available to this session (codereval on alpha): it fails at pre_run — seed_byog.py exits non-zero on ByoGuardrailsUnavailable before the agent starts (status=ERROR score=0.000 iterations=0, ~5s). This is the fail-closed design working as intended, not a task defect: no tenant currently reachable from this environment has the BYOG feature flag enabled, so the full task can't run to completion anywhere until that flag is turned on somewhere — hence smoke-blocked rather than smoke.
• Once BYOG is enabled on any reachable tenant: flip the tag to smoke and run the two-model validation per the Run Skill Smoke Tests runbook.

Are there any breaking changes?

• None

[github-actions]

Claude finished @apetraru-uipath's task in 2m 23s —— View job

---

Coder-eval task lint (advisory)

1 task YAML changed; verdicts: 0 Critical, 0 High, 0 Medium, 1 Low, 0 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

❌ High — PR body does not claim a full coder-eval run of the changed task passed. The author describes component-level verification ("The check script passes on a correctly pinned artifact and fails a pin mismatch against a stubbed byo-configurations list") and CLI-level validation (uip agent validate accepts the byoConfigurationId field), but no claim of a full coder-eval run passing.

This is understandable given the task is tagged smoke-blocked (BYOG feature flag is off on the shared smoke tenant), but the convention is to claim a passing run against some tenant. Please edit the PR description to add a line like:

Ran skill-agent-guardrail-byog-pinning locally and it passed.

…or note explicitly that the full task cannot be run on any available tenant until the feature flag is flipped.

Per-task lint

tests/tasks/uipath-agents/lowcode/guardrails/byog_pinning/byog_pinning.yaml — verdict: Low

Issues:

• [Low] Meaningful coverage: sibling guardrail tasks (e.g. pii_detection, custom_word) include a json_check on the .uipx project registration and an advisory command_executed for uip solution init. This task omits both. The gap is minor — file_exists on ByogPinSol/ByogPinAgent/agent.json implicitly proves the solution was scaffolded, and the heavy run_command check (check_byog_pinning.py, weight 5.0) is strong — but adding the .uipx check would align with sibling conventions.
• [Info] CLI verb reachability: scripts/check-cli-verbs.py could not be run (permission). Verbs in command_pattern fields (agent guardrails list, agent init, agent refresh, agent validate) are standard; manual inspection shows no retired verbs. The --byo flag in the lookahead on line 38 is a filter flag, not a verb — no concern.

Suggested fixes:

• Consider adding an advisory (pass_threshold: 0.0) command_executed criterion for uip\s+solution\s+init and a json_check on ByogPinSol/ByogPinSol.uipx asserting length(Projects) >= 1 (weight 1.5, pass_threshold: 1.0), matching pii_detection.yaml lines 20–48.

Within-PR duplicates

• No duplicate clusters detected (only 1 new task YAML).

Conclusion

• ⚠ 1 task has issues, max severity Low. The missing passing-run claim is the main action item. Advisory only — not blocking merge.

---

[github-actions]

Claude finished @apetraru-uipath's task in 5m 19s —— View job

---

PR Review: docs(guardrails) — BYOG configurations AL-513

• Read review criteria and project conventions
• Analyze full PR diff
• Read all changed files in full context
• Check links, cross-skill refs, secrets, CODEOWNERS
• Verify test task structure
• Post review findings

---

Summary

Documents the bring-your-own guardrail (BYOG) feature across the uipath-platform, uipath-agents, uipath-review, and uipath-troubleshoot skills, plus a new byog_pinning coder-eval task. Well-structured, thorough, and consistent with the existing BYO LLM convention. Two actionable issues found.

Change-by-Change Review

1. skills/uipath-platform/SKILL.md — frontmatter

Severity: Medium

The description field is now 1038 characters — 14 chars over the repo's 1024-character cap (.claude/rules/skill-structure.md). The pre-PR value was 1001 chars; the addition of , BYO guardrail (BYOG) configurations pushed it over. The when_to_use field additions are fine (it has a separate budget).

Suggestion: shorten to , BYOG configurations (drop "BYO guardrail" — redundant with the acronym expansion) or abbreviate another part of the description to reclaim ~14 chars.

Fix this →

The body additions (bullet point at line 55, task-navigation table rows at lines 174-175, references list entry at line 179) are clean, well-placed, and all links resolve.

2. skills/uipath-platform/references/guardrails/byo-configurations.md (new)

Severity: OK

Excellent new reference. Covers the full list/create/update/delete surface with copy-paste CLI commands, JSON response examples, field-level documentation, error paths (feature-not-available vs. config-not-found vs. permission), diagnostics, and the agent-authoring handoff. The --output json convention is followed throughout. All relative links (../llmgateway/byo-connections.md, ../integration-service/connections.md) resolve. Cross-skill references use the /uipath: plugin delegation syntax correctly.

3. skills/uipath-agents/references/coded/capabilities/guardrails/guardrails.md

Severity: OK

New ## BYO (bring-your-own) validators section and Critical Rule 18 are well-placed. The "never fabricate" posture is consistent with the existing EscalateAction pattern (Critical Rule 14). Discovery steps include both --byo filter and the admin-side byo-configurations list cross-check. The cross-reference to uipath-platform uses the plugin link syntax, not a file import.

4. skills/uipath-agents/references/coded/capabilities/guardrails/guardrails-recommend.md

Severity: OK

Three additions: disambiguation callout at the lookup step, "default to built-in" guidance at the matching step, and Critical Rule 16. All reference the new guardrails.md#byo-bring-your-own-validators anchor which exists. The correctness-check update at line 239 is precise — it explains how to match against the right list entry when both built-in and BYO exist.

5. skills/uipath-agents/references/lowcode/capabilities/guardrails/guardrails.md

Severity: OK

Mirrors the coded version with the low-code-specific shape (byoConfigurationId in JSON instead of ByoValidator(...) in Python). The IsByo row added to the CLI-to-JSON mapping table correctly notes it's "Not itself a JSON field." Critical Rule 21 is consistent with coded Rule 18.

6. skills/uipath-agents/references/lowcode/capabilities/guardrails/guardrails-recommend.md

Severity: OK

Same pattern as the coded counterpart. Critical Rule 15 added. The correctness-check update at line 215 correctly explains the byoConfigurationId → ByoConfigurationId matching path for disambiguation.

7. skills/uipath-review/references/agents/guardrails/guardrails-review.md

Severity: OK

Two clean additions: the (Validator, IsByo) disambiguation callout and the Status: "Disabled" is-a-config-switch callout. Both reference the uipath-agents skill via plugin link syntax.

8. skills/uipath-review/references/agents/guardrails/coded-guardrails-review.md

Severity: OK

Same pattern as the low-code review file. The "Status: Disabled is a configuration switch, not a wiring bug" callout is valuable — prevents the review flow from mis-diagnosing a disabled BYOG config as a code defect.

9. skills/uipath-troubleshoot/references/products/agents/playbooks/guardrail-violation.md

Severity: OK

Adds BYOG as a cause in the "What can cause it" list, a cross-check step in the investigation flow (step 4), and a resolution path. The --output-filter usage is consistent with the rest of the file. The resolution paragraph links to uipath-platform via plugin syntax.

10. tests/tasks/uipath-agents/lowcode/guardrails/byog_pinning/byog_pinning.yaml

Severity: OK

Well-designed test task. Tags follow conventions (uipath-agents first, smoke-blocked tier, mode:build). task_id matches the skill-<domain>-<capability> pattern. Success criteria cover discovery (--byo flag), scaffolding (init/refresh/validate), artifact existence, and the pinning check via run_command with the highest weight (5.0). The pre_run/post_run scripts handle seeding and cleanup correctly.

The smoke-blocked tag is consistent with the existing discovery.yaml in the same directory and is well-documented with a comment explaining when to flip to smoke.

11. tests/tasks/uipath-agents/lowcode/guardrails/byog_pinning/seed_byog.py

Severity: OK

Fails closed on feature-flag-off, unknown state, and delete failures. The placeholder connection GUID design is well-documented and avoids routing PII to a third-party service. Reset semantics ensure clean reruns.

12. tests/tasks/uipath-agents/lowcode/guardrails/byog_pinning/check_byog_pinning.py

Severity: OK

Re-fetches the configuration id live rather than reading from a seed artifact — the agent can't shortcut discovery. Validates the full guardrail shape: $guardrailType, validatorType, byoConfigurationId match, block action, PascalCase entities. Failure messages are descriptive with actual-vs-expected values.

13. tests/tasks/uipath-agents/lowcode/guardrails/byog_pinning/cleanup_byog.py

Severity: OK

Best-effort sweep, exits 0 always. Consistent with the cleanup pattern noted in the docstring (tests/tasks/uipath-platform/cleanup.py).

What's Missing

• CODEOWNERS entry for skills/uipath-platform/references/guardrails/: Every other reference subdirectory under uipath-platform has a dedicated CODEOWNERS entry (traces/, integration-service/, data-fabric/, llmgateway/, context-grounding/). The parent /skills/uipath-platform/ path covers it, so this is Low severity, but adding a dedicated entry (with the guardrails team) would follow the established pattern.

Area Ratings

Area	Status	Notes
Frontmatter	Issue	description 1038 chars — 14 over the 1024 cap
E2E Tests	OK	byog_pinning task is well-structured with smoke-blocked tier and documented unblock criteria
Skill Body	OK	Bullet points, table rows, and reference links are clean
References & Assets	OK	New byo-configurations.md is thorough; all relative links resolve
Repo Hygiene	OK	No secrets, no cross-skill file imports, scoped changes; CODEOWNERS coverage via parent path (dedicated entry recommended)

Issues for Manual Review

• CLI flag accuracy: The --byo filter for uip agent guardrails list and the --output-filter JMESPath expressions for byo-configurations list cannot be verified without the actual CLI. If --byo is not yet shipped in the @latest CLI, agents will hit a flag-not-recognized error.
• FallbackOnUiPath default: The docs state --fallback-on-ui-path defaults to false on create. Verify this matches the backend — a wrong default here means agents omitting the flag get the opposite behavior.

Conclusion

Clean PR. One Medium issue: the uipath-platform description is 14 chars over the 1024-char repo cap — shorten it before merging. The CODEOWNERS gap is low-severity but worth adding. The documentation is well-structured, consistent across skills, and the test task is thoughtfully designed with fail-closed seeding, live re-fetch verification, and documented unblock criteria.

---

|

[valentinabojan]

byoConfigurationId is not used. byoValidatorName is the one used by the lo coded agents

[apetraru-uipath]

Good catch — byoConfigurationId isn't a real field. Fixed to ByoValidatorName (matches the coded-agent ByoValidator(<ValidatorName>) pattern — both low-code and coded now reference a configuration the same way). This file has since moved to a dedicated PR (#2550) so it doesn't ride on this PR's approvals — the fix is there: d4ace1ec4

[valentinabojan]

We also need a cli command for test configuration similar to BYO llm config.

We enforce the same probe before saving a new connection via the UI.

[apetraru-uipath]

Confirmed against the CLI source — guardrails byo-configurations create/update run no server-side validate/probe, unlike llm-configuration byo-connections which always does POST .../validate before saving. That's a real CLI feature gap (a validate step, or a dedicated test verb), not something a docs PR can add on its own — worth a follow-up ticket against the guardrails-tool CLI. In the meantime I added a 'Known gap' section documenting this explicitly and pointing at uip is connections ping <connection-id> as the closest available partial check (confirms the connection is alive, not that it implements the guardrail contract). Moved with the rest of this file to #2550: d4ace1ec4

[valentinabojan]

Do we need to refer to this here as well? Isn't it enough to have it on line 177?

[apetraru-uipath]

This mirrors the existing convention for every other reference doc in this file, not something new for BYOG — LLM Gateway, Traces, Integration Service, and Data Fabric are each listed in both the Task Navigation table (quick 'I need to...' lookup) and the bottom References list (the file's own index). Line 177 and line 314 serve different browsing patterns; dropping just the BYOG entry from References would make it the one exception. Happy to revisit the whole file's convention separately if that duplication should go away everywhere, but keeping it consistent for now. This content is now on #2550.