fix(maestro-flow): teach one-solution-one-folder; repin and regrade slack_weather_pipeline - #2554
Draft
jiyangzh wants to merge 3 commits into
Draft
fix(maestro-flow): teach one-solution-one-folder; repin and regrade slack_weather_pipeline#2554jiyangzh wants to merge 3 commits into
jiyangzh wants to merge 3 commits into
Conversation
`--all-folders` is mandated for connection discovery across the connector, connector-trigger and HTTP plugins, but nothing told the agent how to CHOOSE among the folders it widens into. A deployed solution carries one folder binding, and `<bindings.FolderKey>` matches by name only, so a flow that binds connections from two folders resolves one folder key and every connection outside it 404s. The failure is silent until runtime: mixed-folder bindings pass `flow validate` and fault at `flow debug` with `[102001] Integration Services resource not found` plus `Unresolved context binding placeholder "<bindings.FolderKey>"`. State the constraint once at the selection step in the connector plugin impl, add a Common Errors row for the signature, and qualify the `--all-folders` mandate at each site with a one-line pointer. Found while auditing skill-flow-slack-weather-pipeline, which has failed deterministically since #1030 (ac39db7) for exactly this reason: the tenant holds two `is-sandboxes` Slack connections in different folders.
PR #1030 (ac39db7) deleted the `Shared/uipath-maestro-flow` folder hint from this prompt, assuming `uip is connections list --all-folders` made it redundant. It does not. The tenant holds two `is-sandboxes` Slack connections in different folders; `--all-folders` surfaces the maestro-case one; the solution's single `<bindings.FolderKey>` then resolves to the weather connection's folder and the Slack lookup 404s. Two archived runs, same failure, otherwise correct agent work: - skill-flow-slack-weather-pipeline/00, 2026-08-10 — FAILURE, weighted 0.375, `<bindings.FolderKey>` unresolved on connection e03f734e, folder 5da18ec0, element readOfficeBellevueChannel1. - eval/skill-flow-slack-weather-pipeline/00, 2026-08-06 — FAILURE, weighted 0.375, [102001] on element readOfficeBellevueDescription1. The prompt states the folder as a bare environment fact. The general lesson — one solution binds one folder — belongs to the skill, not to every task prompt, and lands in the preceding commit. The same audit found three grading flaws, fixed here: - The prompt demanded only `validate` while the 5.0-weight criterion graded a live `flow debug`. The completion bar now names debug. - The checker swept every runtime global and element output through assert_outputs_contain, so verdict text echoed anywhere in the debug payload scored a pass. It now scopes to the named `weatherVerdict` output and requires exactly one of the two verdicts to appear — both-or-neither fails. - No output name was pinned, so no named assertion was possible. The prompt now declares the `weatherVerdict` out variable contract, and the criterion description matches the checker.
…nism The previous wording claimed "a solution binds exactly ONE Orchestrator folder" as platform law. Source verification disproved it. Multi-folder binding state is valid and expressible; what actually breaks is a CLI upsert bug, now filed as MST-13421. The real mechanism, verified at source: - Every connector manifest names its folder binding with the bare literal `FolderKey`, while `ConnectionId` is namespaced per connector (flow-workbench flow-core/src/manifest/mappers/connector.ts:186-195). - The resolver is strictly per-connection — exact `(name, resourceKey)` match, no fallback; on a miss the placeholder ships literal and warns (flow-schema/src/packaging/resolve-context.ts:83-90). - `node configure`'s fallback matches `(name, resource, propertyAttribute)` and ignores `resourceKey`, so it OVERWRITES the single `FolderKey` row — the second connection claims the first's row (cli flow-tool/src/services/connector-service.ts:360-399). So the same-folder advice survives as the simple path, not as law, and cross-folder gets the real remedy: `binding add` dedupes on the full `(name, resource, resourceKey, propertyAttribute)` quadruple and can hold the row `node configure` cannot. Also reconcile the § Top-level bindings shape and § Multi-connector sections, which asserted `node configure` emits two rows per connection. It does not; two rows is the healthy target state that `binding add` repairs toward.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
slack_weather_pipelinehas failed deterministically since #1030 deleted its folder hint: the tenant holds duplicateis-sandboxesSlack connections in two folders, agents bind the wrong-folder one, and the solution's single<bindings.FolderKey>resolves to the weather connection's folder — Slack lookup 404s with[102001]after a clean validate. Two runs fail identically on otherwise-correct agent work (2026-08-10:<bindings.FolderKey>unresolved on connectione03f734e, folder5da18ec0; 2026-08-06:[102001]onreadOfficeBellevueDescription1), both weighted 0.375.Two commits:
[102001]troubleshooting row inconnector/impl.md; one-sentence cross-linked qualifiers at the four other--all-foldersteaching sites + SKILL.md.--all-foldersstays a discovery aid; selection now has a rule.Shared/uipath-maestro-flow") + a named output contract (weatherVerdict, two exact values) + the debug success bar the 5.0-weight criterion always graded but the prompt never demanded. Checker: the all-globals substring sweep →assert_output_nonempty("weatherVerdict")+ exactly-one-of-two-verdicts (tolerant of wrapping/punctuation, fails both-or-neither), diagnostics via the module's own_fail_with_capture.Deliberately NOT changed:
slack_channel_descriptionstays unpinned (dormant in the nightly set; serves as the canary proving the skill rule alone steers agents), channel name→ID resolution stays the agent's work.Testing
_sharedpytest 68 green; skill-description hook green; all new relative links resolvecreate-debug-instanceendpoint is currently returning 500s (4 distinct traceIds over several hours) — will run when the backend recovers and the result posted here