Skip to content

fix(watch): re-accept agentforce-generate after upstream docs-only drift#65

Merged
askalf merged 1 commit into
masterfrom
fix/watch-reaccept-agentforce-generate
Jul 16, 2026
Merged

fix(watch): re-accept agentforce-generate after upstream docs-only drift#65
askalf merged 1 commit into
masterfrom
fix/watch-reaccept-agentforce-generate

Conversation

@askalf

@askalf askalf commented Jul 16, 2026

Copy link
Copy Markdown
Owner

What

Re-accepts agentforce-adlc:agentforce-generate in support/watch-accepted.json with the hash of its current bytes (551ada69…), after the 2026-07-13 weekly sweep re-flagged it and the public badge went red.

Why the re-flag happened (the system working as designed)

The skill was reviewed benign on 2026-07-09 (security-testing fixtures — Salesforce ships attack strings as test payloads) and accepted for exactly those bytes. Salesforce then updated the skill; the catalog pin moved f17012bdf7630726; the hash no longer matched, so the acceptance correctly lapsed and the finding resurfaced for human review.

Triage of the delta (hand-reviewed)

Only two files changed between the pins:

  • SKILL.md — an emoji removed from a warning, plus two sentences cross-referencing the new verification section.
  • references/agent-user-setup.md — new section 3b.4 'Verify the assignment stuck': read-only sf data query SOQL asserting the agent-user Id appears in PermissionSetAssignment / PermissionSetLicenseAssign, with a retry-once-then-STOP failure path; the old 3b.4 (Data Space scope UI fallback) renumbered to 3b.5.

No new instructions, no exfil shapes, no secrets handling — docs-only hardening of their own setup guide.

Verification

  • Old accepted tree (f17012bd) scans to the identical single finding (instruction-override, exit 1) — the trigger content is the already-reviewed fixture prose in files the delta never touched.
  • skillHash of the old tree reproduces the accepted 67403dd2… hash byte-for-byte (pipeline control).
  • skillHash of the current tree = 551ada69…, which is what this PR records; entry re-verified to match the current bytes via the same scan + skillHash path the watch uses.

After merge, a marketplace-watch dispatch should flip the badge back to green (0 poisoned · 13 accepted).

The 2026-07-13 sweep re-flagged agentforce-adlc:agentforce-generate (instruction-override) because its bytes drifted from the reviewed set - exactly what exact-bytes acceptance is for. Hand-review of the delta (old pin f17012bd vs current pin f7630726):

- SKILL.md: emoji removal + two cross-reference sentences pointing at the new verification section
- references/agent-user-setup.md: new section 3b.4 'verify the permset assignment stuck' - read-only SOQL queries (sf data query) asserting the agent-user Id appears in PermissionSetAssignment; old 3b.4 renumbered to 3b.5

The instruction-override trigger content is untouched by the delta: the old accepted bytes scan to the identical single finding (verified locally, exit 1 on both trees). Old-tree skillHash reproduces the accepted 67403dd2 hash exactly, proving the hash pipeline; new hash 551ada69 recorded with reviewed=2026-07-16 and a provenance note.

@sprayberry-reviewer sprayberry-reviewer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review from the Sprayberry Labs fleet code reviewer (sprayberry-reviewer bot).

Verdict: No blocking issues — approving.


What I checked

Diff surface: Single file, single commit — support/watch-accepted.json (+3 / −3 lines), no workflow or code changes.

The change:

-    "hash": "67403dd2058ba1d21cdd0918aa2728f745186ea092cae3f0d22d5e512e7f3a2c",
+    "hash": "551ada69fa206d78846aa758f14a603028a09925816d9cb9a7bb01bb30a8d348",
     "class": "security-testing fixtures",
-    "note": "Salesforce Agentforce docs ship attack strings as test payloads",
-    "reviewed": "2026-07-09"
+    "note": "Salesforce Agentforce docs ship attack strings as test payloads — re-accepted 2026-07-16 after upstream drift: delta was docs-only (permset-assignment verification steps, read-only SOQL); fixture content unchanged",
+    "reviewed": "2026-07-16"

(support/watch-accepted.json:3–6)

Runtime semantics verified: support/marketplace-watch.mjs:89 performs a.hash === skillHash(r.skill) — exact-bytes acceptance. The entry key (agentforce-adlc:agentforce-generate) and class field (security-testing fixtures) are unchanged; only the content hash and review provenance metadata are updated. A new upstream commit moving the hash from 67403dd2551ada69 correctly lapsed the prior acceptance, and this PR records the re-review result.

Hash format: The new hash 551ada69fa206d78846aa758f14a603028a09925816d9cb9a7bb01bb30a8d348 is 64 hex characters (SHA-256), consistent with all other entries in the file.

CI: All checks green on the PR head commit — 8 test-matrix jobs (Node 20/22 × ubuntu/macos/windows), CodeQL, and verify pinned skills (13s, pass). No workflow files were modified.

No findings to report. The update is minimal, follows the established re-acceptance pattern (exact-bytes semantics, provenance note, updated reviewed date), and is validated by the CI suite on the PR branch.


What's good

The enriched note field carries a clear audit trail of the delta triage (docs-only permset-assignment steps, read-only SOQL, fixture text unchanged) — exactly the right level of provenance for a security-sensitive allowlist entry.

@askalf
askalf merged commit 80affec into master Jul 16, 2026
9 checks passed
@askalf
askalf deleted the fix/watch-reaccept-agentforce-generate branch July 16, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants