Skip to content

Guard JudgmentKit deck skill discoverability #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mikeylong merged 1 commit into from
Jul 7, 2026
Merged

Guard JudgmentKit deck skill discoverability #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions scripts/verify-codex-plugin.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,24 @@ export const REQUIRED_DESIGN_SYSTEM_GATE_WORDING = [
"If the active design-system review fails, the next action is repair or regeneration against the MCP-returned constraints, not acceptance with caveats.",
];

export const REQUIRED_SLIDE_DECK_WORKFLOW_WORDING = [
"Use this skill for JudgmentKit slide deck requests",
"## Slide Deck Creation",
"## Deck Workflow",
"deck_creation_status",
"mcp__judgmentkit.create_slide_deck",
"If no deck creation tool is listed by the active endpoint",
"Do not fabricate a JudgmentKit packet, deck, or MCP result",
"explicit `template_id` values or strong selection metadata",
];

export const REQUIRED_AGENT_DECK_DISCOVERY_WORDING = [
"slide deck creation",
"create a JudgmentKit slide deck",
"presentation, PowerPoint, or PPTX",
"deck creation tool is available",
];

const CODEX_SUFFIX_PATTERN = new RegExp(`\\+${"codex"}\\b`, "i");
const LOCAL_HOME_PATTERN = new RegExp(`/Users/${"mike"}\\b`);
const CODEX_CACHE_PATTERN = new RegExp(`\\.codex/${"plugins"}/cache`);
Expand Down Expand Up @@ -214,6 +232,16 @@ function collectMissingWordingFailures(skillText) {
}
}

for (const phrase of REQUIRED_SLIDE_DECK_WORKFLOW_WORDING) {
if (!normalizedSkillText.includes(normalizeText(phrase))) {
failures.push({
check: "slide_deck_workflow_wording",
file: "skills/judgmentkit-hosted-mcp/SKILL.md",
message: `Missing required slide-deck workflow wording from hosted MCP skill: ${phrase}`,
});
}
}

return failures;
}

Expand Down Expand Up @@ -310,6 +338,16 @@ function collectAgentYamlFailures(agentYamlText, mcpUrl) {
});
}

for (const phrase of REQUIRED_AGENT_DECK_DISCOVERY_WORDING) {
if (!normalizedAgentYaml.includes(normalizeText(phrase))) {
failures.push({
check: "agent_deck_discovery",
file: "skills/judgmentkit-hosted-mcp/agents/openai.yaml",
message: `OpenAI agent metadata must keep slide-deck discovery wording: ${phrase}`,
});
}
}

return failures;
}

Expand Down
58 changes: 56 additions & 2 deletions tests/codex-plugin.test.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,16 @@ function validSkillText() {
"",
"# JudgmentKit Hosted MCP",
"",
"Use this skill for JudgmentKit slide deck requests when the user asks to create, draft, generate, export, or turn source material into a deck, presentation, PowerPoint, or PPTX.",

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require deck terms in skill frontmatter

For Codex skill selection, the discoverable skill summary comes from the SKILL.md frontmatter description, but this new happy-path fixture adds deck/PPTX wording only in the body while leaving the fixture's description: UI-only. Since collectMissingWordingFailures() searches the whole file, deleting the slide-deck terms from the real frontmatter would still pass codex-plugin:check as long as these body snippets remained, so the new guard does not catch the discoverability regression it is meant to prevent. Please parse and check the frontmatter description, or include a fixture that fails when only that metadata goes stale.

Useful? React with 👍 / 👎.

"",
"## Slide Deck Creation",
"",
"Treat dry-run deck planning as JudgmentKit guidance when the MCP response uses `schema: \"judgmentkit.mcp.slide-deck/v1\"` and `deck_creation_status: \"planned\"`.",
"",
"Treat an exported PPTX as JudgmentKit output only when the MCP response or sidecar receipt confirms `tool_name: \"mcp__judgmentkit.create_slide_deck\"`, `deck_creation_status: \"exported\"`, and matching `sha256`, `bytes`, and `mime_type` artifact fields.",
"",
"If no deck creation tool is listed by the active endpoint, state that the current JudgmentKit endpoint cannot create the deck yet. Do not fabricate a JudgmentKit packet, deck, or MCP result.",
"",
"## Design-System Acceptance Gate",
"",
"A generated UI that does not pass the active design system is not an artifact. It is a failed candidate.",
Expand All @@ -127,15 +137,19 @@ function validSkillText() {
"",
"For unreleased designs, proprietary design-system details, source code, customer data, or internal roadmaps, prefer a local checkout, local stdio server, or self-hosted JudgmentKit MCP endpoint instead of `https://judgmentkit.ai/mcp`.",
"",
"## Deck Workflow",
"",
"For portfolio or case-study decks, pass explicit `template_id` values or strong selection metadata when layout variety matters; heed layout repetition warnings.",
"",
].join("\n");
}

function validAgentYaml() {
return [
"interface:",
' display_name: "JudgmentKit Hosted MCP"',
' short_description: "UI workflow review and hard design-system gates"',
' default_prompt: "Use $judgmentkit-hosted-mcp to review this UI brief for workflow fit, surface choice, disclosure boundaries, and the hard design-system acceptance gate."',
' short_description: "UI workflow review, slide deck creation, and hard design-system gates"',
' default_prompt: "Use $judgmentkit-hosted-mcp to review this UI brief for workflow fit, surface choice, disclosure boundaries, and the hard design-system acceptance gate, or to create a JudgmentKit slide deck, presentation, PowerPoint, or PPTX from allowed source material when the deck creation tool is available."',
"dependencies:",
" tools:",
' - type: "mcp"',
Expand Down Expand Up @@ -310,6 +324,46 @@ async function assertVerifyRejects(options, expectedChecks) {
);
}

{
const fixture = await createFixture({
skillText: validSkillText().replace(
/## Slide Deck Creation[\s\S]*?## Design-System Acceptance Gate/,
"## Design-System Acceptance Gate",
).replace(
/## Deck Workflow[\s\S]*$/,
"",
),
});
await assertVerifyRejects(
{
sourceDir: fixture.sourceDir,
packageJsonPath: fixture.packageJsonPath,
},
["slide_deck_workflow_wording"],
);
}

{
const fixture = await createFixture({
agentYaml: validAgentYaml()
.replace(
' short_description: "UI workflow review, slide deck creation, and hard design-system gates"',
' short_description: "UI workflow review and hard design-system gates"',
)
.replace(
' default_prompt: "Use $judgmentkit-hosted-mcp to review this UI brief for workflow fit, surface choice, disclosure boundaries, and the hard design-system acceptance gate, or to create a JudgmentKit slide deck, presentation, PowerPoint, or PPTX from allowed source material when the deck creation tool is available."',
' default_prompt: "Use $judgmentkit-hosted-mcp to review this UI brief for workflow fit, surface choice, disclosure boundaries, and the hard design-system acceptance gate."',
),
});
await assertVerifyRejects(
{
sourceDir: fixture.sourceDir,
packageJsonPath: fixture.packageJsonPath,
},
["agent_deck_discovery"],
);
}

{
const fixture = await createFixture({
extraFiles: {
Expand Down
Loading