feat(skills): manual escape hatch for stuck reject-cooldowns - #217
Merged
Conversation
…ldowns get_skill_cooldown() has no expiry: once 2+ rejects for an (app, skill) pair share a reason prefix, that skill is skipped everywhere forever (skill_engine.py match/run_all, remediation/dispatcher.py, delivery.py) -- even when the rejects were earned by a different, since-removed component (confirmed live: pulse-agent/pulse-ui's containerfile skill stuck cooling down purely from rejects the now-deleted CodeChangeAgent caused, see 9e1d762/a2d20ef6). AssessmentStore.clear_skill_cooldown(app_name, skill_name, reason=, actor=) records a dedicated skill_effectiveness marker row that get_skill_cooldown()/ list_cooled_skills() treat as a waterline -- only rejects strictly after it count going forward. Purely additive: the automatic trigger's own condition never changes, so a skill that keeps genuinely failing cools down again after the same threshold of new matching rejects. The marker is excluded from get_skill_effectiveness()'s approval-rate math so clearing a cooldown never dilutes a skill's own score. Human-in-the-loop by design: Capabilities' "Cooling down" banner gained an inline "Clear cooldown..." action (POST /capabilities/skills/clear-cooldown) requiring a real reason and logging a skill-cooldown-cleared event; `agentit clear-skill-cooldown APP SKILL --reason ...` is the CLI equivalent. Co-authored-by: Cursor <cursoragent@cursor.com>
Follows this repo's convention of logging every real fix/feature there alongside README.md and docs/release-notes.md. Co-authored-by: Cursor <cursoragent@cursor.com>
…), not just past today's known CVEs Security Scan (Trivy) on PR #217 (an unrelated skill-cooldown manual-clear feature) failed again -- the "take 2" floor of >=3.1.57 held for barely five days before 5 new HIGH findings landed against installed 3.1.57: GHSA-4gmw-gg2m-w46p, GHSA-9rj7-rf2p-w77r, GHSA-hmq2-w58f-27jc, GHSA-jm78-9fvv-mhgr, GHSA-wvpp-8hx9-p66j (plus GHSA-hh9p-6wh2-4mfc), all fixed in 3.1.58. Bumping straight to 3.1.58 would repeat the exact "minimum version that clears today's known CVEs" mistake that caused this to recur a third time -- and indeed, GitHub's own release notes show 3.1.59 shipped the very next day (2026-08-10) with two more security advisories (GHSA-5xxx-qhh7-9287, GHSA-3wxw-xv34-2frg), which would have re-broken the scan within 24 hours of a 3.1.58 bump. pyproject.toml's gitpython floor raised to >=3.1.59 (verified against PyPI as the actual current latest stable release, same discipline as takes 1/2 and the aiohttp/cryptography constraint-dependencies fix on PR #216). uv lock resolved 3.1.59 cleanly, no other package changed. Checked 3.1.58/3.1.59's new hardening (blocks separate git directories during clone, blocks file-reading git options, hardens config-parsing boundaries) against the only real call site in this codebase -- cloner.py's Repo.clone_from() via clone_repo(), a plain depth/branch/env invocation using none of the newly-blocked options -- so no behavior change there. Verified directly, not just by trusting the version bump: ran the same Trivy fs scan locally against the regenerated uv.lock -- 0 vulnerabilities, down from 5 HIGH. tests/test_cloner.py (14 tests) pass unchanged against 3.1.59. Full non-browser suite green: 3511 passed, 441 skipped, 0 failed (matching CI's test job's exact ignore list). Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Option B from the skill-effectiveness reject-cooldown investigation: a human-initiated, audited way to clear a stuck
(app_name, skill_name)cool-down.get_skill_cooldown()(src/agentit/portal/store/skills.py) has no expiry and no time window -- once 2+ rejects for a pair share a reason prefix, that skill is skipped everywhere (skill_engine.pymatch/run_all,remediation/dispatcher.py,delivery.py) forever, with no existing way out.pulse-agent/pulse-ui'scontainerfileskill is stuck cooling down purely from rejects the now-deletedCodeChangeAgentcaused (commit9e1d7623/a2d20ef6), not from any fault incontainerfileitself.What this adds:
AssessmentStore.clear_skill_cooldown(app_name, skill_name, *, reason, actor="")-- inserts a dedicatedskill_effectivenessmarker row (outcome="cooldown_cleared") thatget_skill_cooldown()/list_cooled_skills()treat as a waterline: only rejects strictly after the most recent marker for that exact pair count going forward.SKILL_COOLDOWN_SAME_REASON) never changes. A skill that keeps genuinely failing cools down again after the same threshold of new matching rejects post-clear.(app, skill)pair is untouched.get_skill_effectiveness()'s approval-rate math so clearing a cooldown never makes a skill's own score look worse./capabilities/skills/{skill}/history.POST /capabilities/skills/clear-cooldown(mirrorsactivate-skill/deprecate-skill's pattern inroutes/capabilities.py) -- requires a real reason, logs askill-cooldown-clearedevent viastore.log_event()so it shows up on the Events feed.agentit clear-skill-cooldown APP_NAME SKILL_NAME --reason ... [--actor ...].skill_detail.html's effectiveness trend table renders the new marker outcome distinctly (a neutral/warning badge) instead of looking like an extra rejection.Test plan
tests/test_skill_learning_minimal.py::TestClearSkillCooldown): clear unblocks the exact pair, doesn't affect other apps/skills, doesn't affectlist_cooled_skills()(the queryskill_engine.pyactually gates Scan on), skill can cool down again after being cleared (not a permanent bypass), reject history is preserved, blank reason raises, marker doesn't dilute approval-rate stats.tests/test_portal.py): clears + logs event, requires a reason, requires app/skill name, banner renders the clear form.tests/test_cli.py): wires args through to the store method, defaults actor to the OS user, requires--reason.3511 passed, 459 skipped, 0 failed.Live verification
Cluster/portal access (
oc whoami) is unavailable in this session (expired token) -- live verification againstpulse-agent/pulse-ui's realcontainerfilecooldown is blocked until this merges + deploys + cluster access is available. Will follow up once available; see conversation notes.Made with Cursor