Remove unused skill evaluation harness#7
Conversation
There was a problem hiding this comment.
Code Review
This pull request removes the custom cross-provider evaluation harness (scripts/run_skill_eval.py), its test fixture (tests/fixtures/fake_eval_runner.py), and its unit tests (tests/test_run_skill_eval.py). It updates the documentation in README.md, README.ko.md, AGENTS.md, and CONTRIBUTING.md to remove references to this harness and the reports/ directory, advising users to perform evaluations manually or using official vendor tools instead. Additionally, test files and validation/experiment-fairness.md are updated to align with these simplified guidelines. There are no review comments provided, so I have no feedback to offer.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Summary
reports/so fresh checkouts do not require that directoryWhy
The repository ships a backend development skill, while the harness had no real vendor adapter and could only prove its own protocol. Keeping it added about 2,400 lines of maintenance code without measuring skill quality or runtime token savings.
Impact
The installable
skills/boring-backend/package and its mirrors are unchanged. Runtime behavior and context size are unaffected. Repository maintenance is smaller, and evaluation claims still require real vendor telemetry.Verification
python scripts/verify_all.pyon Windows Python 3.13: 53 tests passedpython3 scripts/verify_all.pyon WSL Linux Python 3.14: 53 tests passedreports/: 53 tests passedskill-creatorpackage validation passed