This repository was archived by the owner on Jun 25, 2026. It is now read-only.
feat(agricogla): win-rate round scoring (1 for the winner, 0 otherwise), 50 episodes/round - #54
Closed
daveey wants to merge 1 commit into
Closed
feat(agricogla): win-rate round scoring (1 for the winner, 0 otherwise), 50 episodes/round#54daveey wants to merge 1 commit into
daveey wants to merge 1 commit into
Conversation
…e), 50 episodes/round (#53) Add a third `round_score` mode, `win`, alongside `mean`/`rank`: each episode's top scorer earns 1 and everyone else 0 (a tie for first shares the win), and a policy's round score is its win rate across the episodes it played. Generalizes `_rank_points_lists_by_policy` into `_episode_points_lists_by_policy(results, fn)` so rank and win share the aggregation; adds the score-kind tag, scoring-mechanics copy, and a tie-sharing test. Switch agricogla to `round_score: win` and `episodes: 50` (was rank / 100). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Landed on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two changes to how the agricogla Coworld league scores competition rounds, requested by daveey:
round_scoremode,win, alongsidemean/rank. Each episode's top scorer earns 1 and everyone else 0 (a tie for first shares the win), and a policy's round score is its win rate across the episodes it played — so the league rewards winning games outright, not placement or margin.winis not expressible by config alone (onlymean/rankexisted), so this adds the mode to theruleset_strategycommissioner and flipsconfigs/agricogla.yaml.Changes
common/utils.pyWIN_EPISODE_ROUND_SCORE_KIND+ mechanics copy;_episode_win_points(1 for the top score, ties shared); generalized_rank_points_lists_by_policy→_episode_points_lists_by_policy(results, fn)so rank & win share the per-policy aggregationcommon/ruleset_strategy/config.pyround_score: Literal["mean","rank","win"];round_score_kind&scoring_mechanicshandlewincommon/ruleset_strategy/commissioner.py_round_scores_by_policyroutes rank/win through the shared aggregator; falls back to super only formeanconfigs/agricogla.yamlround_score: win,episodes: 50(was rank / 100) + headertests/test_commissioner_strategies.pytest_ruleset_strategy_win_round_score_uses_binary_win_points, incl. a tie-for-first case (both tied seats win)Tie semantics
Top-scorer(s) get 1; a tie for first means all top scorers get 1 (consistent with agricogla's "ties share the win" rule and with
rank-mode tie handling).Verification
tests/test_commissioner_strategies.py— 72 passed locally.episodes=50,round_score=win,kind=win_episode_round_score.Deploy follow-up (after merge)
ghcr.io/metta-ai/commissioners-agricogla:latest+:coworld-<sha>and prints the new@sha256:digest.packages/cogweb/games/agricogla/coworld/compose.yaml(and fix its stale… episodes/round … rankcomment).coworld build → certify → upload-coworld).🤖 Generated with Claude Code