test(scoring): defuse the repo_quality recency time bomb that turned main red - #60
Merged
Merged
Conversation
main went red on 2026-07-31 with no code change behind it. repo_quality awards recent_activity (6 pts) for a non-fork commit inside a rolling 90-day window measured from datetime.now(UTC), but the fixtures carry hardcoded last_commit_at values. profile_oss.json's newest commit (2026-05-01) turned 91 days old overnight, so the 20-point expectation became 14. profile_senior.json (2026-05-10) was 8 days from the same fate. Pin the recency-sensitive dates relative to now, the convention test_consistency.py and test_learning_trajectory.py already use for the other time-windowed scorers, so these assert the scoring rule rather than the calendar. Both directions are now stated explicitly: a new test covers recent_activity NOT firing once every commit is stale, which the old suite only ever asserted by accident of the fixture's age.
Shaan-alpha
added a commit
that referenced
this pull request
Jul 31, 2026
All CHANGELOG/PROGRESS_LOG entries for the three open PRs are consolidated here so the branches do not conflict on these two files. Records the merge order (#60 first -- it is what makes CI green) and which audit findings remain unfixed.
This was referenced Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mainwent red on 2026-07-31 with nobody pushing anything. Found while investigating why #59's backend check failed — it reproduces identically onmain, so it is not caused by that PR.Root cause
repo_quality.score()awardsrecent_activity(6 pts) for a non-fork commit inside a rolling 90-day window measured fromdatetime.now(UTC)(app/scoring/repo_quality.py:48). The fixtures carry hardcodedlast_commit_atvalues.profile_oss.json's newest commit is2026-05-01— exactly 91 days before today. The window lapsed overnight, the signal stopped firing, andtest_oss_profile_scores_twenty_without_deployment_hintwentassert 14 == 20.profile_senior.json's newest is2026-05-10, 82 days out.test_senior_profile_scores_twenty_six_until_license_signal_landswas 8 days from the same failure (2026-08-08).Fix
Pin the recency-sensitive dates relative to
nowin the test, which is the conventiontest_consistency.pyandtest_learning_trajectory.pyalready use for the other time-windowed scorers — they load the same static fixtures and then override the time-sensitive fields.test_repo_quality.pywas the one file relying on the baked-in dates. No production code and no fixture files change, and no new dependency.Also adds
test_recent_activity_does_not_fire_once_every_commit_is_stale. The old suite only asserted the negative direction by accident of how old a fixture happened to be; now both directions are stated.Scope
Checked the other fixture-driven scoring tests before settling on this.
test_engine.pyasserts invariants (report.total == report.breakdown.total()) rather than absolute scores, so it rides out the drift;test_badges.py,test_consistency.py,test_depth.pyandtest_learning_trajectory.pyalready build their time-sensitive state relative tonow.test_repo_quality.pywas the only exposed file.Four scorers call
datetime.now(UTC)internally (badges,consistency,learning_trajectory,repo_quality) and none take an injectable clock. Threading one through would be the deeper fix and would let these tests pin an exact instant instead of working relative to a moving one — deliberately not done here, since this PR exists to getmaingreen and that refactor touches production signatures across all four.Verification
uv run pytest -q→ 360 passed, 72 skipped (was 358 passed / 1 failed, plus the 1 new test).ruff checkclean,ruff format --checkclean on 164 files.