integrity: reject fabricated training_log (forged-down wall_clock) - #93
Merged
Merged
Conversation
- postmortem of taohunter v0.3.3: forged wall_clock_s to hit exactly 50% MFU
(under the 70% ceiling) + shipped a perfectly-linear log (step-0 elapsed 0.0)
agreeing with it, keeping normalized-H100h under the ~5 cap. op3 only checked
monotonic/finite; nothing cross-checked the log against itself or wall_clock_s.
- add check_training_log_integrity (FP-safe: identities + a physical floor, never a
variance threshold that would flag a legit torch.compile run):
(1) step-0 compile/init floor (real first step ~6-8s; fake ~0)
(2) tokens_per_sec == tokens_seen/elapsed_s identity (majority)
(3) final log elapsed == final_state.wall_clock_s
degenerate perfectly-linear timing is warn/audit-only.
- wired fail-closed into op1 after check_compute_plausibility; RALPH_LOG_INTEGRITY_OFF=1 disables.
- validated on real data: rejects taohunter's log, accepts both real king logs; 7 tests.
- NOTE: closes the fabricated-log vector; wall_clock_s stays forgeable without
checkpoint-bound re-derivation on canonical data (plan P4).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
karpabot
force-pushed
the
fix/training-log-integrity
branch
from
July 3, 2026 10:43
5cf00f8 to
9608543
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Postmortem fix for the taohunter v0.3.3 crown. It forged wall_clock_s to land at exactly 50% MFU (under the 70% ceiling) and shipped a perfectly-linear training_log (step-0 elapsed 0.0) that agreed with the forged wall, keeping normalized-H100h under the cap. Nothing cross-checked the log against itself or wall_clock_s.
Adds
check_training_log_integrity— FP-safe (arithmetic identities + a physical step-0 compile floor, not a variance threshold that would flag legit torch.compile runs): step-0 floor + tokens_per_sec==tokens_seen/elapsed_s identity + final-elapsed==wall_clock_s. Perfectly-linear timing is warn/audit-only. Wired fail-closed into op1 (RALPH_LOG_INTEGRITY_OFF=1disables). Validated on the real taohunter log (rejected) + both real king logs (accepted); 8 tests incl. evasion variants.Closes the fabricated-log vector; wall_clock_s stays forgeable until checkpoint-bound re-derivation (P4).
🤖 Generated with Claude Code