Add Ohio State University mirror (port 40020) with deterministic verifiers - #63
Merged
Conversation
Integrate the osu mirror from PR #12 onto the current main as an additive 17th site instead of the PR's stale base (which replaced merriam_webster and reused port 40015). Registered in all three places (websyn_start.sh, control_server.py, Dockerfile EXPOSE 40000-40016); the readiness gate in websyn_start.sh is now derived from ${#SITES[@]} so it no longer hardcodes the site count. osu ships no frozen DB/images in git or HF: its data is generated from seed_data.py, so a Dockerfile build step creates instance_seed/osu.db so the boot reset and /reset/osu have a seed to restore from. Verified: image builds, osu serves on 40016, and /reset byte-identity holds.
Reviewer role for PR #12 (previously unreviewed). One verify_N.py per task over a shared verify_lib.py, mirroring sites/merriam_webster/verify: deterministic-first (navigation anti-shortcut + hardcoded on-site ground truth) with an anchored llm_screenshot_shows confirmation that is skipped under --no_llm. tasks.jsonl gains verifier_path + judge_rubric per row (ground truth stays out of the agent-facing file). selfcheck.py proves all 20 verifiers PASS a correct trajectory and FAIL a wrong-answer / no-navigation one (20/20). TASK_REVIEW.md records feasibility: all 20 tasks are on-site answerable; 6 are flagged as answerable-from-prior-knowledge (mitigated by the nav check) and task 1 has a 36-vs-27 varsity-sports inconsistency.
Raibows
changed the title
Contributor
|
Thanks for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Ohio State University mirror as site 21 on port 40020
This branch now includes the original OSU mirror and grading contract, current
mainthrough TED, and the review remediation for PR #63.Integration
40020.websyn_start.sh,control_server.py,Dockerfile, contributor documentation, and task URLs use the same 21-site order and40000-40020range.sites/osu/.build-generated-seedmarks its deterministic source-generated database. The pinned Hugging Face asset revision contains the requiredosu.tar.gzofficial-image bundle, enforced bysites/osu/.requires-images,scripts/build.sh,scripts/check_assets.sh, and the Docker build.Review remediation
sites/osu/image_sources.jsonrecords source pages, source URLs, dimensions, alt text, and source/output SHA-256 values;sites/osu/fetch_images.pyreproduces the WebP bundle.Validation
db9c73e62d853ed91f6152b5b7105571502b7e07containsosu.tar.gz./reset/osuand/reset-allpass.The consolidated audit is in
review-reports/PR-63-FINAL-AUDIT.md.