Feature/bkt fitting - #16
Open
mjyang00001 wants to merge 6 commits into
Open
Conversation
Adds scripts for pulling student submissions, fitting Bayesian Knowledge Tracing models, and exploring knowledge components.
- fill_disqualified_kc.py: fill disqualified KCs with domain-average BKT params (subject-level, global fallback), with provenance + validation - evaluate_bkt.ipynb: KC quality tiering (param flags + RMSE/AUC baseline) - refit_merged_kc.py: refit merged typo-variant KCs
Both fit_bkt.py and fill_disqualified_kc.py now use indent=4, matching the committed experimentalBKTParams.json so re-runs don't reformat the whole file.
Rework fill_disqualified_kc.py into the full param-write pipeline: pull the current CAHLR/OATutor-Content remote, write fitted values for 76 donor KCs (Tier 3+4) and donor-average fills for 29 disqualified KCs (Tier 1+2), handle the 7 "_in_openstax_precalc" renames, and emit per-repo files for the external Chemistry/Calbright KCs. Outputs land in bkt/output/ (nothing edited in place); defaultBKTParams.json untouched.
Document the full BKT fitting/evaluation/fill/deploy pipeline in bkt/bkt_process.md (step-by-step, known pitfalls, current PR status), and add a condensed overview + pointer in the top-level README.
…perimental The OATutor content-staging cron regenerates defaultBKTParams.json on every run and previously blind-copied it to experimentalBKTParams.json, which would also collapse the fitted values from this repo. merge_experimental.py keeps default as the source of truth for the KC set (so new/renamed/removed KCs stay in sync) while applying this repo's fitted value per-KC where present.
mjyang00001
pushed a commit
to mjyang00001/OATutor-CalbrightContent
that referenced
this pull request
Jul 21, 2026
…erimental A straight cp from Tooling's committed output decouples experimental's KC set from default's — if default gains/renames/drops KCs, experimental goes stale until someone reruns the fitting pipeline. Use OATutor-Tooling's merge_experimental.py instead: default's KC set wins (stays in sync automatically), Tooling's fitted value wins per-KC where present. Must merge after CAHLR/OATutor-Tooling#16, which adds merge_experimental.py and bkt/output/experimentalBKTParams.json to Tooling's main branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.