Add counterfactual dataset#615

Draft

Jiawen-CS wants to merge 48 commits into

mainfrom

thesis/counterfactual-dataset

Commits on Apr 14, 2026

Add counterfactual dataset (255 entries + problem statements)
Jiawen Sun
committed
Add counterfactual experiment notebooks (aligned with main, deps stubbed)
Jiawen Sun
committed
Add counterfactual-evaluation category
Jiawen Sun
committed
Add analysis package: family models, aggregator, metrics, annotation
Jiawen Sun
committed

Commits on Apr 16, 2026

Merge branch 'main' of https://github.com/microsoft/BC-Bench into thesis/counterfactual-dataset
Jiawen Sun
committed
Split counterfactual-evaluation into per-variant categories (cf-1..cf-4)
Jiawen Sun
committed
Add cf-1..cf-4 to workflow category dropdowns
Jiawen Sun
committed
Auto-detect counterfactual dataset path from InstanceId
Jiawen Sun
committed

Commits on Apr 18, 2026

Merge branch 'main' of https://github.com/microsoft/BC-Bench into thesis/counterfactual-dataset
Jiawen Sun
committed
Fix load base intances problem
Jiawen Sun
committed
fix matrix issue
Jiawen Sun
committed
fix container problems
Jiawen Sun
committed
remove low quality dataset
Jiawen Sun
committed
change validation action for dataset
Jiawen Sun
committed
fix patch failed issue
Jiawen Sun
committed

Commits on Apr 20, 2026

update cf3 and cf4 datasets
Jiawen Sun
committed
remove images
Jiawen Sun
committed
Merge branch 'main' into thesis/counterfactual-dataset
Jiawen-CS
authored
fix cf1 and cf2
Jiawen Sun
committed
Merge branch 'thesis/counterfactual-dataset' of https://github.com/microsoft/BC-Bench into thesis/counterfactual-dataset
Jiawen Sun
committed
impovement for cf3
Jiawen Sun
committed
update dataset
Jiawen Sun
committed
./
Jiawen Sun
committed

Commits on Apr 21, 2026

Update cf2
Jiawen Sun
committed
./cf2
Jiawen Sun
committed
update dataset
Jiawen Sun
committed

Commits on Apr 24, 2026

fix: resolve CF COMPILE failures from run 24733495613

Jiawen Sun
and
Copilot
committed
Fix 3 failing CF-3 counterfactual entries

Jiawen Sun
and
Copilot
committed
fix(dataset): redesign 4 TEST_3b CF entries with real semantic mutations

Jiawen Sun
and
Copilot
committed
update cf2 issue
Jiawen Sun
committed
Merge branch 'main' of https://github.com/microsoft/BC-Bench into thesis/counterfactual-dataset
Jiawen Sun
committed
fix(dataset): auto-fix 19 CF PATCH hunk headers (run 24890066865)

Jiawen Sun
and
Copilot
committed
Fix 14 CF test failures (11 TEST_3a + 3 TEST_3b)

Jiawen Sun
and
Copilot
committed
Fix 7 failing CF-3 counterfactual entries

Jiawen Sun
and
Copilot
committed
fix(dataset): fix 22 CF-1 entries with COMPILE/TEST/PATCH failures

Jiawen Sun
and
Copilot
committed

Commits on Apr 25, 2026

fix(dataset): combined cf-1 fixes + auto-fix 6 cf-2 PATCH hunk headers

Jiawen Sun
and
Copilot
committed
fix(cf-2): fix 14 COMPILE failures in counterfactual dataset

Jiawen Sun
and
Copilot
committed
fix(cf-2): fix 13 TEST_3a failures in counterfactual dataset

Jiawen Sun
and
Copilot
committed
fix(dataset): convert 6 cf-2 entries with list-typed patch/test_patch to strings

Jiawen Sun
and
Copilot
committed

Commits on Apr 26, 2026

fix(dataset): fix 10 CF entries with corrupt/mismatched patches

Jiawen Sun
and
Copilot
committed

Commits on Apr 27, 2026

Merge branch 'main' of https://github.com/microsoft/BC-Bench into thesis/counterfactual-dataset
Jiawen Sun
committed

Commits on Apr 28, 2026

fix(dataset): remove 82 failed CF entries and renumber variants

Jiawen Sun
and
Copilot
committed
Merge branch 'main' of https://github.com/microsoft/BC-Bench into thesis/counterfactual-dataset
Jiawen Sun
committed

Commits on May 1, 2026

Merge branch 'main' of https://github.com/microsoft/BC-Bench into thesis/counterfactual-dataset
Jiawen Sun
committed
remove problem_statement_override
Jiawen Sun
committed

Commits on May 3, 2026