[POC] Introducing Code Review category

[haoranpb]

A POC showing how Code Review category can be implemented.

Try it out locally:

# See the placeholder task in the dataset
uv run bcbench dataset view microsoft__BCApps-4699 --category code-review

# See the default copilot review live
uv run bcbench -v evaluate copilot microsoft__BCApps-4699 --category code-review --repo-path C:\depot\BCApps

Getting Started

Code Review team to implement

• Replace the dataset, see codereview.jsonl
• Implement the scoring and evaluation metrics calculation, we currently have placeholder metrics: count of comments
• Add the AL Code Review skill

Leaderboard related logic is intentionally left unimplemented for now.

[haoranpb]

Solid progress.

One thing we should discuss: do we want this synthatic dataset? Or do we want to invest in real-world production PRs?

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.