Reproduced results

[fiee574]

While reproducing EMPO using the trl library, I noticed the results don't match the data reported in the paper. Could you please provide the exact dataset used in the experiments? e.g.Original dataset source and version. Thank you!

[QingyangZhang]

Sure. For math reasoning tasks, we use this dataset. And this dataset for natural reasoning.

Would you mind sharing more details of training? Happy to help reproducing the results.

By the way, I wound recommend to use verl for further development, which is much more efficient.

[fiee574]

​Could you provide detailed information about the datasets referenced in the paper for the results (math, Minerva Math, Olympiad Bench, AMIE24, AMC23)? Relevant details about these datasets are not mentioned in the code.​​

[QingyangZhang]

The evaluation datasets and script are the same with SimpleRL. You can find the json files in here. We only modify the chat template to be consistent with Online-DPO-R1.

"qwen25-math-cot": (
    "<|im_start|>system\nPlease reason step by step, and put your final answer within \\boxed{{}}.<|im_end|>\n"
    "<|im_start|>user\n{input} Let's think step by step and put the final answer within \\boxed{{}}.<|im_end|>\n"
    "<|im_start|>assistant\n",
    "{output}",
    "\n\n",
),