Questions about training code for UltraRM/UltraCM

[halfrot]

Great Work! And thanks for the contribution. May I ask you if you have plans to release the training code for UltraRM/UltraCM?

[wenxueru]

+1，Any plan for this issue?

[lifan-yuan]

Thanks for your interest!

For reward modeling, we use code in this repo: https://github.com/Dahoas/reward-modeling
For critique modeling, we use the code in our sister repo: https://github.com/thunlp/UltraChat

I also recommend HuggingFace TRL for easy implementation: https://huggingface.co/docs/trl/index

[halfrot]

Thank you!
And do you have plans to open-source the out-of-box code for training? I'm somehow interested in continuing fine-tuning UltraRM for a domain-specific dataset so the detailed training code might help that.