Thanks for providing the code to generate UltraFeedback data. I tried running the files in comparison_data_generation folder. First of all, the bash script run_vllm.sh points to python script named main_vllm_batch.py which is not there. Secondly, I tried to run the main_vllm.py, it seems like the function instruction_completion, which actually completes the responses from the model is not used anywhere. Could you please provide a description of how to run these scripts: generating the data from models and then annotating?
Thanks!
Thanks for providing the code to generate UltraFeedback data. I tried running the files in comparison_data_generation folder. First of all, the bash script run_vllm.sh points to python script named main_vllm_batch.py which is not there. Secondly, I tried to run the main_vllm.py, it seems like the function instruction_completion, which actually completes the responses from the model is not used anywhere. Could you please provide a description of how to run these scripts: generating the data from models and then annotating?
Thanks!