Dear Dr. Zhang:
Thank you for your work and insights—they have deeply inspired me.
Therefore, I plan to modify EMPO's code to validate my hypothesis. However, since vLLM cannot return the hidden states of the last layer, I intend to use sglang for resampling. In other words, modify the underlying logic of actor_rollout_wg.generate_sequences in ray_trainer.py to switch from using vllm_rollout_spmd.py to using sglang_rollout.py. Have you tried this approach? If so, I would appreciate it if you could share your experience.
Wishing you all the best.
Breakices
尊敬的张博士：
感谢您的工作和想法——他们深深的启发了我。
因此我也想在EMPO的代码基础上做修改，来验证一些想法。然而，由于vLLM不能返回最后一层的隐藏状态/embedding, 我希望使用sglang来做actor阶段的重采样。也就是更换actor_rollout_wg.generate_sequences的底层推理框架，不使用vllm_rollout_spmd.py的generate方法，而是用sglang_rollout.py来推理。不知道您是否尝试过这个修改，如果您尝试过的话，希望您能传授一下经验。
祝一切顺利，
breakices
Dear Dr. Zhang:
Thank you for your work and insights—they have deeply inspired me.
Therefore, I plan to modify EMPO's code to validate my hypothesis. However, since vLLM cannot return the hidden states of the last layer, I intend to use sglang for resampling. In other words, modify the underlying logic of
actor_rollout_wg.generate_sequencesinray_trainer.pyto switch from usingvllm_rollout_spmd.pyto usingsglang_rollout.py. Have you tried this approach? If so, I would appreciate it if you could share your experience.Wishing you all the best.
Breakices
尊敬的张博士：
感谢您的工作和想法——他们深深的启发了我。
因此我也想在EMPO的代码基础上做修改，来验证一些想法。然而，由于vLLM不能返回最后一层的隐藏状态/embedding, 我希望使用sglang来做actor阶段的重采样。也就是更换actor_rollout_wg.generate_sequences的底层推理框架，不使用vllm_rollout_spmd.py的generate方法，而是用sglang_rollout.py来推理。不知道您是否尝试过这个修改，如果您尝试过的话，希望您能传授一下经验。
祝一切顺利，
breakices