Hello,
Kindly asking your input about this issue vllm-project/vllm#38266 (comment): it seems that the Rust binding is blocking the whole context execution although it shouldn't in such case:
trying to make async a call to render_conversation_for_completion with run_in_executor
Hello,
Kindly asking your input about this issue vllm-project/vllm#38266 (comment): it seems that the Rust binding is blocking the whole context execution although it shouldn't in such case:
trying to make async a call to
render_conversation_for_completionwith run_in_executor