SamitHuang · SamitHuang · Mar 2, 2026 · Mar 3, 2026 · Mar 3, 2026 · Mar 4, 2026
diff --git a/convert_qwen2.5_ckpt.sh b/convert_qwen2.5_ckpt.sh
@@ -0,0 +1,5 @@
+source scripts/models/qwen2.5-0.5B.sh
+PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
+    ${MODEL_ARGS[@]} \
+    --hf-checkpoint /root/Qwen2.5-0.5B-Instruct \
+    --save /root/Qwen2.5-0.5B-Instruct_torch_dist/
-PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
-    ${MODEL_ARGS[@]} \
-    --hf-checkpoint /root/Qwen2.5-0.5B-Instruct \
-    --save /root/Qwen2.5-0.5B-Instruct_torch_dist/
+export WORK_DIR=/root
+export PYTHONPATH=${WORK_DIR}/Megatron-LM
+
+HF_CHECKPOINT_PATH=${WORK_DIR}/Qwen2.5-0.5B-Instruct
+SAVE_PATH=${WORK_DIR}/Qwen2.5-0.5B-Instruct_torch_dist/
+
+python tools/convert_hf_to_torch_dist.py \
+    ${MODEL_ARGS[@]} \
+    --hf-checkpoint ${HF_CHECKPOINT_PATH} \
+    --save ${SAVE_PATH}
-PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
-    ${MODEL_ARGS[@]} \
-    --hf-checkpoint /root/Qwen2.5-0.5B-Instruct \
-    --save /root/Qwen2.5-0.5B-Instruct_torch_dist/
+export WORK_DIR=/root
+export PYTHONPATH=${WORK_DIR}/Megatron-LM
+
+HF_CHECKPOINT_PATH=${WORK_DIR}/Qwen2.5-0.5B-Instruct
+SAVE_PATH=${WORK_DIR}/Qwen2.5-0.5B-Instruct_torch_dist/
+
+python tools/convert_hf_to_torch_dist.py \
+    ${MODEL_ARGS[@]} \
+    --hf-checkpoint ${HF_CHECKPOINT_PATH} \
+    --save ${SAVE_PATH}
diff --git a/docs/en/advanced/rfc-vllm-rollout-backend.md b/docs/en/advanced/rfc-vllm-rollout-backend.md
diff --git a/docs/en/examples/deepseek-r1.md b/docs/en/examples/deepseek-r1.md
@@ -171,7 +171,7 @@ OPTIMIZER_ARGS=(
 
 These are the parameters required by sglang. Here, `--rollout-num-gpus-per-engine` basically corresponds to sglang's `tp_size`. Other sglang parameters are passed to slime by adding a `--sglang-` prefix. To fully leverage sglang's large EP inference capabilities, we have added configurations like ep64, dp\_attention dp8, and deepep mode auto.
 
-The final `--sglang-server-concurrency` is a parameter specific to slime. It is used to prevent the sglang server's concurrent requests from becoming too large and crashing the HTTP server. The default is 512. However, since we now have one server for 8 nodes, we have adjusted it to 1024 to ensure that each dp rank can have a concurrency of 128.
+The final `--rollout-server-concurrency` is a parameter specific to slime. It is used to prevent the sglang server's concurrent requests from becoming too large and crashing the HTTP server. The default is 512. However, since we now have one server for 8 nodes, we have adjusted it to 1024 to ensure that each dp rank can have a concurrency of 128.
 
 ```bash
 SGLANG_ARGS=(
@@ -190,7 +190,7 @@ SGLANG_ARGS=(
    --sglang-deepep-mode auto
 
    # make every dp rank have 128 concurrency
-   --sglang-server-concurrency 1024
+   --rollout-server-concurrency 1024
 )
 ```
 

diff --git a/docs/en/examples/qwen3-4B.md b/docs/en/examples/qwen3-4B.md
@@ -280,10 +280,10 @@ In this case, 2 GPUs will be allocated for training, and 6 GPUs will be allocate
 
 ⚠️  If the concurrency on each sglang server is too high, it may exceed sglang's default CUDA graph concurrency limit (the default maximum is 160), which will affect inference speed. You can adjust this in the following two ways:
 
-1.  Use `--sglang-server-concurrency` to limit the maximum number of concurrent requests sent to a single sglang server. For example:
+1.  Use `--rollout-server-concurrency` to limit the maximum number of concurrent requests sent to a single sglang server. For example:
 
     ```bash
-    --sglang-server-concurrency 160
+    --rollout-server-concurrency 160
     ```
 
 2.  Use `--sglang-cuda-graph-bs` (which corresponds to sglang's native `--cuda-graph-bs` argument) to increase the number of CUDA graphs initialized by sglang. For example:

diff --git a/docs/en/vllm/ROUTER_DESIGN.md b/docs/en/vllm/ROUTER_DESIGN.md
diff --git a/docs/zh/examples/deepseek-r1.md b/docs/zh/examples/deepseek-r1.md
@@ -171,7 +171,7 @@ OPTIMIZER_ARGS=(
 
 sglang 所需的参数，这里 `--rollout-num-gpus-per-engine` 基本对应 sglang 的 `tp_size`，除此之外的 sglang 参数均通过添加 `--sglang-` 的前缀来传给 slime。为了充分利用 sglang 的大 EP 推理能力，我们加上了 ep64、dp_attention dp8、deepep mode auto 等配置。
 
-最后的 `--sglang-server-concurrency` 是 slime 的特有参数，是为了方式同时发给 sglang server 的并发太大打爆 http server，默认为 512。但是我们现在是 8 机一个 server，为了保证每个 dp rank 能有 128 的并发，我们调整为 1024。
+最后的 `--rollout-server-concurrency` 是 slime 的特有参数，是为了方式同时发给 sglang server 的并发太大打爆 http server，默认为 512。但是我们现在是 8 机一个 server，为了保证每个 dp rank 能有 128 的并发，我们调整为 1024。
 
 ```bash
 SGLANG_ARGS=(
@@ -190,7 +190,7 @@ SGLANG_ARGS=(
    --sglang-deepep-mode auto
 
    # make every dp rank has 128 concurrency
-   --sglang-server-concurrency 1024
+   --rollout-server-concurrency 1024
 )
 ```
 

diff --git a/docs/zh/examples/qwen3-4B.md b/docs/zh/examples/qwen3-4B.md
@@ -280,10 +280,10 @@ ray job submit ... \
 
 ⚠️  在进行训推分离的时候，每个 sglang server 上的并发度太大，超过了 sglang 默认的 cuda graph 的并发度（默认最大 160），影响推理速度。可以用以下 2 种方式进行调整：
 
-1. 通过 `--sglang-server-concurrency` 限制发给一个 sglang server 的最大并发量，例如：
+1. 通过 `--rollout-server-concurrency` 限制发给一个 sglang server 的最大并发量，例如：
 
    ```bash
-   --sglang-server-concurrency 160
+   --rollout-server-concurrency 160
     ```
 
 2. 使用 `--sglang-cuda-graph-bs`，即 sglang 原生的 `--cuda-graph-bs`, 增大 sglang 初始化的 cuda graph 数量，例如：

diff --git a/examples/fully_async/fully_async_rollout.py b/examples/fully_async/fully_async_rollout.py
@@ -20,7 +20,7 @@ def get_global_worker(args, data_buffer):
     with _worker_lock:
         if _global_worker is None or not _global_worker.worker_thread.is_alive():
             print("Creating new global async worker...")
-            _global_worker = AsyncRolloutWorker(args, data_buffer, concurrency=args.sglang_server_concurrency)
+            _global_worker = AsyncRolloutWorker(args, data_buffer, concurrency=args.rollout_server_concurrency)
             _global_worker.start()
         return _global_worker
 

diff --git a/examples/tau-bench/run_qwen3_4B.sh b/examples/tau-bench/run_qwen3_4B.sh
@@ -100,7 +100,7 @@ SGLANG_ARGS=(
    --rollout-num-gpus-per-engine 1
    --sglang-mem-fraction-static 0.7
    # If gemini API reports concurrency limit error, set this parameter to reduce the concurrency
-   # --sglang-server-concurrency 32
+   # --rollout-server-concurrency 32
 )
 
 MISC_ARGS=(

diff --git a/goal_plan.md b/goal_plan.md
@@ -0,0 +1,33 @@
+### 阶段一：打通Qwen2.5-0.5B GRPO 8卡同步/异步训练（train.py和train_async.py），GSM8K 数据集，loss/reward 收敛与 SGLang backend 基本一致，且满足确定性计算，多次重复运行Loss曲线完全一致。
+
+First Design and RFC by 03/06 
+
+#### 初步方案：
+- 对标SGLang，Slime 在 Ray 内管理 vLLM 的完整生命周期，包括进程拉起、权重同步、推理暂停/恢复
+- 暂不使用Router，SGLang Model Gateway仅只支持SGLang Worker，SlimeRouter仅在 R3 / radix-tree caching 时需要，Qwen2.5-0.5B 非 MoE 且用 token-in/token-out
+- 单vLLM实例，无router，通过vLLMClient 直连本地 vLLM 进程端口
+- 先支持训推不共卡(non-colocate)，权重同步采用NCCL broadcast，对标SGLang update_weights_from_distributed  (默认）
+- 再支持和验证colocate，权重同步采用GPU IPC（vLLM update_weights_from_ipc, update_weights_from_tensor），对标SGLang update_weights_from_tensor，以验证Reproductivity。**IPC 依赖vllm 0.17**
+
+#### 风险：
+- slime, sglang版本依赖，和vllm 0.16的版本依赖冲突(numpy, torch, transformers, etc)
+- slime代码较挫，可靠性差，强依赖preset docker
+- 算力
+
+
+#### Reference
+
+https://thudm.github.io/slime/advanced/reproducibility.html
+
+
+### 阶段二：接入vllm-project/router，支持多实例vLLM
+
+- vllm router forked from SGLang Model Gateway
+
+### 阶段三：多节点大规模验证，MoE模型，optional：验证MTP Speculative Decoding，FP8 rollout 等高级特性
+
+- Model: Qwen/Qwen3-30B-A3B or GLM4.7
+- Parallel: 16卡 or 128卡, Train mixed EP+FSDP, Rollout EP+DP
+- Verify more features:
+    - Bf16 train, FP8 rollout 
+    - MTP Speculative Decoding