Skip to content

fix(converter): detect SGLang device backend - #116

Merged
chaokunyang merged 1 commit into
inclusionAI:mainfrom
Le8r0nJames:zjw/fix-cuda-triton-moe-layout
Aug 19, 2026
Merged

fix(converter): detect SGLang device backend#116
chaokunyang merged 1 commit into
inclusionAI:mainfrom
Le8r0nJames:zjw/fix-cuda-triton-moe-layout

Conversation

@Le8r0nJames

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR makes the SGLang weight converter use AWEX's shared runtime device
detection instead of inferring the device backend from visibility environment
variables.

Explicit device_backend/device_type configuration and the HCCL backend
continue to take precedence. When they are absent, the converter now delegates
to device_util.get_device_type().

Some launchers export both CUDA_VISIBLE_DEVICES and
ASCEND_RT_VISIBLE_DEVICES. Previously, the presence of the Ascend variable
caused a CUDA process to be misclassified as NPU, which could incorrectly apply
NPU-specific weight conversion behavior.

A regression test covers the case where both visibility variables are present
while CUDA is the available runtime device.

Related issues

None.

Does this PR introduce any user-facing change?

No public API or binary protocol changes. Device backend selection now reflects
the configured or actually available runtime device instead of relying on
visibility environment variables.

  • Does this PR introduce any public API change?
  • Does this PR introduce any binary protocol compatibility change?

Validation

  • ruff check awex/converter/sglang_converter.py awex/tests/test_qwen3_moe_sglang_converter.py
  • ruff format --check awex/converter/sglang_converter.py awex/tests/test_qwen3_moe_sglang_converter.py
  • pytest awex/tests/test_qwen3_moe_sglang_converter.py awex/tests/test_qwen3_dense_sglang_converter.py awex/tests/test_meta_resolver.py awex/tests/test_model_registry.py -q
    • 30 passed
    • 2 skipped

@Le8r0nJames
Le8r0nJames marked this pull request as ready for review August 17, 2026 07:18
@chaokunyang
chaokunyang merged commit 7005e52 into inclusionAI:main Aug 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants