Skip to content

Update run-e2e skill fixing inconsistencies for the Qwen suite - #306

Merged
jiangkuaixue123 merged 3 commits into
vllm-project:mainfrom
ms-llmd:fix-e2e-skill
Sep 7, 2026
Merged

Update run-e2e skill fixing inconsistencies for the Qwen suite#306
jiangkuaixue123 merged 3 commits into
vllm-project:mainfrom
ms-llmd:fix-e2e-skill

Conversation

@ronenkat

@ronenkat ronenkat commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

PLEASE FILL IN THE PR DESCRIPTION AND MAKE SURE THE CHECKLIST ITEMS HAVE BEEN CONSIDERED.

Purpose

  1. Correct the skill's Scope section: the four gate scenarios and the 2A2F
    topology belong to DeepSeek-V2-Lite only; the Qwen3 MoE and Qwen3.6 suites
    are baseline-graph plus three 2A1F scenarios, which use the first two
    devices for Attention and the third for FFN and leave the fourth idle.
    Apply the same correction to the opening paragraph of
    tests/e2e/README.md.
  2. Add HF_ENDPOINT to the skill's prerequisite checks and to its environment
    reference table, with the reason: gsm8k.py defaults the lm-eval child to
    hf-mirror.com, so an unreachable mirror is not visible until GSM8K starts,
    with the servers already up.

Issue

Scope

  • In scope: Update run-e2e skill and e2e readme
  • Out of scope:

Implementation Notes

None

Test Plan

N/A

Test Result

N/A

Docs Impact

  • Files updated:
  • tests/e2e/README.md
  • .agents/skills/run-e2e/SKILL.md
  • If none, reason:

Essential PR Checklist
  • [ X ] Purpose is clear and linked to public context when possible.
  • [ X ] Scope is bounded.
  • Compatibility with vLLM v0.26.0 is considered.
  • [ X ] No changes are made to the vLLM source checkout.
  • Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
  • Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested.
  • Imports remain CPU-safe; CUDA-heavy work is delayed or GPU-gated.
  • Validation evidence is included, including skipped GPU tests when applicable.
  • [ X ] Documentation impact is stated.

Signed-off-by: ronenkat <16743404+ronenkat@users.noreply.github.com>

@jiangkuaixue123 jiangkuaixue123 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The topology correction matches the Qwen suites, and the HF_ENDPOINT guidance matches gsm8k.py. Two remaining passages should be aligned with the corrected scope.

Comment thread tests/e2e/README.md
leaving the fourth device idle. For DeepSeek-V2-Lite these are local-only
cases; for Qwen3 MoE and Qwen3.6 MoE they are the suite's gate scenarios.

Tests run sequentially and must not skip. Every GSM8K evaluation uses 8

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also update the “Local 2A1F cases” section below (lines 94–98). It still says all 2 Attention + 1 FFN scenarios are local-only and that CI gates do not run them, which contradicts the Qwen gate scope clarified here. Qualify both the heading and that sentence as DeepSeek-V2-Lite-specific, e.g. “The DeepSeek-V2-Lite 2A1F scenarios are local-only; its CI gate selects the 2A2F scenarios.”

`baseline-graph` uses native DP4/TP1/EP4. The 2A1F cases (`afd-eager-2a1f`,
`afd-graph-2a1f`, `afd-graph-dbo-2a1f`) use 2 Attention ranks and 1 FFN rank;
for DeepSeek-V2-Lite they are local-only scenarios, while for Qwen3 MoE and
Qwen3.6 MoE they are the suite's gate scenarios.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also align the device-order paragraph below (lines 89–90), which still calls these “The local 2A1F scenarios.” Since this skill covers Qwen suites whose gate scenarios use this same topology, remove “local” there or explicitly qualify it as DeepSeek-V2-Lite-only. This will keep the execution guidance consistent with the corrected scope here.

Signed-off-by: ronenkat <16743404+ronenkat@users.noreply.github.com>
@ronenkat

ronenkat commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

The topology correction matches the Qwen suites, and the HF_ENDPOINT guidance matches gsm8k.py. Two remaining passages should be aligned with the corrected scope.

Thank you for the feedback. Text updated.

@jiangkuaixue123
jiangkuaixue123 merged commit 069eeef into vllm-project:main Sep 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Doc]: run-e2e skill misstates the Qwen suite topology and omits the HF_ENDPOINT prerequisite

2 participants