Performance regression

[XuehaoSun]

commit: f1af646
test/test_cpu/utils/test_model_scope.py

Model	Before	After	Increase
Qwen/Qwen2-VL-2B-Instruct	5 min	12 min	+7 min (+140%)
Qwen/Qwen2.5-0.5B-Instruct	1.5 min	2.5 min	+1 min (+66.7%)

[wenhuach21]

How about cuda and large models? if it's only cpu, I have no time to fix it.

[XuehaoSun]

It can be reproduced on CUDA.

Model	Before	After	Increase	Change
Qwen2.5-0.5B-Instruct	01:15	01:59	+44s	+58.7%
Qwen3-0.6B	01:22	01:56	+34s	+41.5%