基于vLLM-Ascend的Qwen3.5-122B模型Atlas 800I A3单机混部部署实践

​作者​昇腾实战派​知识地图​https://blog.csdn.net/Lumos_Lovegood/article/details/161601003背景概述本文档将介绍基于vLLM-Ascend的Qwen3.5-122B模型在Atlas 800I A3上的单机混部部署实践包括支持的特性、特性配置、环境信息以及性能测试典型case。基本信息软件版本设备信息组网形态总卡数数据格式0.18.0NPU: Atlas 800I A3-560T, HBM 128GCPU: Kunpeng 920 (80核-2900MHz)内存: 32根64G5200MHzOS: OpenEuler 22.03 LTS-SP4-2W4A8C16服务化配置低时延exportASCEND_RT_VISIBLE_DEVICES0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15exportPYTORCH_NPU_ALLOC_CONFexpandable_segments:TrueexportHCCL_IF_IPXXXXexportHCCL_OP_EXPANSION_MODEAIVexportHCCL_BUFFSIZE1024exportOMP_NUM_THREADS1echoperformance|tee/sys/devices/system/cpu/cpu*/cpufreq/scaling_governorsysctl-wvm.swappiness0sysctl-wkernel.numa_balancing0sysctlkernel.sched_migration_cost_ns50000exportLD_PRELOAD/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportTASK_QUEUE_ENABLE1exportVLLM_ASCEND_ENABLE_FUSED_MC21vllm serve /mnt/share/weights/Qwen3.5-122B-A10B-w8a8-org/\--served-model-nameqwen3.5\--host0.0.0.0\--port8000\--data-parallel-size1\--tensor-parallel-size4\--max-model-len5000\--max-num-batched-tokens8192\--max-num-seqs128\--gpu-memory-utilization0.9\--compilation-config{cudagraph_capture_sizes:[1,6,12,18,24,30,36,42,48,54,72,78,84,90,96,102,108,144,192], cudagraph_mode:FULL_DECODE_ONLY}\--speculative_config{method: qwen3_5_mtp, num_speculative_tokens: 5}\--trust-remote-code\--async-scheduling\--allowed-local-media-path /\--quantizationascend\--mm-processor-cache-gb0\--additional-config{enable_cpu_binding: true, enable_shared_expert_dp: true}典型测试用例平均输入平均输出并行策略上下文长度Prefix Cache命中率总请求数最大并发数请求频率(req/s)20482048MLADP1TP4500009624035001500MLADP1TP48000064160163841024MLADP1TP4184320328032768512MLADP1TP43430401640655361024MLADP1TP467584016401310721024MLADP1TP41331200410高吞吐exportASCEND_RT_VISIBLE_DEVICES0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15exportPYTORCH_NPU_ALLOC_CONFexpandable_segments:TrueexportHCCL_IF_IPXXXexportHCCL_OP_EXPANSION_MODEAIVexportHCCL_BUFFSIZE1024exportOMP_NUM_THREADS1echoperformance|tee/sys/devices/system/cpu/cpu*/cpufreq/scaling_governorsysctl-wvm.swappiness0sysctl-wkernel.numa_balancing0sysctlkernel.sched_migration_cost_ns50000exportLD_PRELOAD/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportTASK_QUEUE_ENABLE1exportVLLM_ASCEND_ENABLE_FUSED_MC21exportVLLM_ASCEND_ENABLE_FLASHCOMM11vllm serve /mnt/share/weights/Qwen3.5-122B-A10B-w8a8-org/\--served-model-nameqwen3.5\--host0.0.0.0\--port8000\--data-parallel-size1\--tensor-parallel-size4\--enable-expert-parallel\--max-model-len8000\--max-num-batched-tokens4096\--max-num-seqs128\--gpu-memory-utilization0.9\--compilation-config{cudagraph_capture_sizes:[1,4,8,12,16,24,32,48,56,64,72,84,96,108,112,128,160,172,196,200,212,232,256,160,172,196,200,212,232,256,272,288,312,328,344,360,384,400,416,432,448,480,512], cudagraph_mode:FULL_DECODE_ONLY}\--speculative_config{method: qwen3_5_mtp, num_speculative_tokens: 3, enforce_eager: true}\--trust-remote-code\--async-scheduling\--allowed-local-media-path /\--quantizationascend\--mm-processor-cache-gb0\--additional-config{enable_cpu_binding: true, enable_shared_expert_dp: true}典型测试用例平均输入平均输出并行策略上下文长度Prefix Cache命中率总请求数最大并发数请求频率(req/s)20482048MLADP1TP250000600150035001500MLADP1TP280000320800163841024MLADP1TP21843209624032768512MLADP1TP23430403690655361024MLADP1TP267584032801310721024MLADP1TP213312001640测试命令参考aisbench官方测试指南。aisbench测试命令vllm-ascend社区官网特别声明以上配置均未开启Prefix Cache若实际生产环境需要使用该特性参考vLLM-Ascend社区参数指南开启–enable-prefix-caching