xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-09-09 03:37:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,691 Commits 155 Branches 89 Tags
Commit Graph

6627 Commits

Author SHA1 Message Date
Zhewen Li
7d4651997a
[CI/Build] Add bc-linter to vLLM CI (#21234) 
Signed-off-by: zhewenli <zhewenli@meta.com>
 2025-09-11 15:34:36 -07:00
Woosuk Kwon
569bf1c9c0
[Qwen3-Next] MoE configs for H200 TP=1,2,4 (#24695) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-11 14:38:16 -07:00
Wentao Ye
1ec20355f5
[Bugfix] Set VLLM_ALLREDUCE_USE_SYMM_MEM default to False (#24696) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-11 14:32:27 -07:00
Xiaozhu Meng
e42af78b18
[flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197) 
Signed-off-by: Xiaozhu <mxz297@gmail.com>
 2025-09-11 14:20:09 -07:00
Duncan Moss
074854b24f
[Kernel][B200] mxfp4 fused cutlass moe (#23696) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-11 17:04:56 -04:00
Andrew Xia
79ac59f32e
Update Spec Decode metrics to include drafted and accepted token throughput (#24127) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-11 19:58:43 +00:00
Nick Hill
b971f91504
[BugFix] Fix tokenize asyncio task leak (#24677) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-11 19:44:04 +00:00
Woosuk Kwon
c733bd5e87
[Qwen3-Next] Add MoE Config for H200 (#24688) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-11 12:40:15 -07:00
Wentao Ye
a892b259b4
[Doc] Remove Useless Comments (#24687) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-11 12:25:47 -07:00
Peter Salas
127ded0a9e
[Ultravox] Use wrapped_model_config to instantiate inner model (#24679) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2025-09-11 18:52:24 +00:00
Isotr0py
bb2b5126da
[VLM] Migrate remain DP-supported ViT models to use disable_tp (#24363) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-11 18:30:41 +00:00
co63oc
e26fef8397
fix some typos (#24616) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-11 10:48:46 -07:00
Harry Mellor
c1eda615ba
Fix model name included in responses (#24663) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 10:47:51 -07:00
Konrad Zawora
4aa23892d6
[Bugfix] Fix platform-specific routing in CustomOp implementations (#24444) 
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
 2025-09-11 17:15:01 +00:00
Ilya Markov
1fdd5c42d7
[Kernels] Enable Torch Symmetric Memory All-Reduce By Default (#24111) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-11 09:45:31 -07:00
Isotr0py
bcbe2a4d9e
[VLM] Optimize GLM4.5-V-style video processing to only decode necessary frames (#24161) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-11 09:44:34 -07:00
Jee Jee Li
817beef7f3
[Bugifx] Fix qwen-next packed_modules_mapping (#24656) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 22:26:17 +08:00
Mengqing Cao
4f6593b058
[HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-09-11 21:47:58 +08:00
Boyuan Feng
94e6b2d55f
Allow users to specify kv cache memory size (#21489) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 13:41:07 +00:00
Harry Mellor
5f5271f1ee
Move LoRAConfig from config/__init__.py to config/lora.py (#24644) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 11:01:38 +00:00
Harry Mellor
d6249d0699
Fix typing for safetensors_load_strategy (#24641) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 10:41:39 +00:00
wang.yuqi
25bb9e8c65
[CI Failure] fix models/language/pooling/test_auto_prefix_cache_support.py (#24636) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-11 03:31:23 -07:00
Kyuyeun Kim
ed5ae4aace
[Bugfix] Fix _synced_weight_loader (#24565) 
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
 2025-09-11 16:52:33 +08:00
Cyrus Leung
6aeb1dab4a
[Bugfix] Fix incorrect import of CacheConfig (#24631) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-11 01:48:25 -07:00
Tao He
e93f4cc9e3
Add the support for the qwen3 next model (a hybrid attention model). (#24526) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 15:32:09 +08:00
Jerry Zhang
2048c4e379
[torchao] Support quantization configs using module swap (#21982) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
 2025-09-10 23:53:24 -07:00
Chenxi Yang
d13360183a
Remove redundant all gather + split (#23441) 
Co-authored-by: Chenxi Yang <cxyang@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2025-09-10 23:45:07 -07:00
TaehyunKim
9bd831f501
[Model] New model support for Motif-1-Tiny (#23414) 
Signed-off-by: ca1207 <ca1207zzz@gmail.com>
Signed-off-by: TaehyunKim <73943231+ca1207@users.noreply.github.com>
Co-authored-by: WyldeCat <skan1543@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-10 23:29:40 -07:00
Didier Durand
e2b1f863aa
[Doc]: fixing doc typos (#24635) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-10 23:19:28 -07:00
shengshiqi-google
41329a0ff9
[Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469) 
Signed-off-by: Shiqi Sheng <shengshiqi@google.com>
Signed-off-by: shengshiqi-google <160179165+shengshiqi-google@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-10 23:10:01 -07:00
Tomas Ruiz
ee0bc5e1b4
Enable --profile in 'vllm bench throughput' (#24575) 
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2025-09-10 23:06:19 -07:00
Saman A. Pour
3d1393f6fc
Kimi K2 Fused MoE kernels Optimization configs (#24597) 
Signed-off-by: Saman Keon <samanamp@outlook.com>
 2025-09-10 23:06:16 -07:00
Guy Stone
8a894084d2
[Engine][Chore] use local variable and remove output var assignment (#24554) 
Signed-off-by: Guy Stone <guys@spotify.com>
 2025-09-10 23:05:42 -07:00
Nick Hill
e2d8c27f68
[BugFix] Fix pipeline parallel (#24621) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-10 23:05:30 -07:00
Li, Jiang
29799ddacc
[Bugfix] Add missing VIT backend dispatch on CPU (#24623) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-10 22:28:41 -07:00
Peter Salas
f17a6aa4ec
[Ultravox] Fix Gemma instantiation, support quantization via --hf-overrides (#24131) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2025-09-10 22:25:34 -07:00
Alexandre Marques
5931b7e5d9
[Models][Quantization] Add quantization configuration update in Voxtral model (#24122) 
Signed-off-by: Alexandre Marques <almarque@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-10 19:13:56 -07:00
Jonathan Berkhahn
cc99baf14d
[Misc] Make timeout passable in init_distributed_environment (#24522) 
Signed-off-by: jberkhahn <jaberkha@us.ibm.com>
 2025-09-10 15:41:12 -07:00
Hanjie Qiu
dcb28a332b
[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078) 
Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-10 15:31:10 -07:00
Michael Goin
fba7856581
[Perf] Warmup FlashInfer attention during startup (#23439) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-10 15:03:17 -07:00
Chen Zhang
b5e383cd8b
[gpt-oss] raise error for flashinfer backend without trtllm (#24482) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-10 14:33:13 -07:00
Gregory Shtrasberg
9a161307f5
[torch.compile][ROCm][V1] Enable attention output FP8 fusion for V1 attention backends (#19767) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-10 13:59:55 -07:00
Russell Bryant
37e8182bfe
[v1] Add Whisper model support (encoder-decoder) (#21088) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2025-09-10 13:53:35 -07:00
Nick Hill
4db4426404
[CI] Fail subprocess tests with root-cause error (#23795) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-10 13:53:21 -07:00
Thien Tran
a0933c3bd6
[Bugfix] Enable FP8 KV cache for FlashInfer and Triton backend on non-sm100 GPUs (#24577) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-09-10 12:33:41 -07:00
rongfu.leng
09e68bce34
[Misc] update log level debug to warning when process port is used by (#24226) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-10 11:32:57 -07:00
Xingyu Liu
9fb74c27a7
[Core] Support configuration parsing plugin (#24277) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Signed-off-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-10 11:32:43 -07:00
Ming Yang
4032949630
[Bugfix] Fix DeepEP config for DP4TP4 (#23619) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-09-10 10:37:56 -07:00
tomeras91
08abfa78ec
[Bugfix] fix modelopt exclude_modules name mapping (#24178) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-10 10:20:46 -07:00
Shiyan Deng
2bef2d1405
[Logging] allow config logging stream (#24336) 
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
 2025-09-10 15:02:01 +00:00