xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-06-03 05:10:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,129 Commits 155 Branches 89 Tags
Commit Graph

347 Commits

Author SHA1 Message Date
Huamin Li
7d6b03381e
[CI Failure] fix_test_auto_prefix_cache_support (#26053) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
 2025-10-04 02:44:49 -07:00
Stan Wozniak
ea507c3a93
[V1] [Hybrid] Mamba2 Automatic Prefix Caching (#25752) 
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Ortner <boh@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-04 06:34:22 +02:00
Wenlong Wang
79aa244678
[Multi Modal] Configurable MM Profiling (#25631) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-03 03:59:10 -07:00
Harry Mellor
10d765482d
FusedMoE support for the Transformers backend (#22650) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-02 23:12:15 -07:00
ElizaWszola
502640c3f9
[Perf] Fix and reapply move apply w8a8 block fp8 linear to class (#25696) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-10-02 19:35:13 +00:00
billishyahao
2518230d3e
[MISC] Fix misleading batch_size_capture_list when cuda_graph_sizes < 4 (#25829) 
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Luka Govedic <ProExpertProg@users.noreply.github.com>
 2025-10-01 08:39:45 -04:00
Lucia Fang
001e50c92c
[Model] MTP fallback to eager for DeepSeek v32 (#25982) 
Signed-off-by: Lu Fang <fanglu@fb.com>
 2025-10-01 01:53:22 +00:00
Harry Mellor
a388252ac4
Add explicit pooling classes for the Transformers backend (#25322) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-30 23:07:06 +01:00
Yongye Zhu
fa7e254a7f
[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
Simon Danielsson
e23cacda35
[Bugfix]: Clean up chunked prefill logging when using whisper (#25075) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2025-09-30 08:17:49 +00:00
acisseJZhong
e47433b3c1
[BugFix] Pass config_format via try_get_generation_config (#25912) 2025-09-30 05:09:50 +00:00
Harry Mellor
61aedb5ffe
MoveVllmConfig from config/__init__.py to config/vllm.py (#25271) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-29 19:49:49 -07:00
Chenxi Yang
d0d138bc55
[Nixl][P/D] Add cuda2cpu support (HD->DH transfer) (#24690) 
Signed-off-by: Chenxi Yang <cxyang@fb.com>
Co-authored-by: Chenxi Yang <cxyang@fb.com>
 2025-09-29 14:31:51 +00:00
Jiangyun Zhu
43227236ec
[torch.compile] serialize cudagraph_mode as its enum name instead of value (#25868) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-09-29 13:54:52 +00:00
Jiangyun Zhu
c0ec81836f
[torch.compile]: Add VLLM_DEBUG_DUMP_PATH environment variable (#25651) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-27 16:09:00 +00:00
Tyler Michael Smith
a5354b3ed2
[Bugfix][WideEP] Apply TP Attn + EP MoE fix to other models (#24982) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-27 14:22:28 +00:00
Cyrus Leung
27d7638b94
[Bugfix] Merge MM embeddings by index instead of token IDs (#16229) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-27 08:15:12 +00:00
Russell Bryant
3958b96bf5
Add option to restrict media domains (#25783) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Chenheli Hua <huachenheli@outlook.com>
 2025-09-27 01:23:52 +00:00
qizixi
c70ac4b8ff
[spec decode] Consolidate speculative decode method name for MTP (#25232) 
Signed-off-by: zixi-qi <qizixi@meta.com>
 2025-09-26 22:27:05 +00:00
fhl2000
f075693da7
[V1] address post issues related to #20059 (part 1) (#23046) 
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-26 15:58:19 -04:00
阿丹(adan)
33f6aaf972
Eagle3 that supports the Minicpm3 model (#24243) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: liudan <adan@minicpm.com>
Co-authored-by: liudan <liudan@qq.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
 2025-09-26 10:04:57 -07:00
Isotr0py
d4d9899860
[Quantization] Add field to skip unquantized modules for GPTQ config (#25455) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-26 15:47:41 +00:00
yyzxw
19f76ee68e
[misc] refactor speculative config (#25657) 
Signed-off-by: zxw <1020938856@qq.com>
 2025-09-26 01:22:06 -07:00
Eugene Khvedchenya
392edee34a
EVS Support (Video tokens pruning) (#22980) 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-26 11:54:54 +08:00
Russell Bryant
13dd93c667
[Core] Force PIECEWISE CUDAGraph mode for encoder-decoder (#25701) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-09-25 18:21:56 -07:00
Isotr0py
71b25b0d48
[V0 deprecation] Clean up V0 fallback in compilation config (#25675) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-25 17:29:51 +00:00
Cyrus Leung
0ea80c87d9
[Model] Define merge_by_field_config MM interface (#25676) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 17:13:07 +00:00
wang.yuqi
7f570f1caa
[V0 deprecation] Remove unreachable model_config.supported_tasks (#25642) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-25 11:26:31 +00:00
Tyler Michael Smith
1260180c67
Revert "[Performance] Move apply_w8a8_block_fp8_linear to an op class… (#25607) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-25 08:05:21 +00:00
XuruiYang
845adb3ec6
[Model] Add LongCat-Flash (#23991) 
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
 2025-09-24 21:53:40 -07:00
Harry Mellor
8938774c79
Move DeviceConfig, ObservabilityConfig, SpeechToTextConfig to their own files (#25564) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-24 13:59:05 +00:00
Woosuk Kwon
2e19a848d4
[V0 Deprecation] Remove max_seq_len_to_capture (#25543) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-24 01:51:39 -07:00
ahao-anyscale
c8bde93367
[BUG] Allows for RunAI Streamer and Torch.compile cache to be used together (#24922) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2025-09-23 18:13:32 -06:00
jiahanc
d5944d5146
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2025-09-23 15:44:35 -04:00
Michael Goin
24fab45d96
[Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 15:29:26 -04:00
ElizaWszola
63400259d0
[Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-09-23 12:03:10 -07:00
Lucas Wilkinson
cc1dc7ed6d
[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-23 16:02:10 +00:00
Luka Govedič
d5e0fca264
[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-22 12:30:05 -07:00
Lucia Fang
922979bfcc
[DP] support torchrun external launcher with Data Parallelism (#24899) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-22 12:06:05 -07:00
Yizhou
b6f01bd9a7
refactor: abstract graph mode support into platform interface (#25161) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2025-09-22 10:22:29 +00:00
Eldar Kurtić
21467f9a1c
Enable Eagle3 speculative decoding for GPT-OSS model (#25246) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
 2025-09-22 08:50:39 +00:00
Woosuk Kwon
bc6e542d9f
Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Rahul Tuli
c438b2951c
feat: Enable engine-level arguments with speculators models (#25250) 
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-09-21 11:04:45 -06:00
Woosuk Kwon
0ff8ebb2d7
[V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 08:52:32 -07:00
Cyrus Leung
c60e6137f0
[Optimization] Avoid repeated model architecture conversion for pooling models (#25261) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 13:30:22 +08:00
Boyuan Feng
8945b001db
[torch.compile] CUDAGraph Inductor partition integration (#24281) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-20 01:02:15 +00:00
Zhiyu
431535b522
Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-19 22:40:33 +00:00
Cyrus Leung
6c117cff7d
[Frontend] Pass API server count to each process (#23717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 01:15:19 +08:00
Harry Mellor
aed16879a9
Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Harry Mellor
058525b997
Move PoolerConfig from config/__init__.py to config/pooler.py (#25181) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 11:02:55 +00:00