xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-14 21:45:49 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,386 Commits 155 Branches 89 Tags
Commit Graph

119 Commits

Author SHA1 Message Date
Matthew Bonanni
d4fd2768ef
[Bugfix][Attention] Fix FlashInfer MLA block size logic (#24692) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-11 22:39:42 +00:00
Mengqing Cao
4f6593b058
[HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-09-11 21:47:58 +08:00
Hanjie Qiu
dcb28a332b
[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078) 
Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-10 15:31:10 -07:00
Thien Tran
a0933c3bd6
[Bugfix] Enable FP8 KV cache for FlashInfer and Triton backend on non-sm100 GPUs (#24577) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-09-10 12:33:41 -07:00
Wentao Ye
a55cf41a09
[Compilation][WideEP] Enable Piecewise CUDAGraph for DeepEPHT (#24123) 2025-09-09 10:21:10 -04:00
Lucas Wilkinson
402759d472
[Attention] FlashAttn MLA (#14258) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-04 02:47:59 -07:00
Matthew Bonanni
a742322092
[Attention] Blackwell FP8 MLA support with CUTLASS_MLA backend (#23289) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-03 14:05:24 -04:00
Kunshang Ji
fce10dbed5
[XPU] Add xpu torch.compile support (#22609) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-08-27 05:33:27 +00:00
Chaojun Zhang
8a044754bd
[XPU] Delay BF16 check to worker init for spawn compatibility (#22979) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2025-08-25 13:09:26 -07:00
Matthew Bonanni
19fe1a0510
[Kernel] Add FP8 support with FlashMLA backend (#22668) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-08-22 02:26:32 +00:00
Woosuk Kwon
14006840ea
[V0 Deprecation] Remove V0 FlashInfer attention backend (#22776) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-18 19:54:16 -07:00
fhl2000
74f441f4b5
[Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059) 
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-08-15 10:01:39 -04:00
Yongye Zhu
39cd09dc86
[Bugfix] use flash attn on sm90 (#22933) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-14 16:37:22 -07:00
Michael Goin
b1361c7273
[Bugfix] Fix default enable for CUTLASS MLA on SM100 (#22738) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-12 21:22:05 -07:00
Woosuk Kwon
71683ca6f6
[V0 Deprecation] Remove multi-step scheduling (#22138) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-08-12 20:18:39 -07:00
Yongye Zhu
007dd90859
[gpt-oss] Enable gpt-oss on ampere (#22714) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-08-12 03:21:44 -07:00
Giancarlo Delfin
469b3ffaaa
[V1] port xformers backend to v1 (#21342) 
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
 2025-08-05 10:04:46 -07:00
Giancarlo Delfin
aa7012eb6d
Add tree attention backend for v1 (part 1) (#20401) 
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
 2025-08-03 22:13:26 -07:00
vllmellm
d3a6f2120b
[FEAT][ROCm] Enable running Flash Attention as ViT attn backend for Qwen-VL models on ROCm platform. (#22069) 
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com>
 2025-08-01 23:53:18 -07:00
Michael Goin
61445453df
[UX] Rename CUTLASS_MLA_VLLM_V1 to CUTLASS_MLA (#21966) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-30 20:40:34 -07:00
Alexander Matveev
8f605ee309
[Attention] Make CutlassMLA the default backend for SM100 (blackwell) (#21626) 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-27 20:13:00 +00:00
elvischenv
2dec7c1a5d
[Bugfix][CUDA] fixes CUDA FP8 kv cache dtype supported (#21420) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-07-22 20:34:50 -07:00
Varun Sundar Rabindranath
8425f785ad
[Misc] DeepEPHighThroughtput - Enable Inductor pass (#21311) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-21 23:35:45 -07:00
Konrad Zawora
c17231e827
Fix kv_cache_dtype handling for out-of-tree HPU plugin (#21302) 
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
 2025-07-21 23:35:14 -07:00
Woosuk Kwon
dd572c0ab3
[V0 Deprecation] Remove V0 Spec Decode workers (#21152) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-18 21:47:50 -07:00
Woosuk Kwon
54cf1cae62
[Misc] Do not print async output warning for v1 (#21151) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 21:57:02 -07:00
Alexander Matveev
8cdc371217
SM100 Cutlass MLA decode with unrestricted num_heads (< 128) for DeepSeek TP (#20769) 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
 2025-07-15 01:06:38 +00:00
Cyrus Leung
e8cc53af5e
[Misc] Log the reason for falling back to FlexAttention (#20699) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-14 04:16:51 -07:00
Congcong Chen
2c11a738b3
[Model] New model support for microsoft/Phi-4-mini-flash-reasoning (#20702) 
Signed-off-by: Congcong Chen <congcongchen@microsoft.com>
 2025-07-12 06:02:10 -07:00
Pavani Majety
7bd4c37ae7
[Core] Add Flashinfer TRTLLM Backend for Flashinfer decode path (SM100). (#19825) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: shuw <shuw@nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 09:23:23 +00:00
Nick Hill
ffbcc9e757
[BugFix] Fix VllmConfig() construction on all platforms (#20695) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-10 07:00:20 +00:00
Kunshang Ji
0b407479ef
[misc]refactor Platform.set_device method (#20262) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-09 01:39:47 +00:00
Yang Yang
6e2c19ce22
[Refactor]Abstract Platform Interface for Distributed Backend and Add xccl Support for Intel XPU (#19410) 
Signed-off-by: dbyoung18 <yang5.yang@intel.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-07 04:32:32 +00:00
Cyrus Leung
9fb52e523a
[V1] Support any head size for FlexAttention backend (#20467) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-06 09:54:36 -07:00
Isotr0py
32c9be2200
[v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-05 09:41:10 +00:00
Liangliang Ma
a0389e0554
[UT][intel GPU] use current_platform instead of device hardcode in v1 tests (#20169) 
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
 2025-07-02 09:06:04 +08:00
kourosh hakhamaneshi
5e666f72cd
[Bugfix][Ray] Set the cuda context eagerly in the ray worker (#19583) 2025-06-19 22:01:16 -07:00
Michael Goin
36239f79dd
Fix FA2 fallback for Blackwell V1 (#19781) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-19 09:53:55 +08:00
Michael Goin
af09b3f0a0
[Bugfix][V1] Allow manual FlashAttention for Blackwell (#19492) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-12 10:40:24 +00:00
Michael Goin
7484e1fce2
Add cache to cuda get_device_capability (#19436) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-11 17:37:05 +08:00
Isotr0py
5f1ac1e1d1
Revert "[v1] Add fp32 support to v1 engine through flex attn" (#19404) 2025-06-10 01:30:20 -07:00
Isotr0py
b8089195b4
[v1] Add fp32 support to v1 engine through flex attn (#19319) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-06-09 22:10:44 +08:00
Driss Guessous
cf02f9b283
Add FlexAttention to V1 (#16078) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-06 21:58:55 -07:00
Benjamin Chislett
3465b87ef8
[Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
 2025-06-05 19:10:08 -07:00
Michael Goin
87360308b7
[V1] Use FlashInfer by default on Blackwell GPUs (#19118) 2025-06-05 15:40:39 -04:00
Kaixi Hou
41aa578428
[NVIDIA] Add Cutlass MLA backend (#17625) 2025-06-03 21:40:26 -07:00
Varun Sundar Rabindranath
fa98d77773
[Kernel] DeepEP dispatch-combine kernel integration (#18434) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-03 12:30:02 -07:00
Simon Mo
02f0c7b220
[Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Varun Sundar Rabindranath
7951d78738
[Core] Enable CUDA graphs for DP + All2All kernels (#18724) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-05-28 22:55:30 +00:00
Mengqing Cao
d781930f90
[Platform][Dist] Make torch distributed process group extendable (#18763) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-05-28 10:52:34 +00:00