xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-23 10:06:19 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,170 Commits 155 Branches 89 Tags
Commit Graph

61 Commits

Author SHA1 Message Date
Lucas Wilkinson
abe93bce59
[Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable true async spec-decode (#29624) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-12-09 17:18:10 -08:00
Lucas Wilkinson
aed846917f
[Attention] Make split_decodes_and_prefills(..., require_uniform=True) support padding (#29644) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-12-09 07:24:01 +00:00
Cyrus Leung
e83b7e379c
Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) 2025-12-07 00:00:22 -08:00
Cyrus Leung
27f4c2fd46
[Renderer] Separate out RendererConfig from ModelConfig (#30145) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 23:15:42 -08:00
Lucas Wilkinson
c8ab988b15
[BugFix] Fix DBO assert assert B_block_table == B_q (#29933) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-04 14:48:54 -05:00
Harry Mellor
951445a52d
Remove default values from InitVars so that they're not stored (#29859) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-02 12:16:37 +00:00
Isotr0py
b95db244ee
[v1] Add real sliding window calculation to FlexAttention direct BlockMask building (#26015) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Co-authored-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
 2025-12-01 13:12:51 +00:00
Matthew Bonanni
fc1d8be3dc
[Attention] Update attention imports (#29540) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-27 11:19:09 -05:00
Micah Williamson
ef1f7030f0
[ROCm][CI] Fix test_cudagraph_mode failure in AMD CI (#29367) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2025-11-25 07:55:09 +00:00
vllmellm
64deead719
[Bugfix] [ROCm] [UX]: revert Flex attention backend (#29371) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-11-25 06:56:06 +00:00
vllmellm
e48b2e6848
[Bugfix] [ROCm] [UX] Reorganize ROCm Backend Selection Logic (#26980) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-11-24 15:24:49 +00:00
Nicolò Lucchesi
066209a045
[Attention] Refactor FA block_size limitations to hybrid models only (#29084) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-11-22 06:38:44 -08:00
Matthew Bonanni
b30dfa03c5
[Attention] Refactor CUDA attention backend selection logic (#24794) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-11-11 07:40:44 -05:00
Matthew Bonanni
01baefe674
Add TP parameter to attention tests (#27683) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-03 13:04:40 -08:00
Matthew Bonanni
f29aeb5a25
Add FLASHINFER_MLA to test_mla_backends and add B200 CI run (#27663) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-31 11:12:19 -07:00
Lucas Wilkinson
b5d70751d8
[BugFix] Reordering extend logic fix (#27739) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-10-29 21:39:34 -07:00
Lucas Wilkinson
141e6a0505
[Misc] Make reorder batch also separate extends (#27367) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-10-28 10:55:10 -07:00
Mohammad Miadh Angkad
a8c02fb5bf
[Bugfix][CI] Fix v1 attention backend tests and add CI coverage (#26597) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
Signed-off-by: Mohammad Miadh Angkad <mangkad.bsdsba2027@aim.edu>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-28 11:42:05 -04:00
Yeshwanth N
71b1c8b667
[Chore]:Extract math and argparse utilities to separate modules (#27188) 
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>
 2025-10-26 04:03:32 -07:00
Isotr0py
6ac5e06f7c
[Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Huamin Li
c312320764
[CI/Build] tests(v1): feed Triton attention the (num_blocks, 2, …) KV cache layout in backend-correctness tests (#26663) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-17 21:11:26 -07:00
Cyrus Leung
4d4d6bad19
[Chore] Separate out vllm.utils.importlib (#27022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 00:48:59 +00:00
Matthew Bonanni
82af928c41
[Attention][Spec Decode] FlashMLA spec decode support (#26541) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-14 19:38:20 +00:00
Harry Mellor
8fcaaf6a16
Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Matthew Bonanni
76879cc160
[Attention] Implement universal BACKEND_MAP (#25900) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-08 12:00:25 -07:00
Harry Mellor
2f99f2f506
Tidy vllm/config/__init__.py to only add classes and functions (#26405) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 07:10:00 -07:00
Lucas Wilkinson
f80e7866c0
[Misc] Clean up cruft from previous FlashMLA sparse implementation (#26125) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-10-08 10:09:34 +08:00
Cyrus Leung
1e4ecca1d0
[V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:42:31 +00:00
Sage Moore
2111b4643c
[Core] Simplify the Dp padding/should ubatch coordination logic (#25768) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-10-07 01:57:49 +00:00
Harry Mellor
6c04638214
Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Matthew Bonanni
2aaa423842
[Attention] Move Backend enum into registry (#25893) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-02 20:32:24 -07:00
Chen Zhang
1e50f1be70
[Deepseek v3.2] Support indexer prefill chunking (#25999) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-02 10:29:12 -07:00
Yongye Zhu
fa7e254a7f
[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
fhl2000
f075693da7
[V1] address post issues related to #20059 (part 1) (#23046) 
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-26 15:58:19 -04:00
Matthew Bonanni
3468f17ebe
[V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-25 17:37:50 +00:00
Jonas M. Kübler
58c360d9be
[Bug] fix import and unit test (#25558) 
Signed-off-by: Jonas M. Kübler <44084297+jmkuebler@users.noreply.github.com>
 2025-09-24 10:17:59 +00:00
Benjamin Chislett
c30b405b8f
[Spec Decode] Enable FlashInfer Spec Decoding (#25196) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
 2025-09-23 22:29:58 -04:00
Lucas Wilkinson
cc1dc7ed6d
[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-23 16:02:10 +00:00
Isotr0py
cf56cf78b4
[V1] Add sliding window support to Flex Attention backend (#24089) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-21 05:08:07 +00:00
Sage Moore
567939953b
[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
Matthew Bonanni
5fe643fc26
Add FLASHINFER_MLA to backend selector test (#24753) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-12 22:30:07 +00:00
Matthew Bonanni
7ba32aa60b
[Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-12 15:45:53 -06:00
Matthew Bonanni
620db1fc58
[Attention] FlashAttention MLA cudagraph support (#23958) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-08 22:05:26 +00:00
Lucas Wilkinson
402759d472
[Attention] FlashAttn MLA (#14258) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-04 02:47:59 -07:00
co63oc
1bd007f234
fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
Driss Guessous
e0329ed4b4
Updates to Flex + VLLm integration (#21416) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-08-25 09:32:42 -04:00
Ayush Satyam
5c4b6e66fe
[Attention] Unify mamba and attention backend selection (#23171) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-08-25 09:09:36 +00:00
Matthew Bonanni
10cc12ba66
Feature/mla tests (#23195) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-08-20 21:46:47 +00:00
Woosuk Kwon
d6d13bd49e
[Misc] Add max_seq_len to CommonAttentionMetadata (#23216) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 09:05:29 -07:00