xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-06-01 22:57:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,101 Commits 155 Branches 89 Tags
Commit Graph

305 Commits

Author SHA1 Message Date
Paul Pak
5f42fc53b6
[backends][short_conv] CUDA graph piecewise edits (#24215) 
Signed-off-by: Paul Pak <paulpak58@gmail.com>
 2025-10-03 12:59:48 +00:00
Sage Moore
5f2cacdb1e
Quick fix for IMA with the Prefix Prefill kernel during graph capture (#25983) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-10-03 11:28:22 +00:00
Michael Goin
f1fc2107a3
[Bugfix] Disable cascade attention with FlashInfer (#26130) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-02 16:30:37 -07:00
Chen Zhang
1e50f1be70
[Deepseek v3.2] Support indexer prefill chunking (#25999) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-02 10:29:12 -07:00
Lucas Wilkinson
decf7f794b
[BugFix] Fix FI accuracy issue when used for MLA prefill (#26063) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-10-02 17:18:13 +00:00
Gregory Shtrasberg
0b018d8baf
[ROCm][Bugfix] Add missing parameter to ROCm backend (#26029) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-10-01 19:23:14 -07:00
Huamin Li
c36f0aa300
Fix test_mamba_ssm_ssd.py due to missing _query_start_loc_to_chunk_indices_offsets (#25995) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
 2025-10-01 18:18:36 +00:00
Lucia Fang
001e50c92c
[Model] MTP fallback to eager for DeepSeek v32 (#25982) 
Signed-off-by: Lu Fang <fanglu@fb.com>
 2025-10-01 01:53:22 +00:00
Yongye Zhu
fa7e254a7f
[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
Thomas Parnell
fea3e476aa
[Kernel] Chunk-aligned mamba2 (#24683) 2025-09-29 23:18:25 +02:00
Bram Wasti
dc48ba0c75
Kernel-override Determinism [1/n] (#25603) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-09-26 16:59:09 -07:00
Chih-Chieh Yang
2b6b1d7809
[Model] Mamba2 varlen refactor (#21467) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: RishiAstra <40644327+RishiAstra@users.noreply.github.com>
 2025-09-26 11:31:14 +00:00
Icey
dd70437a4f
Remove cuda hard-code in compute_causal_conv1d_metadata (#25555) 
Signed-off-by: Icey <1790571317@qq.com>
 2025-09-26 01:19:20 -07:00
Tao He
99b3a504c5
[Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride bug in causal_conv_1d. (#25743) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-26 01:18:58 -07:00
Matthew Bonanni
3468f17ebe
[V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-25 17:37:50 +00:00
Li, Jiang
eb32335e35
[CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-25 13:29:11 +00:00
Jonas M. Kübler
69a8c8e99a
[torch.compile] Make Query Quantization Fusable (#24914) 
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
 2025-09-25 09:25:12 -04:00
Wei Wei
05c19485a5
[Kernel] Support DCP for Triton backend (#25132) 
Signed-off-by: Wei Wei <wwei6@meta.com>
 2025-09-24 18:09:34 -07:00
Woosuk Kwon
e6750d0b18
[V0 Deprecation] Remove unused classes in attention (#25541) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-24 13:24:40 -07:00
Lucas Wilkinson
2338daffd3
[BugFix] Potential Fix for FA3 full-cudagraph IMA (#25490) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-24 02:04:04 -07:00
Benjamin Chislett
c30b405b8f
[Spec Decode] Enable FlashInfer Spec Decoding (#25196) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
 2025-09-23 22:29:58 -04:00
Lucas Wilkinson
9df8da548e
[BugFix] Fix MLA assert with CUTLASS MLA (#25478) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-23 21:09:43 -04:00
Benjamin Chislett
1983609239
[Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520) 2025-09-24 00:19:56 +00:00
Burkhard Ringlein
100b630a60
[V1][Kernel] Add triton implementation for reshape_and_cache_flash (#24503) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-23 12:52:40 -04:00
Lucas Wilkinson
cc1dc7ed6d
[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-23 16:02:10 +00:00
Thomas Parnell
a903669e10
[V1] Remove V0 code paths for Hybrid models (#25400) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-09-23 08:26:13 -07:00
Wentao Ye
4741239db7
[Bug] Fix Long Context OOM Issue (#25290) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-22 22:04:15 -04:00
Alexander Matveev
0b7bed9c38
[Performance] Remove input pads in cutlass_mla and optimize v_proj output handling (#25184) 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
 2025-09-22 19:20:53 -06:00
Daisy-Ma-coder
cfbee3d0e7
[CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) 
Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-22 10:37:43 -07:00
Burkhard Ringlein
175811e3b5
[V1][Attention] Split triton_attn in triton-only and rocm specific backends (#24648) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
 2025-09-22 15:20:28 +00:00
Cyrus Leung
f92d952632
[V0 Deprecation] Remove MultiModalPlaceholderMap (#25366) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-22 08:49:19 +00:00
Isotr0py
cf56cf78b4
[V1] Add sliding window support to Flex Attention backend (#24089) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-21 05:08:07 +00:00
Lucia Fang
ee7a66dd9a
allow disable flashinfer prefill (#25276) 
Signed-off-by: Lu Fang <fanglu@fb.com>
 2025-09-19 22:59:41 +00:00
nvjullin
b1a63d1b3b
[BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) 
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-19 20:36:34 +00:00
Wentao Ye
75fb112d80
[Bug] Fix returned_lse not Defined issue (#25106) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-18 19:32:24 +00:00
Rohan Potdar
bbdc0f2366
[ROCm][AITER][Bugfix] Switch AITER to use PIECEWISE_AND_FULL compilation (#25104) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2025-09-18 17:46:47 +00:00
wang.yuqi
5f696c33b1
[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-18 23:22:01 +08:00
Vadim Gimpelson
072d7e53e5
[PERF] Add conv1d metadata to GDN attn (#25105) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-09-18 14:27:49 +00:00
elvischenv
e67a79db03
[Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-17 15:36:29 -07:00
Douglas Lehr
1a456c7c90
Aiter mha fp8 fix (#24991) 
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
 2025-09-17 22:29:14 +00:00
Matthew Bonanni
8f3616f422
Remove old cutlass mla (#23961) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-17 14:31:43 +00:00
Tao He
dd6a910aac
[Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP implementation. (#24957) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-17 21:59:09 +08:00
Aleksandr Malyshev
3053a22b33
fp8 kv cache support fix for torch.compile (#22758) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
 2025-09-16 21:27:11 +00:00
Sage Moore
567939953b
[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
Wentao Ye
b42566f440
[Bug] Fix is_flashmla_supported Check Error (#24774) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-15 20:10:55 -06:00
Alexander Matveev
aae725af7c
[Performance] Remove redundant clone() calls in cutlass_mla (#24891) 2025-09-15 20:21:53 +00:00
Nicolò Lucchesi
2e41f5abca
[XPU] Set consistent default KV cache layout (#24745) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-15 18:09:34 +08:00
Tao He
8226dd56bf
[Qwen3Next] Fixes the cuda graph capture conditions under large batch sizes (#24660) (#24667) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-12 22:31:32 +00:00
Matthew Bonanni
7ba32aa60b
[Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-12 15:45:53 -06:00
Ignacio Sica
7a1c4025f1
[Kernel] [CPU] refactor cpu_attn.py:_run_sdpa_forward for better memory access (#24701) 
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
 2025-09-12 19:23:07 +08:00