xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-01-19 02:14:28 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,204 Commits 155 Branches 89 Tags
Commit Graph

327 Commits

Author SHA1 Message Date
Gregory Shtrasberg
f231e5bc21
[ROCm] Split AITER unified attention into its own backend (#25507) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-10-06 22:49:23 +00:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Hank_
17edd8a807
[Platform][Kernel] platform-specific kernel loading (#25823) 
Signed-off-by: Hank <hcc.mayday@gmail.com>
 2025-10-05 13:25:15 +02:00
ihb2032
3303cfb4ac
[Bugfix][Hardware][RISC-V] Limit supported dtypes to float32 to avoid scheduler segfault (#26228) 
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
 2025-10-05 10:36:54 +00:00
TJian
9c5ee91b2a
[ROCm] [VL] [Bugfix] Fix vit flash attn dispatcher logic for ROCm (#26104) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-10-02 22:34:53 -07:00
Matthew Bonanni
2aaa423842
[Attention] Move Backend enum into registry (#25893) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-02 20:32:24 -07:00
Yongye Zhu
fa7e254a7f
[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
Aaron Pham
6a113d9aed
[V0 Deprecation] Remove vllm.worker and update according imports (#25901) 2025-09-29 23:26:11 +00:00
Chenxi Yang
d0d138bc55
[Nixl][P/D] Add cuda2cpu support (HD->DH transfer) (#24690) 
Signed-off-by: Chenxi Yang <cxyang@fb.com>
Co-authored-by: Chenxi Yang <cxyang@fb.com>
 2025-09-29 14:31:51 +00:00
Roger Wang
65ecb4f134
[Bugfix] Fallback ViT attn backend to SDPA for blackwell (#25851) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-29 06:03:51 +00:00
Matthew Bonanni
3468f17ebe
[V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-25 17:37:50 +00:00
chenlang
1e9a77e037
[Hardware][RISC-V] Add riscv64 support for vLLM with scalar (#22112) 
Signed-off-by: chenlang <chen.lang5@zte.com.cn>
Co-authored-by: chenlang <10346245@zte.com.cn>
 2025-09-25 20:46:11 +08:00
Tyler Michael Smith
fea8006062
[Logging] Improve log for when DeepEP HT disables CUDA Graphs (#25531) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-24 22:43:06 +00:00
Gregory Shtrasberg
487745ff49
[ROCm][Bugfix] Only enable +rms_norm based on aiter if not explicitly disabled (#25275) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-24 11:24:39 -04:00
Juan Villamizar
bde2a1a8a4
[ROCm] Small functional changes for gptoss (#25201) 
Signed-off-by: jpvillam <jpvillam@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
 2025-09-23 23:39:50 +00:00
Chendi.Xue
5774b0a1da
[NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
 2025-09-23 04:17:42 +00:00
Kunshang Ji
f225ea7dd9
[XPU] Fix compile_size is None case. (#25433) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-23 03:09:00 +00:00
Isotr0py
6fa78d8f23
[V0 deprecation] Remove platform v1 controling interface (#25410) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-23 01:48:12 +00:00
Michael Goin
239ef0c1ac
[CI Failure] Fix fp8 kv cache on <SM90 (#25396) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-22 18:27:51 +00:00
Burkhard Ringlein
175811e3b5
[V1][Attention] Split triton_attn in triton-only and rocm specific backends (#24648) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
 2025-09-22 15:20:28 +00:00
Yizhou
b6f01bd9a7
refactor: abstract graph mode support into platform interface (#25161) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2025-09-22 10:22:29 +00:00
Woosuk Kwon
bc6e542d9f
Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Woosuk Kwon
0ff8ebb2d7
[V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 08:52:32 -07:00
Chendi.Xue
a6149aa587
[OOT] Support sync_model_loading for OOT (#25126) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
 2025-09-19 05:41:53 +00:00
Wentao Ye
d2a30a2d93
[Bug] Fix torch Compilation Cache Hit Error (#25093) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-18 12:38:37 -07:00
Li, Jiang
9fccd04e30
[Bugfix] Fix Stream usage in CPU model runner and OneDNN kernel check (#25046) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-17 05:54:02 -07:00
Woosuk Kwon
5801e49776
[V0 Deprecation] Remove MQLLMEngine (#25019) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-16 21:29:27 -07:00
Kunshang Ji
5206ab20ba
[XPU] Fix circular import error. (#24927) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-16 03:35:36 +00:00
Nicolò Lucchesi
2e41f5abca
[XPU] Set consistent default KV cache layout (#24745) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-15 18:09:34 +08:00
Victor Ziliang Peng
973c9d01da
[Minor] Simplify duplicative device check for cuda (#24793) 
Signed-off-by: Ziliang Peng <ziliangdotme@gmail.com>
 2025-09-13 18:28:38 +00:00
Matthew Bonanni
7ba32aa60b
[Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-12 15:45:53 -06:00
Wenlong Wang
72fc8aa412
[Multi Modal] Add FA3 in VIT (#24347) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-12 21:27:24 +08:00
Matthew Bonanni
d4fd2768ef
[Bugfix][Attention] Fix FlashInfer MLA block size logic (#24692) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-11 22:39:42 +00:00
Mengqing Cao
4f6593b058
[HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-09-11 21:47:58 +08:00
Hanjie Qiu
dcb28a332b
[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078) 
Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-10 15:31:10 -07:00
Thien Tran
a0933c3bd6
[Bugfix] Enable FP8 KV cache for FlashInfer and Triton backend on non-sm100 GPUs (#24577) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-09-10 12:33:41 -07:00
Lifans
d6069887c6
[rocm] enable torchao quantization for rocm (#24400) 
Signed-off-by: Lifan Shen <lifans@meta.com>
 2025-09-10 06:16:21 -07:00
baonudesifeizhai
6cbd41909e
Feature/vit attention unification# 23880 (#23978) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-10 06:10:14 -07:00
vllmellm
7c195d43da
[ROCm][Bugfix] Fix Aiter RMSNorm (#23412) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-09-10 21:08:03 +08:00
Ignacio Sica
3c2156b3af
[Hardware][Apple-CPU] Enable native bfloat16 on Apple Silicon (M2 and later) (#24129) 
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
 2025-09-10 03:50:21 +00:00
Wentao Ye
a55cf41a09
[Compilation][WideEP] Enable Piecewise CUDAGraph for DeepEPHT (#24123) 2025-09-09 10:21:10 -04:00
Woosuk Kwon
4172235ab7
[V0 deprecation] Deprecate V0 Neuron backend (#21159) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-06 16:15:18 -07:00
liuzhenwei
e599e2c65e
[XPU][P/D] Add XPU support in NixlConnector (#22436) 
Signed-off-by: zhenwei <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-04 21:03:12 -07:00
Kunshang Ji
16ded21eeb
[XPU] support Triton Attention backend on Intel GPU (#24149) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-04 20:41:08 +08:00
Lucas Wilkinson
402759d472
[Attention] FlashAttn MLA (#14258) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-04 02:47:59 -07:00
Matthew Bonanni
a742322092
[Attention] Blackwell FP8 MLA support with CUTLASS_MLA backend (#23289) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-03 14:05:24 -04:00
Chaojun Zhang
862f2ef893
[XPU] Fix the bug of LoRA logits on the XPU platform (#24081) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2025-09-03 08:21:18 +08:00
Yan Ma
7be0cb8e9e
[XPU][Feature] fp8 online quantization support for XPU (#23148) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Qiming Zhang <qiming1.zhang@intel.com>
 2025-09-02 04:06:53 +00:00
Didier Durand
9701352e4b
[Doc]: fix typos in Python comments (#24001) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-31 08:21:59 +00:00
Li, Jiang
ad39106b16
[CPU] Enable data parallel for CPU backend (#23903) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-08-29 02:19:58 -07:00