xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-26 04:57:09 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,356 Commits 155 Branches 89 Tags
Commit Graph

420 Commits

Author SHA1 Message Date
TJian
d0fb572929
[ROCm] [AITER] [DOC] Add usage description about check functions in _aiter_ops (#30586) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-12-16 13:50:47 +00:00
Isotr0py
ec154c36ee
[Platform] Refactor Platform attention backend selection to avoid breakpoint for OOT platform (#30212) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 17:36:07 +00:00
Shanshan Shen
87b4d1557d
[CustomOp][MM] Extract MMEncoderAttention as CustomOp and replace the backend of QwenVisionAttention with it. (#30125) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-12-15 11:13:32 +08:00
Wentao Ye
6e78ed6ba7
[Logs] Optimize startup logs 4 (#29903) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-13 16:12:53 -05:00
Roberto L. Castro
4fa7ce46f3
[Feature] Add SM103 (Blackwell Ultra) Support to vLLM (#30484) 
Signed-off-by: LopezCastroRoberto <robertol.c510@gmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-12 19:34:23 -08:00
Fadi Arafeh
f355ad5412
[CPU][FIX] Fix build failures on Arm CPUs with torch nightly (#30481) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-12-12 02:09:25 +00:00
Andreas Karatzas
b51255f369
[ROCm] Fix broken import in platform attention backend dispatching (#30432) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2025-12-11 01:12:58 +00:00
vllmellm
ee14644ba9
[ROCm] Aiter Quant Kernels (#25552) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-12-09 14:27:37 +00:00
Lucas Wilkinson
0044c4038c
[BugFix][DeepSeek-V3.2] Fix backend selection logic for Blackwell (#30195) 2025-12-07 10:53:51 -05:00
Isotr0py
b952f4d3c3
[v1] Add PrefixLM support to FlexAttention backend (#27938) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-07 15:51:36 +00:00
Wentao Ye
17eb25e327
[Perf] Enable cuda graph for deepepHT, 5.3% throughput improvement, 4.4% TTFT improvement (#29558) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 04:44:50 +00:00
Matthew Bonanni
66e674cdd5
[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments (#26315) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-12-05 09:48:43 -08:00
Qiu
0098a6e3da
[PCP&DCP] move CUDAGraph check for PCP&DCP to the check func of platforms (#29952) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-04 21:40:51 -05:00
gausah01
28097d5638
[Bugfix][CPU] Fix CPU KV cache fallback memory allocation (#29604) 
Signed-off-by: Gauri Sahnan <gauri.sahnan@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2025-12-04 13:01:15 +08:00
Li, Jiang
e2f56c309d
[CPU] Update torch 2.9.1 for CPU backend (#29664) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-28 13:37:54 +00:00
Isotr0py
38658ec6f3
[Bugfix][MM encoder] Fix ViT attention backend resolving for Turing GPU (#29614) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-27 19:17:37 +00:00
Matthew Bonanni
fc1d8be3dc
[Attention] Update attention imports (#29540) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-27 11:19:09 -05:00
Johnny Yang
3ecabd06ee
Fix tpu-inference platform path (#29554) 
Signed-off-by: Johnny Yang <johnnyyang@google.com>
 2025-11-26 23:25:21 -08:00
Johnny Yang
ba1fcd84a7
[TPU] add tpu_inference (#27277) 
Signed-off-by: Johnny Yang <johnnyyang@google.com>
 2025-11-26 14:46:36 -08:00
Matthew Bonanni
430dd4d9eb
[Attention] Remove imports from vllm/attention/__init__.py (#29342) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-26 10:53:15 -07:00
Michael Goin
dbc3d9991a
[UX] Put CUDA attention backend selection log into one line (#29337) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-25 06:46:18 -08:00
Micah Williamson
ef1f7030f0
[ROCm][CI] Fix test_cudagraph_mode failure in AMD CI (#29367) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2025-11-25 07:55:09 +00:00
Icey
888152bf87
Allow oot custom compiler extension via CompilerInterface (#28623) 
Signed-off-by: wxsIcey <1790571317@qq.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Icey <1790571317@qq.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
 2025-11-25 15:25:15 +08:00
vllmellm
64deead719
[Bugfix] [ROCm] [UX]: revert Flex attention backend (#29371) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-11-25 06:56:06 +00:00
vllmellm
e48b2e6848
[Bugfix] [ROCm] [UX] Reorganize ROCm Backend Selection Logic (#26980) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-11-24 15:24:49 +00:00
Roger Wang
0ff70821c9
[Core] Deprecate xformers (#29262) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-24 04:18:55 +00:00
Chendi.Xue
460d02a417
[NIXL] Fix after virtual block_size for host_buffer with heter kv_layout (#29122) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-11-21 08:55:27 -08:00
Matthew Bonanni
11857a00b0
[Attention] Add ROCM_AITER_MLA_SPARSE to attention backend registry (#29103) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-20 20:24:43 -08:00
Pleaplusone
06c20c9904
[ROCm] Add AMD GPU support on Deepseek v3.2 and SparseMLA (#26670) 
Signed-off-by: ganyi <ygan@amd.com>
 2025-11-20 02:54:01 -08:00
Aleksandr Malyshev
ac10fd3c69
Upstreaming aiter triton attention backend as a new backend (#28701) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
 2025-11-19 19:59:30 +00:00
Matthew Bonanni
4c23690f43
[Attention] FlashAttention ViT support, make default backend (#28763) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-18 20:06:21 -08:00
Strahinja Stamenkovic
814843e021
Enable bitsandbytes quantization on AMD GPUs that use warp size 32 (#27307) 
Signed-off-by: sstamenk <strahinja.stamenkovic@amd.com>
 2025-11-19 03:12:31 +00:00
Cyrus Leung
638e4196d1
[Misc] Make SchedulerConfig.max_model_len init-only (#28733) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-15 01:59:31 -08:00
QiliangCui
9fc81ec765
[TPU] Fix import error in tpu launch (#28758) 
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
 2025-11-15 00:58:32 +00:00
Cyrus Leung
e2741f6cbc
[Chore] Rename SchedulerConfig.chunked_prefill_enabled (#28735) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-14 18:39:57 +00:00
Cyrus Leung
511a6b611d
[Config] Clean up SchedulerConfig initialization (#28665) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-14 22:41:02 +08:00
Huamin Li
07a606aa7e
[CI Failure] Fix backend selection for encoder-only models (#28534) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
 2025-11-13 10:11:27 -05:00
Fanli Lin
dbbe0c756a
[XPU] Support Triton path for LoRA operations on XPU (#28511) 
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
 2025-11-13 05:31:42 +00:00
wangxiyuan
2dacd57394
[platform] Move get_cu_count to utils (#27005) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-13 08:48:47 +08:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟
4ca5cd5740
[Core][AMD] Migrate fully transparent sleep mode to ROCm platform (#12695) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
 2025-11-12 15:24:12 -08:00
vllmellm
d8140b9833
[ROCM] Fix ROCm warnings, environment flag access, and GEMM kernel naming for consistency in _aiter_ops.py (#28464) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-11-12 21:46:57 +00:00
Harry Mellor
54aecd9ed5
Fix pre-commit (and XPU) on main (#28556) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-12 06:13:41 -08:00
wangxiyuan
10138c92a5
[V0 deprecation] Deprecate use_v1 parameter (#28112) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-12 14:03:52 +00:00
Chaojun Zhang
a4730c1b4f
[XPU]Fix crash due to removed VLLM_USE_V1 attribute (#28520) 
Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com>
 2025-11-12 10:20:55 +00:00
Andreas Karatzas
9f0247cfa4
VLLM_USE_TRITON_FLASH_ATTN V0 variable deprecation (#27611) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2025-11-11 18:34:36 -08:00
Li, Jiang
7f829be7d3
[CPU] Refactor CPU attention backend (#27954) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-12 09:43:06 +08:00
Kyuyeun Kim
df4d3a44a8
[TPU] Rename path to tpu platform (#28452) 
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
 2025-11-11 19:16:47 +00:00
Matthew Bonanni
684f254585
Prefer FlashAttention MLA as default over FlashMLA (#27363) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-11 17:13:51 +00:00
Matthew Bonanni
b30dfa03c5
[Attention] Refactor CUDA attention backend selection logic (#24794) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-11-11 07:40:44 -05:00
vllmellm
f080a83511
[RFC][ROCm][AITER] Keep all AITER kernels in _aiter_ops class like _custom_ops and _ipex_ops (#24490) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-11-10 08:20:53 -08:00