xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-04 21:23:32 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,043 Commits 155 Branches 89 Tags
Commit Graph

114 Commits

Author SHA1 Message Date
Lucas Wilkinson
b0d541947a
[Attention] Default to FlashMLA backend for MLA (#14451) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-08 18:18:39 -08:00
youkaichao
6eaf93020d
[platforms] improve rocm debugging info (#14257) 2025-03-04 21:32:18 -08:00
Tyler Michael Smith
72c62eae5f
[V1] EP/TP MoE + DP Attention (#13931) 2025-03-04 21:27:26 -08:00
Michael Goin
6247bae6c6
[Bugfix] Restrict MacOS CPU detection (#14210) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-03-04 22:25:27 +08:00
youkaichao
ac65bc92df
[platform] add debug logging during inferring the device type (#14195) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-04 18:39:16 +08:00
Cody Yu
989f4f430c
[Misc] Remove lru_cache in NvmlCudaPlatform (#14156) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-03-04 11:09:34 +08:00
Mengqing Cao
b87c21fc89
[Misc][Platform] Move use allgather to platform (#14010) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-03-03 15:40:04 +08:00
Woosuk Kwon
3b5567a209
[V1][Minor] Do not print attn backend twice (#13985) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-01 07:09:14 +00:00
Lucas Wilkinson
2e94b9cfbb
[Attention] Flash MLA for V1 (#13867) 
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yang Chen <yangche@fb.com>
 2025-02-27 23:03:41 +00:00
Yang Chen
58d1b2aa77
[Attention] MLA support for V1 (#13789) 
Signed-off-by: Yang Chen <yangche@fb.com>
 2025-02-27 13:14:17 -05:00
Lucas Wilkinson
f95903909f
[Kernel] FlashMLA integration (#13747) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-02-27 10:35:08 +08:00
cjackal
51010a1807
[Misc] set single whitespace between log sentences (#13771) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-02-25 10:26:12 +08:00
Alex Brooks
9621667874
[Misc] Warn if the vLLM version can't be retrieved (#13501) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
 2025-02-20 06:24:48 +00:00
Cyrus Leung
435b502a6e
[ROCm] Make amdsmi import optional for other platforms (#13460) 2025-02-18 03:15:56 -08:00
Divakar Verma
7c7adf81fc
[ROCm] fix get_device_name for rocm (#13438) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-02-18 04:07:12 +00:00
Yan Ma
30513d1cb6
[Bugfix] fix xpu communicator (#13368) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2025-02-17 20:59:18 +08:00
Mengqing Cao
238dfc8ac3
[MISC] tiny fixes (#13378) 2025-02-17 00:57:13 -08:00
Isotr0py
d67cc21b78
[Bugfix][Platform][CPU] Fix cuda platform detection on CPU backend edge case (#13358) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-02-16 18:55:27 +00:00
youkaichao
a0231b7c25
[platform] add base class for communicators (#13208) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-16 22:14:22 +08:00
Lily Liu
80f63a3966
[V1][Spec Decode] Ngram Spec Decode (#12193) 
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-02-15 18:05:11 -08:00
Alexander Matveev
45f90bcbba
[WIP] TPU V1 Support Refactored (#13049) 2025-02-14 00:21:53 -08:00
Sage Moore
ba59b78a9c
[ROCm][V1] Add intial ROCm support to V1 (#12790) 2025-02-13 22:21:50 -08:00
Li, Jiang
565c1efa65
[CI/Build][Bugfix] Fix CPU backend default threads num (#13077) 2025-02-11 16:55:56 +00:00
wangxiyuan
2e3b969ec0
[Platform] add pre_register_and_update function (#12432) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-02-11 22:06:46 +08:00
Gregory Shtrasberg
7539bbc6a6
[ROCm] Using a more precise memory profiling (#12624) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-02-11 21:47:10 +08:00
Russell Bryant
c320ca8edd
[Core] Don't do platform detection at import time (#12933) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-11 07:25:25 +00:00
மனோஜ்குமார் பழனிச்சாமி
2ae889052c
Fix seed parameter behavior in vLLM (#13007) 
Signed-off-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
 2025-02-10 23:26:50 +08:00
Yuan Tang
44607e07d3
Check if selected backend is None in get_attn_backend_cls() (#12975) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2025-02-10 11:45:07 +08:00
TJian
eaa92d4437
[ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation Per-Channel-Weight FP8 Quantization Inferencing (#12501) 2025-02-07 08:13:43 -08:00
youkaichao
0408efc6d0
[Misc] Improve error message for incorrect pynvml (#12809) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-06 15:23:50 +08:00
youkaichao
bc1bdecebf
[core][distributed] exact ray placement control (#12732) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-06 02:03:19 +08:00
Hongxia Yang
c36ac98d01
[AMD][ROCm] Enable DeepSeek model on ROCm (#12662) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
 2025-02-04 08:24:11 +00:00
youkaichao
ad4a9dc817
[cuda] manually import the correct pynvml module (#12679) 
fixes problems like https://github.com/vllm-project/vllm/pull/12635 and
https://github.com/vllm-project/vllm/pull/12636 and
https://github.com/vllm-project/vllm/pull/12565

---------

Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-03 15:58:21 +08:00
Russell Bryant
e489ad7a21
[Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Kunshang Ji
f256ebe4df
[Hardware][Intel GPU] add XPU bf16 support (#12392) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-02-02 10:17:26 +00:00
Lucas Wilkinson
cabaf4eff3
[Attention] MLA decode optimizations (#12528) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
 2025-01-30 23:49:37 -08:00
Robert Shaw
5f671cb4c3
[V1] Improve Error Message for Unsupported Config (#12535) 
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-29 04:56:56 +00:00
Harry Mellor
823ab79633
Update pre-commit hooks (#12475) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-27 17:23:08 -07:00
Konrad Zawora
96f6a7596f
[Bugfix] Fix HPU multiprocessing executor (#12167) 
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
 2025-01-23 02:07:07 +08:00
Kevin H. Luu
64ea24d0b3
[ci/lint] Add back default arg for pre-commit (#12279) 
Signed-off-by: kevin <kevin@anyscale.com>
 2025-01-22 01:15:27 +00:00
Mengqing Cao
c64612802b
[Platform] improve platforms getattr (#12264) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-01-21 14:42:41 +00:00
Işık
af69a6aded
fix: update platform detection for M-series arm based MacBook processors (#12227) 
Signed-off-by: isikhi <huseyin.isik000@gmail.com>
 2025-01-20 22:23:28 +00:00
youkaichao
2b83503227
[misc] fix cross-node TP (#12166) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-18 10:53:27 +08:00
kewang-xlnx
de0526f668
[Misc][Quark] Upstream Quark format to VLLM (#10765) 
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-15 11:05:15 -05:00
youkaichao
ad34c0df0f
[core] platform agnostic executor via collective_rpc (#11256) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-15 13:45:21 +08:00
Shanshan Shen
9ddac56311
[Platform] move current_memory_usage() into platform (#11369) 
Signed-off-by: Shanshan Shen <467638484@qq.com>
 2025-01-15 03:38:25 +00:00
Chen Zhang
a2d2acb4c8
[Bugfix][Kernel] Give unique name to BlockSparseFlashAttention (#12040) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-01-14 15:45:05 +00:00
Shanshan Shen
a7d59688fb
[Platform] Move get_punica_wrapper() function to Platform (#11516) 
Signed-off-by: Shanshan Shen <467638484@qq.com>
 2025-01-13 13:12:10 +00:00
youkaichao
458e63a2c6
[platform] add device_control env var (#12009) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-13 20:59:09 +08:00
youkaichao
89ce62a316
[platform] add ray_device_key (#11948) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-13 16:20:52 +08:00