xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-09 02:27:09 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,217 Commits 155 Branches 89 Tags
Commit Graph

730 Commits

Author SHA1 Message Date
Alex Brooks
ead2110297
[Core][Bugfix] Fix Online MM Beam Search (#19688) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-06-19 17:18:07 +00:00
Maximilien de Bayser
799397ee4f
Support embedding models in V1 (#16188) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-18 21:36:33 -07:00
Chen Zhang
a89209b78d
[v1] Support mamba2 (#19327) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-06-18 20:34:15 +00:00
wangxiyuan
257ab95439
[Platform] Allow platform use V1 Engine by default (#19792) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-06-18 13:03:36 +00:00
Ning Xie
6e9cc73f67
[MISC] correct DeviceConfig device field static type analysis (#19699) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-17 17:21:50 -07:00
Li, Jiang
6458721108
[CPU] Refine default config for the CPU backend (#19539) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-06-13 13:27:39 +08:00
rasmith
c7ea0b56cd
[AMD] [Quantization] Add override flag for attention dtype instead of using kv_cache_dtype trigger (#17331) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-06-11 15:53:28 -04:00
Cyrus Leung
68b4a26149
[Doc] Update V1 User Guide for Hardware and Models (#19474) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-11 00:49:06 -07:00
Gregory Shtrasberg
5241ca50d6
[ROCm][V1] Adding ROCm to the list of plaforms using V1 by default (#19440) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-06-10 22:06:15 +00:00
Isotr0py
5f1ac1e1d1
Revert "[v1] Add fp32 support to v1 engine through flex attn" (#19404) 2025-06-10 01:30:20 -07:00
22quinn
c1c7dbbeeb
[Bugfix][Core] Prevent token lengths exceeding max_model_len in V0 (#19348) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-09 23:01:29 +08:00
Isotr0py
b8089195b4
[v1] Add fp32 support to v1 engine through flex attn (#19319) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-06-09 22:10:44 +08:00
Cyrus Leung
e31ae3de36
[Deprecation] Remove inputs arg fallback in Engine classes (#18799) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-09 10:19:56 +08:00
Driss Guessous
cf02f9b283
Add FlexAttention to V1 (#16078) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-06 21:58:55 -07:00
jmswen
7353492a47
[Core] Raise when non-multi-instance DP clients target a DP rank (#19227) 
Signed-off-by: Jon Swenson <jmswen@gmail.com>
 2025-06-06 19:03:01 +08:00
Chen Zhang
f8a1a2d108
[v1] Hybrid Memory Allocator (#17996) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-06-05 20:47:09 -07:00
Kebe
ef3f98b59f
[Bugfix] fix v1 cpu worker fails on macOS (#19121) 2025-06-04 20:17:38 +00:00
jmswen
c8dcc15921
Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) 
Signed-off-by: Jon Swenson <jmswen@gmail.com>
 2025-06-04 08:26:47 -07:00
Kaixi Hou
41aa578428
[NVIDIA] Add Cutlass MLA backend (#17625) 2025-06-03 21:40:26 -07:00
Li, Jiang
4555143ea7
[CPU] V1 support for the CPU backend (#16441) 2025-06-03 18:43:01 -07:00
Harry Mellor
6865fe0074
Fix interaction between Optional and Annotated in CLI typing (#19093) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Yikun Jiang <yikun@apache.org>
 2025-06-03 21:07:19 +00:00
Simon Mo
02f0c7b220
[Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Reid
17430e3653
[bugfix] small fix logic issue (#18999) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-03 05:35:12 +00:00
Rui Qiao
bdce64f236
[V1] Support DP with Ray (#18779) 2025-06-02 21:15:13 -07:00
jennyyyyzhen
ebb1ec9318
[Model] enable data parallel for Llama4 vision encoder (#18368) 
Signed-off-by: yzhen <yzhen@devgpu093.cco2.facebook.com>
Co-authored-by: yZhen <yZhen@fb.com>
Co-authored-by: yzhen <yzhen@devgpu093.cco2.facebook.com>
 2025-06-02 19:22:54 +08:00
Michael Goin
2ad6194a02
Let max_num_batched_tokens use human_readable_int for large numbers (#18968) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-01 11:41:29 +08:00
Michael Goin
d54af615d5
[Bugfix] Fix PP default fallback behavior for V1 (#18915) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-30 10:13:17 +08:00
Mark McLoughlin
0e98964e94
[V1][Metrics] Remove metrics that were deprecated in 0.8 (#18837) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-28 18:54:12 +00:00
Alex Brooks
321331b8ae
[Core] Add Lora Support to Beam Search (#18346) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-05-28 08:58:24 -07:00
Harry Mellor
4c2b38ce9e
Enable Pydantic mypy checks and convert configs to Pydantic dataclasses (#17599) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-28 12:46:04 +00:00
Cyrus Leung
0f0926b43f
[Deprecation] Remove unused sync methods in async_timeout (#18792) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-28 15:08:48 +08:00
Hyogeun Oh (오효근)
a68e293cb9
[Doc] Convert Sphinx directives ( {class}, {meth}, {attr}, ...) to MkDocs format for better documentation linking (#18663) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-05-27 01:44:20 -07:00
Isotr0py
1f1b1bc03b
[V1][Quantization] Add CUDA graph compatible v1 GGUF support (#18646) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-27 04:40:28 +00:00
Yuqi Zhang
f2faac745d
[Bugfix] Fix cpu usage and cache hit stats reporting on cpu environment (#18674) 
Signed-off-by: zzzyq <zhangyuqi94@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-05-25 02:36:06 -07:00
Feng XiaoLong
4fc1bf813a
[Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) 
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
 2025-05-23 16:16:26 -07:00
Cyrus Leung
7d9216495c
[Doc] Update references to doc files (#18637) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-23 15:49:21 -07:00
Jiayi Yao
2628a69e35
[V1] Support Deepseek MTP (#18435) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-05-23 10:26:28 -07:00
Cyrus Leung
273cb3b4d9
[Doc] Fix top-level API links/docs (#18621) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-23 09:46:56 -07:00
Harry Mellor
a1fe24d961
Migrate docs from Sphinx to MkDocs (#18145) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 02:09:53 -07:00
Sanger Steel
c32e249a23
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
 2025-05-22 18:44:18 -07:00
Hyogeun Oh (오효근)
2b16104557
[Misc] Update deprecation message for --enable-reasoning (#18404) 2025-05-21 07:33:11 -07:00
Kebe
5d7f545204
[Frontend] deprecate --device arg (#18399) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-05-21 01:21:17 -07:00
Woosuk Kwon
fabe89bbc4
[Spec Decode] Don't fall back to V0 when spec decoding is enabled (#18265) 2025-05-16 16:10:27 -07:00
Lucia Fang
3d2779c29a
[Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) 
Signed-off-by: Lucia Fang <fanglu@fb.com>
 2025-05-15 22:28:27 -07:00
Harry Mellor
b18201fe06
Allow users to pass arbitrary JSON keys from CLI (#18208) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-15 21:05:34 -07:00
Sky Lee
f4937a51c1
[Model] vLLM v1 supports Medusa (#17956) 
Signed-off-by: lisiqi23 <lisiqi23@xiaomi.com>
Signed-off-by: skylee-01 <497627264@qq.com>
Co-authored-by: lisiqi23 <lisiqi23@xiaomi.com>
 2025-05-15 21:05:31 -07:00
Nick Hill
55aa7af994
[V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-05-13 10:48:21 -07:00
Cyrus Leung
61e0a506a3
[Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-12 22:40:19 -07:00
Tao He
60f7624334
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844) 2025-05-12 19:52:47 -07:00
Reid
009b3d5382
[Misc] not show --model in vllm serve --help (#16691) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-11 08:47:58 +00:00