xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-01-23 16:14:37 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,680 Commits 155 Branches 89 Tags
Commit Graph

342 Commits

Author SHA1 Message Date
Lucas Kabela
bc997c18ca [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Nick Hill
d55c6010ac [BugFix] Fix async scheduling CPU tensor race take 2 (#25279) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:53 -07:00
Jee Jee Li
2821986450
[Core] Modify the initialization parameters of the lora manager (#25249) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-19 18:01:28 +00:00
Andrew Sansom
9a4600e4dc
[CORE] Prompt Embeddings Support for v1 Engine (#24278) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-19 08:03:09 +08:00
Aziz
38db529f66
[feat]: Create interface for model-specific M-RoPE (#24194) 
Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-18 19:18:56 +00:00
Shanshan Shen
470484a4f5
[Structured Output][Refactor] Move apply_grammar_bitmask() method from ModelRunner to structured output utils (#21999) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-09-18 20:44:31 +08:00
Jee Jee Li
37970105fe
[Model] Improve Pooling Model (#25149) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-18 11:04:21 +00:00
Benjamin Chislett
b7433ca1a4
[Spec Decode] Efficient padded speculation (#24539) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-09-18 01:07:24 -04:00
Russell Bryant
58d4c705a8
[Core] Get num_encoder_tokens from scheduler config (#24989) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-09-16 20:59:07 -07:00
Nick Hill
eeb135eb87
[Core] Use CpuGpuBuffer for block table tensors (#24795) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-16 19:18:06 -07:00
Sage Moore
567939953b
[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
cascade
17871983a2
[Bugfix] Fix sequence parallelism bug when enable pipeline parallelism (#24021) 
Signed-off-by: cascade812 <cascade812@outlook.com>
 2025-09-16 04:32:32 +00:00
Woosuk Kwon
3e903b6cb4
[Chore] Minor simplification for non-PP path (#24810) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-13 17:41:36 -07:00
Didier Durand
bcb06d7baf
[Doc]: fix typos in various files (#24726) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-12 06:43:12 -07:00
Flora Feng
0377802c20
[Multimodal] Remove legacy multimodal fields in favor of MultiModalFeatureSpec (#24548) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-12 21:42:23 +08:00
Boyuan Feng
94e6b2d55f
Allow users to specify kv cache memory size (#21489) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 13:41:07 +00:00
Tao He
e93f4cc9e3
Add the support for the qwen3 next model (a hybrid attention model). (#24526) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 15:32:09 +08:00
Nick Hill
e2d8c27f68
[BugFix] Fix pipeline parallel (#24621) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-10 23:05:30 -07:00
Michael Goin
fba7856581
[Perf] Warmup FlashInfer attention during startup (#23439) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-10 15:03:17 -07:00
Russell Bryant
37e8182bfe
[v1] Add Whisper model support (encoder-decoder) (#21088) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2025-09-10 13:53:35 -07:00
Nick Hill
f4f1a8df22
[BugFix] Ensure integrity of reused CPU tensors during async scheduling (#24527) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: guoze.lin <guozelin@tencent.com>
 2025-09-10 21:15:14 +08:00
Lucas Wilkinson
0ae43dbf8c
[Attention] add DCP support for FLASH_ATTN_MLA backend (#24453) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-10 17:19:26 +08:00
Micah Williamson
1c63a16b65
[Core] Run garbage collector after CUDA graph capture to fix throughput regression (#24128) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-09 10:38:10 -04:00
Woosuk Kwon
2e5d21378d
Skip MM Encoder for non-first PP ranks (#24387) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-07 09:38:35 -07:00
youkaichao
558f0907dc
[attention][DCP] use AttentionImpl.need_to_return_lse_for_decode (#24372) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-07 01:18:59 +00:00
Bangsheng Tang
848562bd49
break execute_model in gpu_model_runner into sub-functions for custom scopes (#24265) 
Co-authored-by: Bangsheng Tang <bangsheng@meta.com>
 2025-09-06 14:02:47 -07:00
Andrew Sansom
305a1cc0d2
refactor: Turn GPUModelRunner.inputs_embeds to a CpuGpuBuffer (#24345) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-09-05 23:01:23 -07:00
yzds
ac201a0eaf
[Feature] Support Decode Context Parallel (DCP) for MLA (#23734) 
Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-09-06 13:24:05 +08:00
Didier Durand
35bf193864
[Doc]: fix typos in Python comments (#24294) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-05 19:41:12 -07:00
Benjamin Chislett
cee182b297
[Perf][V1] Fully overlap model execution (#23569) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
 2025-09-05 18:20:17 -07:00
liuzhenwei
e599e2c65e
[XPU][P/D] Add XPU support in NixlConnector (#22436) 
Signed-off-by: zhenwei <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-04 21:03:12 -07:00
co63oc
1bd007f234
fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
Russell Bryant
e32a0e8678
Upgrade xgrammar to 0.1.23 (#22988) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-09-03 02:32:59 +00:00
Maximilien de Bayser
d59c986444
Remove runtime checks based on pooling params (#24051) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-09-02 11:54:37 +08:00
Woosuk Kwon
b55713683c
[Misc] Move fast prefill logic to separate method (#24013) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-01 05:40:38 +00:00
Woosuk Kwon
8c742a66d1
[Misc] Avoid redundant copy for encoder-only models (#24012) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-01 04:02:43 +00:00
Didier Durand
9701352e4b
[Doc]: fix typos in Python comments (#24001) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-31 08:21:59 +00:00
Andy Lo
038e9be4eb
[LoRA] Much faster startup when LoRA is enabled (#23777) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-30 15:37:39 +00:00
wang.yuqi
d9e00dbd1f
[Performance] V1 Classify Models E2E Performance Optimization (#23541) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-29 03:12:32 -07:00
Yong Hoon Shin
cb293f6a79
[V1] Enable prefill optimization for Gemma3n (#22628) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-28 14:54:30 -07:00
Wentao Ye
321938e9ac
[Feature] Add VLLM_DISABLE_PAD_FOR_CUDAGRAPH to Avoid Hang Issue (#23595) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-27 21:52:24 +00:00
Hyogeun Oh (오효근)
4e4d017b6f
[Docs] Fix warnings in mkdocs build (continued) (#23743) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
 2025-08-27 17:17:29 +00:00
Cyrus Leung
52883ed084
[Model] Merge SupportsMultiModalWithRawInput with SupportsMultiModal (#23749) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-27 10:01:50 -07:00
Woosuk Kwon
04ff1e43fb
[Misc] Move CpuGpuBuffer to vllm/v1/utils.py (#23728) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-27 03:25:00 -07:00
Cyrus Leung
69244e67e6
[Core] Use key-only cache for BaseMultiModalProcessor (#23018) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-27 14:19:13 +08:00
Li, Jiang
9b0187003e
[Bugfix] Fix cuda event usage with CPU model runner (#23643) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-08-26 17:10:42 +00:00
Chen Zhang
2b4fc9bd9b
Support FlashAttention Backend for Hybrid SSM Models (#23299) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-26 12:41:52 +00:00
Zijing Liu
b395b3b0a3
[Disagg][Perf] Use CUDA event sync instead of blocking tolist to avoid unintentional copy ops blocking across different CUDA streams, improving disagg TTIT/TTFT (#22760) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Zijing Liu <liuzijing2014@users.noreply.github.com>
 2025-08-25 21:06:00 -07:00
Woosuk Kwon
0ff902f3b4
[Refactor] Refactor persistent buffers with CpuGpuBuffer (#23515) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-25 08:44:48 -07:00
Ayush Satyam
5c4b6e66fe
[Attention] Unify mamba and attention backend selection (#23171) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-08-25 09:09:36 +00:00