xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-10 11:07:14 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,464 Commits 155 Branches 89 Tags
Commit Graph

1893 Commits

Author SHA1 Message Date
Nicolò Lucchesi
0efd9f867c
[Core] Whisper Enable Encoder Batching (#29421) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-12-11 21:06:51 +00:00
Xingyu Liu
90d6cf921f
[BugFix][MM]support VLLM_RANDOMIZE_DP_DUMMY_INPUTS (#30472) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-11 21:00:15 +00:00
Andreas Karatzas
72aaac5b66
[ROCm][Bugfix] Add MLACommonMetadata to allowed attention types for speculative decoding (#30430) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2025-12-11 19:25:01 +00:00
Martin Hickey
f4417f8449
[KVConnector] Add KV events to KV Connectors (#28309) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2025-12-11 15:30:29 +01:00
Qiu
a11f4a81e0
[Misc][PCP&DCP] relocate PCP feature check (#30050) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-11 03:36:18 -08:00
Ming Yang
fba8906930
[perf] Use direct copy (broadcast) instead of cat for k_nope/k_pe in MLA prefill (#29710) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-12-11 08:20:45 +00:00
Wentao Ye
d6464f2679
[Chore] Fix torch precision warning (#30428) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-11 04:05:56 +00:00
shivampr
8580919ac3
[Bugfix] fix confusing OOM errors during v1 init (#28051) 
Signed-off-by: Shivam <shivamprasad91@gmail.com>
Signed-off-by: shivampr <shivampr.dev@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-12-10 23:17:41 +00:00
Po-Han Huang (NVIDIA)
eea41804a4
[bug] Fix "Current vLLM config is not set." warnings when FlashInfer attention is used (#30241) 
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
 2025-12-10 11:18:51 -08:00
Jialin Ouyang
9f042ba26b
[Perf] Enable environment cache in EngineCore to enable the feature for UniProcExecutor as well (#29289) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-12-10 14:13:01 -05:00
Cyrus Leung
e72d65b959
{Deprecation] Remove tokenizer setter (#30400) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:10:58 +00:00
Will Eaton
a9e4106f28
[P/D] KV Load Failure Recovery/Abort Configuration (#26813) 
Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Will Eaton <me@wseaton.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-12-10 11:00:52 -08:00
Cyrus Leung
253305d5b2
[Chore] Delay recent deprecations (#30398) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 17:48:38 +00:00
Nicolò Lucchesi
c756fb6781
[Core] Whisper enable FULL_DECODE_ONLY CudaGraph (#30072) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-12-10 06:14:24 -08:00
Aditya Tewari
cebda2a4af
[CPU] Support for Whisper (#30062) 
Signed-off-by: Aditya Tewari <aditya.tewari@arm.com>
 2025-12-10 04:58:42 -08:00
Daniele
53d2420b44
[Bugfix] tpu_model_runner: set vllm config context when calling reset_dynamo_cache() (#30331) 
Signed-off-by: Daniele Trifirò <dtrifiro@redhat.com>
 2025-12-10 04:58:35 -08:00
Chauncey
9db78f34dc
[Bugfix] Fix the issue where DeepSeek v3.2 cannot use structured_output (#30371) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-12-10 08:30:16 +00:00
Wilson Wu
3bdd426636
Fix typos in comments across multiple files (#30345) 
Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-09 20:05:28 -08:00
Lucas Wilkinson
abe93bce59
[Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable true async spec-decode (#29624) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-12-09 17:18:10 -08:00
dongbo910220
03b5f940fd
[V1][Spec Decode] Optimize Medusa proposer to avoid GPU-CPU sync (#29723) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-12-10 00:15:01 +00:00
Lucas Wilkinson
95501a70ec
[BugFix] Fix DeepSeek-R1 hang with DP and MTP (#30119) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-12-09 18:51:19 +00:00
Benjamin Chislett
e858bfe051
[Cleanup] Refactor profiling env vars into a CLI config (#29912) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-09 13:29:33 -05:00
Woosuk Kwon
d471b2aff0
[Model Runner V2] Support num NaNs in logits (#30187) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-12-09 10:00:49 -08:00
Woosuk Kwon
9e6562a3f6
[Model Runner V2] Fix Triton warning on tl.where (#30355) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-12-09 09:59:54 -08:00
Wentao Ye
83319b44c2
[Compile] Fix torch warning TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled (#29897) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-09 10:40:37 -05:00
Lucas Wilkinson
56037dfa2f
[BugFix] Fix assert batch_descriptor.num_tokens == num_tokens_padded (#30173) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-09 10:36:12 -05:00
Hubert de La Jonquiere
c72ea10723
[Structured Output][Reasoning] Improves decoding throughput for models using single-token reasoning endings. (#30056) 2025-12-09 18:54:08 +08:00
Jaya Yuan
67475a6e81
[DCP][Bugfix][CI] Fix accuracy issue of DCP when using FLASH_ATTN_MLA (#30309) 
Signed-off-by: FENP <yuanyongjie.yyj@antgroup.com>
 2025-12-09 08:22:14 +00:00
Lucas Wilkinson
aed846917f
[Attention] Make split_decodes_and_prefills(..., require_uniform=True) support padding (#29644) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-12-09 07:24:01 +00:00
Victor Ziliang Peng
f1599ca55d
feat(metrics): Add prefill KV compute metric excluding cached tokens (#30189) 
Signed-off-by: Ziliang Peng <ziliang@character.ai>
 2025-12-09 00:08:48 +00:00
Lain
1fb632fdb6
[Perf] Improve fp8 quant in mla; replace ReduceSum with ReduceScatterSum (#29795) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
 2025-12-08 15:02:34 -08:00
Nick Hill
d726a7b0ed
[BugFix] Unblock use of LoRA with data parallel mode (#30220) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-12-08 12:21:05 +08:00
Isotr0py
b952f4d3c3
[v1] Add PrefixLM support to FlexAttention backend (#27938) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-07 15:51:36 +00:00
Yifan Qiao
1b0482b9d1
[Misc][Core] Remove unused req_index increment in scheduler (#30176) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
 2025-12-07 08:39:21 +00:00
Cyrus Leung
e83b7e379c
Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) 2025-12-07 00:00:22 -08:00
Cyrus Leung
27f4c2fd46
[Renderer] Separate out RendererConfig from ModelConfig (#30145) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 23:15:42 -08:00
Cyrus Leung
671427efbf
[Model] Move multimodal_cpu_fields definition to field config (#30181) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 13:40:02 +00:00
Woosuk Kwon
a238cbd89d
[Model Runner V2] Support min-p sampling (#30171) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-12-05 21:42:47 -08:00
Tova Movshovitz
adb315060c
[KVConnector][Feature] Support KV connector cache reset via /reset_prefix_cache (#27170) 
Signed-off-by: tovam <tovam@pliops.com>
Signed-off-by: Tova Movshovitz <tovam@pliops.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-05 18:33:26 +00:00
Ilya Markov
4e26d3b09e
[Compile] Conditional compilation. Introduce compile_ranges (#24252) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: ProExpertProg <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
 2025-12-05 18:17:32 +00:00
Matthew Bonanni
66e674cdd5
[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments (#26315) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-12-05 09:48:43 -08:00
Nick Hill
dc264bcea1
[BugFix] Eagerly abort cancelled final-step requests (#29987) 
Currently, when requests are cancelled while executing their final
step, "completion" is handled based on normal stop processing (e.g.
length or stop token), so the abort has no effect. This is typically
not a problem, but when a kv connector is involved it thinks the
request completed successfully rather than being aborted.

This is problematic for disaggregated prefill which will free kv
cache blocks if the request was aborted but not if it completed
successfully—since the cancelled request will never be sent to
the decode side, kv cache blocks remain pinned until the fall-back
timeout expires. The problem is exacerbated when many requests
are cancelled and/or there are large prefills whose forward pass
takes a long time (since the window is bigger).

This PR fixes the problem by processing pending aborts
immediately prior to processing model output each step; we process
only aborts, not new requests, since it's preferable for latency to
process model outputs before new incoming requests.

Fixes #26400.

Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-12-05 17:28:32 +00:00
Max Hu
c2894d3883
[Feature] Add Layer-wise NVTX Support (#29990) 
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Signed-off-by: Max Hu <maxhu@nvidia.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>
 2025-12-05 11:20:07 +00:00
Alec S
65ee97288a
[BugFix] Adding env variable to disable async grammar compilation (#29996) 
Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-12-05 00:49:37 -08:00
Jingchun Gao
d698bb382d
[Bugfix] Correct num_q_heads on DCP for Flashinfer backends (#29487) 
Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com>
Signed-off-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com>
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com>
 2025-12-05 05:54:31 +00:00
Lucas Wilkinson
c8ab988b15
[BugFix] Fix DBO assert assert B_block_table == B_q (#29933) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-04 14:48:54 -05:00
Cyrus Leung
b286a311c2
[Chore] Deprecate merge_by_field_config arg (#30035) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-04 17:21:24 +00:00
Woosuk Kwon
cc050558f4
[Model Runner V2] Implement get_num_sampled_and_rejected kernel (#30029) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-12-04 07:19:42 -08:00
wang.yuqi
74c4d80c6c
[Model][6/N] Improve all pooling task | Support chunked prefill with ALL pooling (#27145) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-04 13:44:15 +00:00
Andreas Karatzas
e96a6a6dca
[ROCm][CI][Bugfix] Fixing the Multi-Modal Models Test (Extended) 1 group (#30013) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2025-12-04 11:00:16 +00:00