xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-24 15:06:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,282 Commits 155 Branches 89 Tags
Commit Graph

12282 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Isotr0py
e94384bbad
[Bugfix] Fix broken ViT attention selection for Blackwell device (#30731) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-16 05:24:32 +00:00
jiangkuaixue123
b9ff4f2a8d
[feature] extend DBO to XBO (#30120) 
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2025-12-16 00:04:01 -05:00
Boyuan Feng
c881db364e
improve lazy import test (#30733) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-12-16 03:12:05 +00:00
Shanshan Shen
3bd9c49158
[CustomOp] Extract ApplyRotaryEmb as CustomOp and unify the dispatch logic (#29873) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-12-15 19:08:16 -08:00
Amr Mahdi
ff21a0fc85
[docker] Restructure Dockerfile for more efficient and cache-friendly builds (#30626) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2025-12-15 18:52:19 -08:00
penfree
bbd850e597
[Bugfix] fix streaming final output for non harmony (#30237) 
Signed-off-by: penfree <qiupengfei@baidu.com>
Co-authored-by: penfree <qiupengfei@baidu.com>
 2025-12-16 09:03:11 +08:00
Shengqi Chen
511e81e7c9
[BUILD] use sm_100f when compiling flashmla to fix support on sm103 (#30705) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2025-12-15 14:48:01 -08:00
Matthew Bonanni
a182be4308
[UX][Attention] Add attention_config argument to LLM() (#30710) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-12-15 17:29:09 -05:00
Kevin Musgrave
c01d589813
[Benchmarks] auto_tune.sh: Use hostname variable for server requests (#30529) 
Signed-off-by: Kevin Musgrave <kevin.musgrave@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 22:00:29 +00:00
Matthew Bonanni
60dbf7d8f1
Update batch invariant to use attention config (#30704) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 15:24:16 -05:00
Michael Goin
a450c64a30
[Bugfix] Fail instead of ignoring when CompilationConfig gets invalid args (#30708) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-15 20:18:02 +00:00
Fadi Arafeh
b2191abdca
[docs][fix] Update Arm CPU vLLM wheel installation docs (#30594) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-12-15 19:46:25 +00:00
Matthew Bonanni
51e5b3e3c4
[Bugfix] Fix ViT with FlashAttention on ROCm (#30703) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-12-15 19:45:21 +00:00
Isotr0py
ec154c36ee
[Platform] Refactor Platform attention backend selection to avoid breakpoint for OOT platform (#30212) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 17:36:07 +00:00
Harry Mellor
970713d4a4
Remove SkipValidation from ModelConfig (#30695) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-15 17:34:08 +00:00
mondaylord
17fec3af09
[Bugfix] Fix missing first token in tool calls during reasoning-to-tool transition (#30671) 
Signed-off-by: mondaylord <20212010046@fudan.edu.cn>
 2025-12-15 16:13:37 +00:00
yjc9696
855b101d75
[Frontend] add tools for dsv32 developer role (#30040) 
Signed-off-by: pridejcyang <pridejcyang@tencent.com>
Co-authored-by: pridejcyang <pridejcyang@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-15 15:08:47 +00:00
Robert Shaw
d0502b4928
[MoE][Refactor 1/N] Separate Online Quantization (#30627) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-15 06:54:53 -08:00
Max Hu
3f175f18a2
[Bugfix] Fix multimodal configuration for Qwen3VL MOE model (#30670) 
Signed-off-by: Max Hu <hyoung2991@gmail.com>
 2025-12-15 14:06:01 +00:00
Cyrus Leung
ed586e7724
[Refactor] [3/N] Move tool parser tests and run on CPU (#30693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-15 13:45:36 +00:00
Chauncey
2a1776b7ac
[Refactor] [2/N] Move tool parsers into the vLLM main directory (#30675) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-12-15 12:54:52 +00:00
Nicolò Lucchesi
185c22bf2f
[Misc][Hybrid allocator + kv connector] Optionally enable hybrid allocator + KV cache connector (#29805) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-12-15 11:17:58 +00:00
duke
e4806d973a
[BugFix] Add embed_input_ids method to make QWenLMHeadModel a vllm model (#30674) 
Signed-off-by: root <iwzbi@zju.edu.cn>
Co-authored-by: root <iwzbi@zju.edu.cn>
 2025-12-15 10:38:29 +00:00
wang.yuqi
4429d934de
[Model] Automatic conversion of TokenClassification model (#30666) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-15 08:13:00 +00:00
ゆり
33278073d6
typing: Add type hints to TurnMetrics class in context.py (#30552) 
Co-authored-by: zkexorability <zkexorability@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-14 23:00:39 -08:00
汪志鹏
1adeb3b84c
[New Model] BAGEL support (AR only) (#28439) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-15 14:58:23 +08:00
Kunshang Ji
e3a1cd1c59
[XPU] fix Dockerfile.xpu, avoid wheel conflicts (#30662) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-12-15 13:32:06 +08:00
Wentao Ye
3778673ea8
[Feat] Refactor for parallel_config in FusedMoEModularKernel (#30282) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-12-15 04:21:36 +00:00
Seokhyun An
b337647aa0
[Bugfix] Drop empty tool_calls lists to keep assistant replies in chat template (#30648) 
Signed-off-by: Seokhyun An <iamseokhyun@gmail.com>
 2025-12-15 04:21:12 +00:00
Jee Jee Li
a524d1ba0a
[Bugfix] Fix deepseek_v32 tokenizer_mode (#30658) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-15 04:20:31 +00:00
Shanshan Shen
87b4d1557d
[CustomOp][MM] Extract MMEncoderAttention as CustomOp and replace the backend of QwenVisionAttention with it. (#30125) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-12-15 11:13:32 +08:00
Wenqi Glantz
84e23d103d
additional protection for CVE-2025-62164 (#30649) 
Signed-off-by: Wenqi Glantz <wglantz@nvidia.com>
 2025-12-15 03:07:10 +00:00
Shanshan Shen
738648fb81
[CustomOp] Support object-level enable for CustomOp (#30547) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-12-15 11:02:09 +08:00
Boyuan Feng
917fdae5b2
[Log] Skip piecewise cudagraph warn when using full cudagraph (#30657) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-12-15 02:49:45 +00:00
Robert Shaw
e2ed238885
Revert "[Fix]Load kv-cache dtype from hf_quant_config.json automatically" (#30653) 2025-12-14 19:33:41 -05:00
Or Ozeri
174e39ead7
CPU KV Offloading: Use more CUDA streams (#29013) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-12-14 23:50:45 +00:00
RioS
9ccbf6b692
[responsesAPI]add extra body parameters (#30532) 
Signed-off-by: Ri0S <aa248424@gmail.com>
 2025-12-14 19:25:45 +00:00
Chendi.Xue
ae2e503dda
[NIXL][BUG FIX] Fix a bug for PD with host_buffer after merging 29665 (#30420) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2025-12-14 15:38:28 +00:00
Tsukasa OI
9e33a1a75b
[Model][Quantization] Override HF defaults to GGUF ones (incl. Qwen3 MoE) (#30118) 
Signed-off-by: Tsukasa OI <floss_llm@irq.a4lg.com>
 2025-12-14 15:01:42 +00:00
Vensen
add4b0ca44
[Bugfix][benchmarks] Fix input token calculation for rerank benchmark metrics (#30596) 
Signed-off-by: vensen <vensenmu@gmail.com>
 2025-12-14 14:57:15 +00:00
ZiTian Zhao
ae88aada38
[Feature]Add EVS (Efficient Video Sampling) Support for Qwen3-VL (#29752) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: deitxfge <huhaibo1990@126.com>
 2025-12-14 05:24:56 -08:00
yifant-code
5ccf0efa84
[Bugfix] Improve error messages in ModelConfig validation (#30213) 
Signed-off-by: ytian218 <ytian218@bloomberg.net>
Co-authored-by: ytian218 <ytian218@bloomberg.net>
 2025-12-14 21:23:37 +08:00
ElizaWszola
994acec0cc
[Bugfix] Fix fusion for VL models (#30244) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-12-14 21:22:37 +08:00
zifeitong
48b8456ff9
[Bugfix] Revert Qwen2-VL part of change in #28271 (#30542) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
 2025-12-14 05:20:08 -08:00
Drew Botwinick
5b64ac21f9
[Bugfix] Update get_processor_data to use get_all method (#30583) 
Signed-off-by: Drew Botwinick <6953152+dbotwinick@users.noreply.github.com>
 2025-12-14 21:19:20 +08:00
Bin Bao
a8ec486592
[Misc] Add a script to benchmark compilation time (#29919) 
Signed-off-by: Bin Bao <binbao@meta.com>
 2025-12-14 13:02:39 +00:00
tjp_zju
6ecc1e411b
[Bugfix] fix _get_quant_method of FusedMoE for deepseekV3.2 on non-NV… (#30057) 
Signed-off-by: tjp_zju <tanjianpingzju1990@gmail.com>
 2025-12-14 02:20:51 -08:00
Shengliang Xu
0bb0bae436
Nvidia ModelOpt workaround for issue 28072 (#30164) 
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2025-12-14 18:18:31 +08:00
Johannes F
060893654d
fix: Update json features supported by xGrammar (#30390) 
Signed-off-by: Johannes Flommersfeld <johannes.flommersfeld@tngtech.com>
Signed-off-by: Johannes F <johannesflommersfeld@users.noreply.github.com>
Co-authored-by: Johannes Flommersfeld <johannes.flommersfeld@tngtech.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-14 02:16:06 -08:00
Matthias Gehre
e9add129ad
[Bugfix] awq_gemm: fix argument order swap (#30364) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-14 18:15:37 +08:00