xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-01 03:36:37 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,488 Commits 155 Branches 89 Tags
Commit Graph

8751 Commits

Author SHA1 Message Date
Mark McLoughlin
f790068600
[Core] Add a random suffix to frontend-provided request IDs (#27987) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-12-23 13:05:39 -08:00
Asaf Joseph Gardin
34916ae37f
[Mamba] - Consolidate Mambas Attention Logic (#28133) 2025-12-23 21:57:00 +01:00
Harry Mellor
c016c95b45
Use helper function instead of looping through attribute names (#29788) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-23 17:31:56 +00:00
Harry Mellor
1339878e13
Only patch original_max_position_embeddings for Transformers v4 (#31214) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-23 16:46:32 +00:00
Joachim Studnia
38c361f99d
Fix edge case Mistral tool parser (#30724) 
Signed-off-by: Joachim Studnia <joachim@mistral.ai>
Signed-off-by: Joachim Studnia <studniajoachim@gmail.com>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: juliendenize <julien.denize@mistral.ai>
Co-authored-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-12-23 14:19:58 +00:00
Cyrus Leung
bb62dda2c3
[Misc] Introduce encode_*_url utility function (#31208) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-23 13:45:21 +00:00
Patrick von Platen
3faa8bee57
adapt voxtral (#31095) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-12-23 05:31:55 -08:00
Harry Mellor
b10d47e0e0
Add util function for checking nesting of rope parameters (#31146) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-23 11:41:49 +00:00
R3hankhan
769f27e701
[OpenAI] Add parameter metadata to validation errors (#30134) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2025-12-23 11:30:12 +00:00
Jakub Zakrzewski
23daef548d
[Frontend] Support using chat template as custom score template for reranking models (#30550) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-23 11:19:16 +00:00
Jee Jee Li
27c6c2f98c
[Bugfix] Fix MoE LoRA bin/pt loading (#31161) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-23 19:09:15 +08:00
Weida Hong
73cfb7a722
Correct position of docstring of class attributes (#31209) 
Signed-off-by: Weida Hong <wdhongtw@google.com>
 2025-12-23 02:08:58 -08:00
vllmellm
f32cfd7d97
[ROCm][FEAT] Support AITER RMSNorm quantization fusion pass (#26575) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-12-23 02:07:54 -08:00
Jee Jee Li
6b16fff01b
[Bugfix] Fix Jais2ForCausalLM (#31198) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-23 07:44:01 +00:00
Yan Ma
f1c2c20136
[XPU] decrease IGC_ForceOCLSIMDWidth for speculative decoding triton-xpu kernel compilation (#30538) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-12-23 05:22:15 +00:00
quanliu
a37328fc5c
[Feature] Batch invariant: Lora (#30097) 
Signed-off-by: quanliu <18646313696@163.com>
 2025-12-23 10:32:47 +08:00
Pavani Majety
3e10262356
Revert "[SM100] Enable fp8 compute for prefill MLA (#30746)" (#31197) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-12-22 18:15:33 -08:00
Robert Shaw
b57b967386
[MoE Refactor][7/N] AITER MK (#31102) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-22 16:42:58 -07:00
Benjamin Chislett
85aff45e24
[Perf] Remove blocking copy in GDN Attention (#31167) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-12-22 14:25:22 -08:00
Wentao Ye
5312a7284e
[Bug] Fix 'CutlassMLAImpl' object has no attribute '_workspace_buffer' (#31173) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-22 14:24:27 -08:00
Lucas Wilkinson
de71747655
[SpecDecode] Simplified alternative padded-speculation acceptance rate fix (#29845) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-22 13:06:10 -08:00
Pavani Majety
b10f41c894
[SM100] Enable fp8 compute for prefill MLA (#30746) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-12-22 19:15:57 +00:00
Yongye Zhu
7b926e8901
[MoE Refactor][9/N] Use modular kernel for unquantized Triton MoE (#31052) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-12-22 17:34:19 +00:00
Boyuan Feng
8dd0db687b
[UX] improve profiler error message (#31125) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-12-22 08:45:59 -08:00
dengyunyang
8f8f469b1b
[BugFix] skip language model in Encoder (#30242) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2025-12-22 05:25:59 -08:00
AlonKejzman
bd6d5a7475
[gpt-oss] Fix harmony parser in streaming responses (#30205) 
Signed-off-by: AlonKejzman <alonkeizman@gmail.com>
 2025-12-22 20:56:06 +08:00
Li Wang
256a33ecb4
[Model] Fix bagel failed to run (#31132) 
Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-12-22 02:15:54 -08:00
Roger Young
c02a2705f9
Update MiniMax-M2 ToolCall and add MiniMax-M2.1 in Docs (#31083) 
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-12-22 05:28:40 +00:00
Kevin McKay
cf8eed7bef
[Bugfix][ROCm] Fix typo: is_linear_fp8_enaled -> is_linear_fp8_enabled (#31109) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-21 21:14:58 -08:00
Kevin McKay
14c3e6ade3
[Misc] Fix spelling typos in model comments (#31117) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:14:14 -08:00
Kevin McKay
42b42824ae
[Misc] Fix grammar errors in comments and messages (#31115) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:14:02 -08:00
Kevin McKay
ec58c10ce1
[Misc] Fix quantization-related typos (#31116) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:13:48 -08:00
Kevin McKay
8c084de59d
[Misc] Fix spelling typos in comments (#31114) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:13:14 -08:00
CedricHuang
19cc9468fd
[Feature]: Support NVIDIA ModelOpt HF FP8 variants FP8_PER_CHANNEL_PER_TOKEN and FP8_PB_WO in vLLM (#30957) 2025-12-21 22:34:49 -05:00
Robert Shaw
b471092d3a
[MoE Refactor][4/N] Marlin Fp8 Mk (#31036) 2025-12-21 12:37:42 -05:00
汪志鹏
3e92b2b7ac
[BugFix]fix gpt-oss v1/completions response bug (#30608) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: bbrowning <bbrownin@redhat.com>
 2025-12-21 10:39:31 +08:00
Jinzhen Lin
7c73ceb581
[Quantization] add marlin w4a8/w8a8 check (#31061) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-20 21:58:11 +00:00
Jinzhen Lin
ee52d9901d
[Quantization] support logical_widths for fp8 marlin (#30962) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-20 12:02:57 -08:00
baonudesifeizhai
54c8924384
[MoE Refactor][5/N] Isolate zero expert to LongCatFlash (#28891) 
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: Dongjie Zou <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2025-12-20 18:22:04 +00:00
Yan Ma
560ae9638c
[XPU] enable fp8 online streaming quantization (#30944) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-12-20 13:45:27 +00:00
Jeffrey Wang
1501a4070e
[Bugfix] Read truncate_prompt_tokens from pooling_params in AsyncLLM.encode() (#31013) 
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
 2025-12-20 10:29:31 +00:00
zejunchen-zejun
d52c5096d7
[Bugfix] fix the alias bug of AttentionBackendEnum when register CUSTOM attention backend to vllm (#30869) 
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
 2025-12-20 09:03:35 +08:00
Yuxuan Zhang
8a7a414374
GLM-4.7 Tool Parser and Doc Update (#30876) 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
 2025-12-20 00:09:58 +00:00
Robert Shaw
95befecc18
[MoE Refactor][2/N] Use Modular Kernels for Fp8 (#30825) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 23:36:38 +00:00
Wentao Ye
4cf9429897
[Bug] Fix error 'Dynamo failed to run FX node with fake tensors for Deepseek V3.2 (#31046) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-19 23:31:31 +00:00
Robert Shaw
83a317f650
[MoE Refactor][3/N] Deprecate cutlass block quant fp8 (b200) (#30990) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 13:09:54 -08:00
Lucas Wilkinson
5f6477d1d0
[BugFix] Fix TypeError: unhashable type: 'dict' when serving deepseek32 (#30924) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-19 16:07:54 -05:00
Wentao Ye
3bd8335bd0
[Refactor] Refactor for DeepGemmQuantScaleFMT using cache (#30898) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-19 13:50:39 -07:00
Seiji Eicher
1ab5213531
Make engine core client handshake timeout configurable (#27444) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-12-19 20:38:30 +00:00
Zhonghua Deng
969bbc7c61
[Model] Add MiMo-V2-Flash support (#30836) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
Signed-off-by: Jumiar <liuanqim10@126.com>
Signed-off-by: Zyann7 <zyann7@outlook.com>
Co-authored-by: Jumiar <liuanqim10@126.com>
Co-authored-by: Zyann7 <zyann7@outlook.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-19 17:17:03 +00:00