xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-29 14:13:58 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,444 Commits 155 Branches 89 Tags
Commit Graph

12444 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kevin McKay
ec58c10ce1
[Misc] Fix quantization-related typos (#31116) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:13:48 -08:00
Kevin McKay
8c084de59d
[Misc] Fix spelling typos in comments (#31114) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:13:14 -08:00
CedricHuang
19cc9468fd
[Feature]: Support NVIDIA ModelOpt HF FP8 variants FP8_PER_CHANNEL_PER_TOKEN and FP8_PB_WO in vLLM (#30957) 2025-12-21 22:34:49 -05:00
Jee Jee Li
097978a15d
[Kernel] Enable fused_qknorm_rope_kernel supports partial rope (#30821) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-21 18:39:22 -08:00
Lucas Wilkinson
7e065eba59
[CI] Fix "2 Node Tests (4 GPUs in total)" (#31090) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-22 10:32:40 +08:00
Steve Westerhouse
9d701e90d8
[Doc] Clarify FP8 KV cache computation workflow (#31071) 
Signed-off-by: westers <steve.westerhouse@origami-analytics.com>
 2025-12-22 08:41:37 +08:00
Michael Goin
06d490282f
[NVFP4][Perf] Tune NVFP4 input quant kernel for small batch size (#30897) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-21 09:41:57 -08:00
Robert Shaw
b471092d3a
[MoE Refactor][4/N] Marlin Fp8 Mk (#31036) 2025-12-21 12:37:42 -05:00
Ameen Patel
93cabc417c
ci: add nvidia-smi warmup before Prime-RL integration test (#31093) 
Signed-off-by: AmeenP <ameenp360@gmail.com>
 2025-12-21 15:43:01 +00:00
Chauncey
bb80f69bc9
add aarnphm and chaunceyjiang to the new tool_parser directory (#31088) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-12-21 03:24:34 +00:00
汪志鹏
3e92b2b7ac
[BugFix]fix gpt-oss v1/completions response bug (#30608) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: bbrowning <bbrownin@redhat.com>
 2025-12-21 10:39:31 +08:00
Jinzhen Lin
7c73ceb581
[Quantization] add marlin w4a8/w8a8 check (#31061) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-20 21:58:11 +00:00
Lucas Wilkinson
ae0770fa6b
[CI] Fix H200 Distributed test (#31054) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-20 16:48:49 -05:00
Jinzhen Lin
ee52d9901d
[Quantization] support logical_widths for fp8 marlin (#30962) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-20 12:02:57 -08:00
baonudesifeizhai
54c8924384
[MoE Refactor][5/N] Isolate zero expert to LongCatFlash (#28891) 
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: Dongjie Zou <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2025-12-20 18:22:04 +00:00
Yan Ma
560ae9638c
[XPU] enable fp8 online streaming quantization (#30944) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-12-20 13:45:27 +00:00
Jeffrey Wang
1501a4070e
[Bugfix] Read truncate_prompt_tokens from pooling_params in AsyncLLM.encode() (#31013) 
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
 2025-12-20 10:29:31 +00:00
Lucas Wilkinson
ff2168bca3
[CI] FIx fixture 'siglip_attention_config' not found (#31053) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-20 03:46:15 +00:00
Gregory Shtrasberg
0be149524c
[ROCm][CI/Build] Update ROCm dockerfiles (#30991) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-12-20 03:19:12 +00:00
zejunchen-zejun
d52c5096d7
[Bugfix] fix the alias bug of AttentionBackendEnum when register CUSTOM attention backend to vllm (#30869) 
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
 2025-12-20 09:03:35 +08:00
Yuxuan Zhang
8a7a414374
GLM-4.7 Tool Parser and Doc Update (#30876) 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
 2025-12-20 00:09:58 +00:00
Robert Shaw
95befecc18
[MoE Refactor][2/N] Use Modular Kernels for Fp8 (#30825) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 23:36:38 +00:00
Wentao Ye
4cf9429897
[Bug] Fix error 'Dynamo failed to run FX node with fake tensors for Deepseek V3.2 (#31046) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-19 23:31:31 +00:00
Robert Shaw
83a317f650
[MoE Refactor][3/N] Deprecate cutlass block quant fp8 (b200) (#30990) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 13:09:54 -08:00
Lucas Wilkinson
5f6477d1d0
[BugFix] Fix TypeError: unhashable type: 'dict' when serving deepseek32 (#30924) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-19 16:07:54 -05:00
Wentao Ye
3bd8335bd0
[Refactor] Refactor for DeepGemmQuantScaleFMT using cache (#30898) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-19 13:50:39 -07:00
Seiji Eicher
1ab5213531
Make engine core client handshake timeout configurable (#27444) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-12-19 20:38:30 +00:00
Zhonghua Deng
969bbc7c61
[Model] Add MiMo-V2-Flash support (#30836) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
Signed-off-by: Jumiar <liuanqim10@126.com>
Signed-off-by: Zyann7 <zyann7@outlook.com>
Co-authored-by: Jumiar <liuanqim10@126.com>
Co-authored-by: Zyann7 <zyann7@outlook.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-19 17:17:03 +00:00
Andrey Talman
268a972c62
Update Pytorch version update docs (#30982) 2025-12-19 16:08:53 +00:00
Jinzhen Lin
5fbfa8d9ef
[Quantization] fix marlin w8a8 check (#30961) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-19 07:33:22 -08:00
Shanshan Shen
23a1946e3b
[CustomOp][Refactor] Extract common methods for ApplyRotaryEmb CustomOp (#31021) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-12-19 22:16:09 +08:00
Thomas Parnell
b5545d9d5c
[Bugfix] [Kernel] Triton attention kernels: mask out V blocks that fall outside sliding window (#30887) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-12-19 21:39:54 +08:00
Nishidha Panpaliya
bd2b52fc2d
[CPU][Bugfix] Fix ppc64le CPU build (#30871) 
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
 2025-12-19 12:26:35 +00:00
Li, Jiang
420ba2dbb6
Enable aarch64 CPU performance benchmarks (#26494) 
Signed-off-by: Ioana Ghiban <ioana.ghiban@arm.com>
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Ioana Ghiban <ioana.ghiban@arm.com>
Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-12-19 12:16:18 +00:00
Marko Rosenmueller
455949675d
[Frontend][Bug] allow tool calls in analysis channel (#28139) 
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-12-19 10:47:44 +00:00
lif
086b96339f
[Bugfix] Add validation for tool requests when tool_parser is unavailable (#30613) 
Signed-off-by: majiayu000 <1835304752@qq.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-19 18:23:28 +08:00
Jinzhen Lin
9187de9fac
[Quantization] enable compressed-tensors marlin support for turing (2) (#31008) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-19 08:56:35 +00:00
Isotr0py
ac1c934276
[Bugfix] Fix incorrect tiles creation for mm prefix triton attention (#30974) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:00:33 +08:00
Wenqi Glantz
4924ac582c
Add hidden dimension validation for multimodal embedding inputs (#30968) 
Signed-off-by: Wenqi Glantz <wglantz@nvidia.com>
 2025-12-19 07:59:36 +00:00
Li, Jiang
096b25c9ed
[Doc][CPU] Fix index link for CPU regular release wheels (#31015) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-12-19 07:29:52 +00:00
Jinzhen Lin
de08b8f61b
[Quantization] enable compressed-tensors marlin support for turing (#31000) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-18 20:29:48 -08:00
Nick Hill
2ac85a4544
[BugFix] Fix logprobs with spec decode and modified logits (#30846) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-12-18 19:58:28 -08:00
Andreas Karatzas
7b43db210c
[ROCm][CI][Bugfix] Multi-Modal Model Support Fixes and Attention Backend Improvements (#30270) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2025-12-19 02:17:27 +00:00
PlatinumGod
6a09612b2e
[Bugfix] Fix tool_choice="none" being ignored by GPT-OSS/harmony models (#30867) 
Signed-off-by: yujiepu <pyjapple@gmail.com>
Signed-off-by: PlatinumGod <pyjapple@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-12-19 09:34:27 +08:00
Nick Hill
45c0526ac9
[BugFix] Handle errors when preprocessing added requests (#30895) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-12-19 01:29:11 +00:00
Benjamin Chislett
d6b3d39b6d
[Cleanup] Refactor FlashInferMetadataBuilder (#29128) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-18 14:45:30 -08:00
Chendi.Xue
6ca74bc11a
[NIXL][BUG FIX] Fix both failing issue and accuracy issue with nixl + host_buffer on CUDA (#30419) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-12-18 22:10:02 +00:00
Harry Mellor
19c583398a
Check for truthy rope_parameters not the existence of it (#30983) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-18 13:59:10 -08:00
Nick Hill
b0b77c4655
[BugFix] Fix spec decode + structured outputs + preemption edge case (#30916) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-12-18 12:59:55 -08:00
Kayvan Mivehnejad
634a14bd7d
Strengthen input validation and tests for 'parse_raw_prompts’. (#30652) 
Signed-off-by: Kayvan Mivehnejad <K.Mivehnejad@gmail.com>
 2025-12-18 19:51:58 +00:00