xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-06 07:17:04 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,490 Commits 155 Branches 89 Tags
Commit Graph

1634 Commits

Author SHA1 Message Date
Asaf Joseph Gardin
34916ae37f
[Mamba] - Consolidate Mambas Attention Logic (#28133) 2025-12-23 21:57:00 +01:00
Pavani Majety
3e10262356
Revert "[SM100] Enable fp8 compute for prefill MLA (#30746)" (#31197) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-12-22 18:15:33 -08:00
Robert Shaw
b57b967386
[MoE Refactor][7/N] AITER MK (#31102) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-22 16:42:58 -07:00
Pavani Majety
b10f41c894
[SM100] Enable fp8 compute for prefill MLA (#30746) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-12-22 19:15:57 +00:00
Yongye Zhu
7b926e8901
[MoE Refactor][9/N] Use modular kernel for unquantized Triton MoE (#31052) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-12-22 17:34:19 +00:00
Kevin McKay
cf8eed7bef
[Bugfix][ROCm] Fix typo: is_linear_fp8_enaled -> is_linear_fp8_enabled (#31109) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-21 21:14:58 -08:00
CedricHuang
19cc9468fd
[Feature]: Support NVIDIA ModelOpt HF FP8 variants FP8_PER_CHANNEL_PER_TOKEN and FP8_PB_WO in vLLM (#30957) 2025-12-21 22:34:49 -05:00
Robert Shaw
b471092d3a
[MoE Refactor][4/N] Marlin Fp8 Mk (#31036) 2025-12-21 12:37:42 -05:00
Jinzhen Lin
7c73ceb581
[Quantization] add marlin w4a8/w8a8 check (#31061) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-20 21:58:11 +00:00
Jinzhen Lin
ee52d9901d
[Quantization] support logical_widths for fp8 marlin (#30962) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-20 12:02:57 -08:00
baonudesifeizhai
54c8924384
[MoE Refactor][5/N] Isolate zero expert to LongCatFlash (#28891) 
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: Dongjie Zou <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2025-12-20 18:22:04 +00:00
Yan Ma
560ae9638c
[XPU] enable fp8 online streaming quantization (#30944) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-12-20 13:45:27 +00:00
Robert Shaw
95befecc18
[MoE Refactor][2/N] Use Modular Kernels for Fp8 (#30825) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 23:36:38 +00:00
Robert Shaw
83a317f650
[MoE Refactor][3/N] Deprecate cutlass block quant fp8 (b200) (#30990) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 13:09:54 -08:00
Wentao Ye
3bd8335bd0
[Refactor] Refactor for DeepGemmQuantScaleFMT using cache (#30898) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-19 13:50:39 -07:00
Zhonghua Deng
969bbc7c61
[Model] Add MiMo-V2-Flash support (#30836) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
Signed-off-by: Jumiar <liuanqim10@126.com>
Signed-off-by: Zyann7 <zyann7@outlook.com>
Co-authored-by: Jumiar <liuanqim10@126.com>
Co-authored-by: Zyann7 <zyann7@outlook.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-19 17:17:03 +00:00
Jinzhen Lin
5fbfa8d9ef
[Quantization] fix marlin w8a8 check (#30961) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-19 07:33:22 -08:00
Shanshan Shen
23a1946e3b
[CustomOp][Refactor] Extract common methods for ApplyRotaryEmb CustomOp (#31021) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-12-19 22:16:09 +08:00
Jinzhen Lin
9187de9fac
[Quantization] enable compressed-tensors marlin support for turing (2) (#31008) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-19 08:56:35 +00:00
Jinzhen Lin
de08b8f61b
[Quantization] enable compressed-tensors marlin support for turing (#31000) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-12-18 20:29:48 -08:00
Wentao Ye
97000a2be7
[Bug] Fix compressed tensor not using deepgemm (#30820) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-18 14:45:55 -05:00
navmarri14
b8c477c115
tuned fused configs for B300 (#30629) 2025-12-18 11:41:59 -08:00
jiahanc
53ad423f26
[Perf] enable flashinfer rotary_embedding custom ops in DeepSeek rotary (#30729) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2025-12-18 14:31:18 -05:00
Vasiliy Kuznetsov
f4ee2c3d90
fix fp8 online quantization streaming with tp > 1 (#30900) 
Signed-off-by: vasiliy <vasiliy@fb.com>
 2025-12-18 11:45:15 -05:00
Xin Yang
9a5e96523b
[LoRA] Set default MXFP4 LoRA backend to Marlin (#30598) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-18 08:42:22 -08:00
Wentao Ye
6628758233
[Bug] Fix batch invariant in torch 2.10 (#30907) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-18 07:27:51 -08:00
Ming Yang
8372be2828
[moe] Use enable_chunking func (to support disabling chunking) (#29935) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-12-18 09:02:38 +00:00
Li, Jiang
e3ab93c896
[CPU] Refactor CPU fused MOE (#30531) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-12-18 14:36:49 +08:00
Vadim Gimpelson
717ac33d9c
[PERF] Qwen3-next. Add fp8 cutlass MoE tuned configs. chmod -x *MI308X.json (#29553) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-12-18 13:16:04 +08:00
Bowen Bao
0c738b58bc
[Quantization] Support Quark int4-fp8 w4a8 for MoE (#30071) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2025-12-18 04:20:42 +00:00
Varun Sundar Rabindranath
e3fc374a9a
[BugFix] Workspace allocation during profile run : DeepEPHighThroughput + DeepGEMM (#30899) 2025-12-17 15:00:59 -08:00
Andrey Talman
e06d0bf0aa
2.9.1 PyTorch release update (#28495) 2025-12-17 12:20:22 -08:00
Wentao Ye
f284d7bd0c
[Bug] Fix AttributeError: 'ColumnParallelLinear' object has no attribute weight_scale_inv (#30823) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-17 02:00:35 -08:00
Xinyu Chen
3b1d440ede
CustomOp: grouped topk (#29575) 
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
 2025-12-17 17:43:00 +08:00
Yan Ma
4f735babb7
[XPU] fix broken fp8 online quantization for XPU platform (#30831) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-12-17 00:28:13 -08:00
Li, Jiang
0cd5353644
[Bugfix][CPU] Fix CPU backend ROPE dispatch for VL models (#30829) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-16 23:25:12 -08:00
Jinzhen Lin
ce96857fdd
[Kernel][Quantization][MoE] add marlin kernel support for turing (sm75) (#29901) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-12-16 14:35:28 -08:00
jiahanc
254a7f8fd6
[Perf] Do FP4 quant before All gather on flashinfer trtllmgen MOE (#30014) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2025-12-16 13:01:48 -08:00
Michael Goin
10ee1c64cf
[CI] Generalize gsm8k test args and add Qwen3-Next MTP B200 test (#30723) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-16 14:28:34 -05:00
Ming Yang
ce12b407f2
[TRTLLM] Remove the MoE GEMM weight name change (#30713) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-12-16 11:01:38 -05:00
Wentao Ye
59bd5f6a71
[Feat] Enable eplb with default all2all backend (#30559) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-16 10:33:52 -05:00
Shanshan Shen
3bd9c49158
[CustomOp] Extract ApplyRotaryEmb as CustomOp and unify the dispatch logic (#29873) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-12-15 19:08:16 -08:00
Matthew Bonanni
60dbf7d8f1
Update batch invariant to use attention config (#30704) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 15:24:16 -05:00
Robert Shaw
d0502b4928
[MoE][Refactor 1/N] Separate Online Quantization (#30627) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-15 06:54:53 -08:00
Wentao Ye
3778673ea8
[Feat] Refactor for parallel_config in FusedMoEModularKernel (#30282) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-12-15 04:21:36 +00:00
tjp_zju
6ecc1e411b
[Bugfix] fix _get_quant_method of FusedMoE for deepseekV3.2 on non-NV… (#30057) 
Signed-off-by: tjp_zju <tanjianpingzju1990@gmail.com>
 2025-12-14 02:20:51 -08:00
Shengliang Xu
0bb0bae436
Nvidia ModelOpt workaround for issue 28072 (#30164) 
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2025-12-14 18:18:31 +08:00
Didier Durand
1a55cfafcb
[Doc]: fixing typos in various files (#30540) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Didier Durand <2927957+didier-durand@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-14 02:14:37 -08:00
Wentao Ye
6e78ed6ba7
[Logs] Optimize startup logs 4 (#29903) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-13 16:12:53 -05:00
Tsukasa OI
fdc135d768
[Misc][Quantization] Clarify the intent of GGUF FusedMoE weight materialization (#30310) 
Signed-off-by: Tsukasa OI <floss_llm@irq.a4lg.com>
 2025-12-13 13:55:14 +08:00