xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-06-06 13:42:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,262 Commits 155 Branches 89 Tags
Commit Graph

1114 Commits

Author SHA1 Message Date
Wentao Ye
3af47c3cc6
[Feature] Add Hopper DeepGEMM E8M0 for DeepSeekV3.1 scale_fmt (#23666) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-08-27 14:09:08 +00:00
Dipika Sikka
d272415e57
[Quantization] Expand compressed-tensors MoE matching logic to support NFP4 + FP8 MoEs (#22674) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
 2025-08-27 05:00:21 +00:00
CSWYF3634076
644d57d531
[Model] Add Ernie4.5 VL Model Support (#22514) 
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
 2025-08-26 21:02:55 -07:00
Michael Goin
de02b07db4
[Bugfix] Lazy import gpt_oss_triton_kernels_moe for mxfp4 (#23678) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-27 09:34:57 +08:00
czhu-cohere
2c2b140ae8
[quantization] use channel scales for w4a8 + misc fixes (#23570) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-08-26 18:23:23 -07:00
zixuanzhang226
b1625dbe9c
feat: add triton fused moe config for GLM-4.5-Air-FP8 on B200 (#23695) 
Signed-off-by: Zixuan Zhang <zixuanzhang@bytedance.com>
 2025-08-26 18:06:10 -07:00
Hyogeun Oh (오효근)
730d0ac8b9
[Docs] Fix warnings in mkdocs build (#23649) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-26 18:19:23 +00:00
TianyuLi0
f58675bfb3
[CPU] add cpu fused moe pytorch native implementation (#23146) 
Signed-off-by: Tianyu Li <tianyu.li@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2025-08-26 14:09:17 +00:00
nvjullin
f66673a39d
[Kernel] Added flashinfer fp8 per-tensor gemms (#22895) 
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-26 06:54:04 -07:00
Michael Goin
d52358c1e0
[Perf] Remove duplicated NVFP4 blockscales to save memory (#23379) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-26 19:16:33 +08:00
weiliang
ae067888d6
Update Flashinfer to 0.2.14.post1 (#23537) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: siyuanf <siyuanf@nvidia.com>
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-25 18:30:44 -07:00
Xin Yang
8a3cd90af5
[Kernel] Add fused grouped_topk kernel for MoE (#23274) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-25 11:47:52 -07:00
Ayush Satyam
5c4b6e66fe
[Attention] Unify mamba and attention backend selection (#23171) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-08-25 09:09:36 +00:00
LIYIFAN_liyifan
c9abb10489
[Bugfix] Fix Dense module loading for sentence-transformers embedding models (simplified V2) (#23408) 
Signed-off-by: FFFfff1FFFfff <yifanli0919@gmail.com>
 2025-08-25 05:39:24 +00:00
Jee Jee Li
170e8ea9ea
[Misc] Unified linear print info (#23516) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-24 20:13:51 -07:00
Ming Yang
504d914314
[Perf] Add Triton config for DeepSeek V3 FP8 EP32 H200 (#23504) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-08-24 18:06:35 -07:00
czhu-cohere
e76e233540
[kernel] Support W4A8 on Hopper (#23198) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-08-24 06:18:04 +00:00
Chenxi Yang
308fa287a8
Add glm4.5v tp2,4 fp8 config on H100_80GB (#23443) 
Co-authored-by: Chenxi Yang <cxyang@meta.com>
 2025-08-23 02:54:19 +00:00
Daifeng Li
fa78de9dc3
Quantization: support FP4 quantized models on AMD CDNA2/CDNA3 GPUs (#22527) 
Signed-off-by: feng <fengli1702@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-22 20:53:21 -06:00
elvischenv
24d0c9e6ed
[NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-08-22 22:09:05 +00:00
rasmith
cc7ae5e7ca
[BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being called when it should when using quantized FP8 model (#22281) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-08-22 21:47:57 +00:00
Isotr0py
4645024d3a
[Quantization] Allow GGUF quantization to skip unquantized layer (#23188) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-08-22 13:04:22 -06:00
Wentao Ye
394591e343
[Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improvement (#23351) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-08-21 21:01:08 -07:00
Michael Goin
3bbe11cc13
[Perf] Small optimizations for silu_mul_fp8_quant_deep_gemm (#23265) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-21 17:56:15 -04:00
Wentao Ye
48bfb0c9b7
[Bug] Fix R1 Accuracy 0 Bug (#23294) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-21 13:11:28 -04:00
Yi Liu
0278f1ac3a
Fix nvfp4 swizzling (#23140) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-21 16:54:50 +00:00
wang.yuqi
d70a16625d
[Performance] V1 Pooling Models E2E Performance Optimization (#23162) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-21 13:26:09 +00:00
Paul Pak
2e2000f352
[Model] Add LFM2 architecture (#22845) 
Signed-off-by: Paul Pak <paulpak58@gmail.com>
 2025-08-21 09:35:07 +02:00
Asaf Joseph Gardin
3663870c72
[V1][Mamba1] - Full CUDA and Piecewise CUDA Graphs Support (#23035) 
Signed-off-by: asafg <asafg@ai21.com>
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-08-20 20:08:51 -07:00
Li, Jiang
7be5d113d8
[CPU] Refactor CPU W8A8 scaled_mm (#23071) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-08-21 09:34:24 +08:00
22quinn
4b795020ed
[EP] Add logging for experts map (#22685) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-08-20 23:46:06 +00:00
Michael Goin
0cdbf5e61c
[Kernel/Quant] Remove the original marlin format and qqq (#23204) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-20 15:13:36 -04:00
shixianc
b17109beea
[Kernel] CUTLASS MoE FP8: Integrate cuda moe permute/unpermute (#23045) 
Signed-off-by: Shixian Cui <shixian@amazon.com>
 2025-08-20 10:35:26 -04:00
amirkl94
a38b8af4c3
[NVIDIA] Add SM100 Flashinfer Cutlass MoE fp8 backend (#22357) 
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
 2025-08-19 18:01:53 -04:00
bnellnm
b94faf9d50
[Bugfix] Fix accuracy issue when using flashinfer cutlass moe, TP=1 and modelopt. (#23125) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-19 14:00:51 -04:00
TJian
1298c67795
[FEAT] [Performance] Enable DP for ViT in Qwen2.5VL (#22742) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-08-19 15:25:57 +00:00
wang.yuqi
f856c33ce9
[Model] Add multi_label_classification support (#23173) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-19 12:54:30 +00:00
Chengji Yao
e9d6a3db69
[TPU] make ptxla not imported when using tpu_commons (#23081) 
Signed-off-by: Chengji Yao <chengjiyao@gmail.com>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@gmail.com>
 2025-08-19 11:46:42 +08:00
Michael Goin
6d25e3fd6e
Use Blackwell FlashInfer MXFP4 MoE by default if available (#23008) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-18 15:25:49 -07:00
Simon Mo
0fc8fa751a
fix: gptq marlin weight loading failure (#23066) 2025-08-17 15:56:07 -07:00
Jee Jee Li
4d4061b6e7
[Kernel] Add cuda kernel for gpt_oss activation (#22951) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-17 05:03:24 +00:00
Michael Goin
94096a47c9
[UX] Separate marlin moe config logic from triton moe (#23006) 2025-08-16 22:16:42 -04:00
Jinzhen Lin
a258ad8bcc
[Bugfix] fix qwen3 moe fp8 accuracy issue (#23031) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2025-08-16 17:41:23 -07:00
Michael Goin
4fc722eca4
[Kernel/Quant] Remove AQLM (#22943) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-16 19:38:21 +00:00
bnellnm
8ad7285ea2
[Kernels] Clean up FusedMoeMethodBase and modular kernel setup. Remove extra arguments from modular kernel methods. (#22035) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-15 14:46:00 -04:00
sstamenk
6b04039a72
[BugFix] Skip the Q component for QKVParallelLinear in the case of QKVCrossParallelLinear since its width is 0 (#22369) 
Signed-off-by: sstamenk <sstamenk@amd.com>
 2025-08-15 17:17:31 +00:00
Thomas Parnell
75531a6c13
[V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax) (#22928) 
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Daniel Afrimi <danielafrimi8@gmail.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-15 12:57:06 +00:00
amirai21
fe91ce9591
[V1] - Split Prefill and Decode for Mamba1 models (#22653) 
Signed-off-by: amirk <amirk@ai21.com>
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
Co-authored-by: Asaf Joseph Gardin <39553475+Josephasafg@users.noreply.github.com>
 2025-08-15 08:59:52 +00:00
TJian
b2f6c247a9
Revert "[ROCm][AITER] Support AITER Rope ops in RotaryEmbedding Module." (#22956) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-08-15 06:39:19 +00:00
Asaf Joseph Gardin
3d232dbd19
[Mamba] - refactor: Renamed mamba_attn to mamba2_attn (#22818) 
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-08-15 06:38:05 +00:00