xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-13 02:57:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,324 Commits 155 Branches 89 Tags
Commit Graph

215 Commits

Author SHA1 Message Date
Matthias Gehre
e9add129ad
[Bugfix] awq_gemm: fix argument order swap (#30364) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-14 18:15:37 +08:00
Lucas Wilkinson
3e41992fec
[Attention] Use sparse prefill kernel for fp8 kv-cache in DeepSeek-v3.2 (#27532) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-12 05:57:47 -08:00
haoyangli-amd
03416eada6
[bugfix][quantization] Fix fp8 per_tensor scale shape (#30257) 
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
 2025-12-09 19:28:50 +08:00
czhu-cohere
f6227c22ab
[Kernel]Support W4A8 Grouped GEMM on Hopper (#29691) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-12-08 19:29:06 -08:00
gnovack
ea657f2078
Lora MoE Align Improvements (#29257) 
Signed-off-by: gnovack <gnovack@amazon.com>
 2025-12-09 10:35:16 +08:00
ElizaWszola
af0444bf40
[Performance] Fused blockwise quant RMS norm (#27883) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 16:38:04 +00:00
Jinzhen Lin
879ddb09c3
[Kernel][MoE] optimize moe_align_block_size (#29642) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-07 01:58:47 -08:00
Jinzhen Lin
1656ad3704
[Kernel][Quantization] add w4a8 support for marlin kernel (#24722) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
 2025-11-29 07:19:33 -08:00
Pleaplusone
77e10c9cab
[Perf][Deepseek] optimize gather_and_maybe_dequant_cache kernel's perf for extremely long sequence (#28029) 
Signed-off-by: ganyi <ygan@amd.com>
 2025-11-24 19:05:46 -07:00
zhrrr
a982f5b5ea
[kernel][perf] support uncontiguous input for rms_norm kernel (#28103) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-20 19:39:09 -08:00
Li, Jiang
20852c8f4c
[CPU] Refactor CPU WNA16 (#28826) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-19 10:32:00 +08:00
Jane (Yuan) Xu
06c4873d95
Rewrite C++ meta funcs to Python (#28595) 
Signed-off-by: Jane Xu <janeyx@meta.com>
 2025-11-14 00:52:50 +08:00
Li, Jiang
7f829be7d3
[CPU] Refactor CPU attention backend (#27954) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-12 09:43:06 +08:00
zhrrr
68c09efc37
[Kernel][Perf] fuse QK Norm and RoPE into one cuda kernel for Qwen Model (#27165) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2025-11-11 12:00:31 -05:00
Michael Goin
0852527647
[Perf][DeepSeek] Add sigmoid+bias fusion to fused_grouped_topk from TRTLLM (#28124) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-07 18:20:55 -08:00
Pavani Majety
72b1c2ae2c
[Bugfix] Use latency MOE backend as default for Flashinfer and other misc fixes (#27439) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-11-07 04:18:39 -08:00
Zhewen Li
5ee93a5956
[CI/Build] Update checking logic in cutlass_group_gemm_supported (#27948) 
Signed-off-by: zhewenli <zhewenli@meta.com>
 2025-11-05 15:40:10 -08:00
gnovack
294c805f1d
Early exit for MoE LoRA kernels (#27131) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-03 20:22:17 +08:00
Asaf Joseph Gardin
00b31a36a2
[V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-11-02 04:16:23 -08:00
Xiangyu Li
5cc6bddb6e
[Kernel] Add GPTQv2 format support for low-bit or asymmetric quantization, by adapting gptq_gemm (#26092) 2025-10-23 23:26:13 -04:00
gnovack
8e4ca4d14e
Bugfix - pass 'max_num_tokens_padded' into 'moe_lora_align_block_size' (#27311) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-22 12:23:57 +00:00
Chen Wu
5f6cbf60d6
[Feature][Kernel]FusedMoE LoRA (#21229) 
Signed-off-by: wuchen <cntryroa@gmail.com>
Signed-off-by: banjuede <lmklhc@163.com>
Signed-off-by: Chen Wu <cntryroa@gmail.com>
Signed-off-by: Danielle Robinson <dmmaddix@amazon.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: wuchen <wuchen@zetyun.com>
Co-authored-by: Nathan Van Gheem <vangheem@gmail.com>
Co-authored-by: banjuede <lmklhc@163.com>
Co-authored-by: Danielle Robinson <dmmaddix@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
 2025-10-21 03:01:37 +00:00
Isotr0py
3125d79950
[Chore] Remove unused PolyNorm layer (#27110) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-17 19:03:43 +00:00
Luka Govedič
bd7157a071
[torch.compile] Enable attention and allreduce fusion without custom ops enabled (#24604) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-17 08:10:23 -06:00
zhrrr
75c7ad9918
[Kernel][Performance] Fuse float cast and renormalize to topk softmax kernel (#26717) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
 2025-10-17 07:30:35 +00:00
Varun Sundar Rabindranath
fb0571b077
[GPTOSS][DP/EP][Marlin] Enable GPTOSS Batched DP/EP using Marlin kernels (#25997) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-10-16 12:53:11 -07:00
Harry Mellor
8fcaaf6a16
Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Roberto L. Castro
96ad65b7fe
[Transform] [Quantization] Add QuTLASS support to vLLM (#24440) 
Signed-off-by: LopezCastroRoberto <roberto.lopez.castro@udc.es>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Signed-off-by: Andrei Panferov <andrei@panferov.org>
Co-authored-by: Andrei Panferov <andrei@panferov.org>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-10 09:43:40 -07:00
roikoren755
4069db3f2e
[Bugfix] Enable padded FP4 quantization (#25947) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-10-09 10:59:41 -07:00
Nicolò Lucchesi
4ebc9108a7
[Kernel] Centralize platform kernel import in current_platform.import_kernels (#26286) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-08 20:25:31 +00:00
Barry Kang
127c8b782a
Add gather_indexer_k_quant_cache kernel (#25931) 
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-08 04:58:57 +00:00
Harry Mellor
b893d661b1
Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Hank_
17edd8a807
[Platform][Kernel] platform-specific kernel loading (#25823) 
Signed-off-by: Hank <hcc.mayday@gmail.com>
 2025-10-05 13:25:15 +02:00
Fadi Arafeh
9705fba7b7
[cpu][perf] Accelerate unquantized-linear for AArch64 through oneDNN/ACL and weight prepack (#25948) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2025-10-04 12:16:38 +08:00
Yongye Zhu
fa7e254a7f
[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
Hashem Hashemi
a3a7828010
[ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>
 2025-09-23 14:31:45 -04:00
Matthew Bonanni
8f3616f422
Remove old cutlass mla (#23961) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-17 14:31:43 +00:00
Kyle Sayers
a0b26701c9
[Transform] Deterministic Hadacore Transforms (#24106) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-09-15 12:59:31 -06:00
xiao-llm
01413e0cf5
Fp8 paged attention update (#22222) 
Signed-off-by: Xiao Yu <xiao.yu@amd.com>
Signed-off-by: xiao-llm <xiao.yu.dc@outlook.com>
Co-authored-by: Xiao Yu <xiao.yu@metamaterial.com>
Co-authored-by: Xiao Yu <xiao.yu@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>
 2025-09-15 10:43:26 -04:00
Woosuk Kwon
5febdc8750
[Chore] Remove unused batched RoPE op & kernel (#24789) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-13 00:08:20 -07:00
TaehyunKim
9bd831f501
[Model] New model support for Motif-1-Tiny (#23414) 
Signed-off-by: ca1207 <ca1207zzz@gmail.com>
Signed-off-by: TaehyunKim <73943231+ca1207@users.noreply.github.com>
Co-authored-by: WyldeCat <skan1543@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-10 23:29:40 -07:00
Ming Yang
86173ad593
[Kernel] Support decode context parallelism on Blackwell with CUTLASS MLA (#24385) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-09-08 09:27:12 +08:00
yzds
ac201a0eaf
[Feature] Support Decode Context Parallel (DCP) for MLA (#23734) 
Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-09-06 13:24:05 +08:00
Li, Jiang
57b1ce94f7
[CPU] Refactor CPU unquantized linear (#24150) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-04 14:28:45 +08:00
yzds
186aced5ff
[Kernel] cuda kernels for upcoming decode context parallel feature (#23791) 
Co-authored-by: hongchao <hongchao@msh.team>
 2025-08-28 15:29:11 +08:00
Xin Yang
8a3cd90af5
[Kernel] Add fused grouped_topk kernel for MoE (#23274) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-25 11:47:52 -07:00
czhu-cohere
e76e233540
[kernel] Support W4A8 on Hopper (#23198) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-08-24 06:18:04 +00:00
Matthew Bonanni
19fe1a0510
[Kernel] Add FP8 support with FlashMLA backend (#22668) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-08-22 02:26:32 +00:00
Li, Jiang
7be5d113d8
[CPU] Refactor CPU W8A8 scaled_mm (#23071) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-08-21 09:34:24 +08:00