xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-27 02:37:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,194 Commits 155 Branches 89 Tags
Commit Graph

721 Commits

Author SHA1 Message Date
Bhanu Prakash Voutharoja
6a6fc41c79
gptq marlin quantization support for fused moe with lora (#30254) 
Signed-off-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
 2025-12-12 02:27:22 +00:00
jiahanc
0ab23c2b2b
[fix] fix SM check for Flashinfer TRTLLM MOE (#30314) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2025-12-12 01:00:58 +00:00
Andrew Briand
a00d88973d
[EPLB] Support EPLB w/ NVFP4 (#29804) 
Signed-off-by: Andrew Briand <abriand@nvidia.com>
Co-authored-by: Andrew Briand <abriand@nvidia.com>
 2025-12-11 22:59:40 +00:00
汪志鹏
0e71eaa644
[Feature] AWQ marlin quantization support for fused moe with lora (#30442) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2025-12-11 18:03:32 +00:00
Wilson Wu
3bdd426636
Fix typos in comments across multiple files (#30345) 
Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-09 20:05:28 -08:00
ElizaWszola
2e7035dd8c
[Bugfix] Fix fp8 DeepGemm compilation issues (#30336) 2025-12-09 20:17:25 -05:00
Charlie Fu
3c680f4a17
[Rocm][torch.compile] Adding layernorm + fp8 block quant and silu + fp8 block quant for Aiter (#25693) 
Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Charlie Fu <Charlie.Fu@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: wuhuikx <hattie.wu@amd.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
 2025-12-09 22:39:26 +00:00
Kyle Sayers
fccd532587
[Quantization] FP8 Weight Reloading for Quantized RL Rollout (#28480) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-12-09 13:54:32 -08:00
bnellnm
00e5cbb967
[MoE][Refactor] Remove most arguments to FusedMoEMethodBase.apply (#29066) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-12-09 13:48:25 -08:00
vllmellm
ee14644ba9
[ROCm] Aiter Quant Kernels (#25552) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-12-09 14:27:37 +00:00
Dongjie Zou
1166c31cc7
[Bugfix]: Fix glm46 awq marlin moe wna16 compatibility (#30210) 
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
 2025-12-09 12:20:21 +00:00
Tsukasa OI
58d5b3f514
[Model][Quantization] Restore MoE + GGUF models support (incl. Qwen3 MoE) by allowing Sideload Parameters (#30116) 
Signed-off-by: Tsukasa OI <floss_llm@irq.a4lg.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-09 05:30:05 +00:00
Michael Goin
03b91f7262
[Bugfix] Fix compressed-tensors models failing to load with transformers backend (#30287) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-08 20:44:28 -08:00
czhu-cohere
f6227c22ab
[Kernel]Support W4A8 Grouped GEMM on Hopper (#29691) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-12-08 19:29:06 -08:00
Zhewen Li
ae339b1a67
[Bugfix] Fix DeepGEMM after #29546 (#30267) 
Signed-off-by: zhewenli <zhewenli@meta.com>
Signed-off-by: Zhewen Li <zhewenli@meta.com>
 2025-12-09 01:05:27 +00:00
Vasiliy Kuznetsov
0d402d2600
online fp8 quant with streaming weight post-processing (#29196) 
Signed-off-by: vasiliy <vasiliy@fb.com>
 2025-12-08 20:15:10 +00:00
Zhiwei
c6df05ebb4
[ROCm] [Fused Moe EP] Use binary expert mask for aiter fused moe kernel (#29773) 
Signed-off-by: ZhiweiYan-96 <zhiwei.yan@amd.com>
 2025-12-08 05:23:46 +00:00
ElizaWszola
af0444bf40
[Performance] Fused blockwise quant RMS norm (#27883) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 16:38:04 +00:00
Wentao Ye
541a2ef892
[Perf] Deepgemm fused layout kernel for activations, 4.3% throughput improvement, 10.7% TTFT improvement. (#29546) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 20:31:14 +08:00
yuttian1
c4d62618ca
Fix AWQ MoE marlin check issue in marlin_utils.py for AMD backend (#30102) 
Signed-off-by: yuttian1 <yuttian@amd.com>
 2025-12-05 20:54:38 -08:00
rasmith
dc839ad03d
[CI/Build][AMD][Quantization] Fix test_int8_kernel.py by updating int8_utils to use hip.libdevice.round (#30151) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-12-05 20:52:11 -08:00
Wentao Ye
7b5575fa7d
[Bug] Fix vLLM config is not set error (#29999) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-05 16:42:12 -05:00
Yi Liu
0d8a7d8a26
[Compressed Tensors] Add XPU wNa16 support (#29484) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
 2025-12-05 22:02:09 +08:00
Jee Jee Li
652ba93da3
[Bugfix] Fix FP8 MoE LoRA (#29890) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-04 18:17:49 +00:00
bnellnm
2902c34826
[Kernels] Remove BatchedTritonOrDeepGemmExperts and default fallback to Triton (#29929) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-12-03 20:49:00 +00:00
Varun Sundar Rabindranath
19bee6d12d
[Performance][DP/EP] Add silu_mul_per_token_group_quant_fp8_colmajor kernel (#29470) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-12-03 18:04:59 +00:00
HDCharles
b294e28db2
[refactor] CTMoEMethods to use QuantizationArgs (#28871) 
Signed-off-by: HDCharles <charlesdavidhernandez@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-03 11:00:56 +00:00
Fanli Lin
f37e8938d2
[XPU] Fix AWQ skipped layer detection in IPEX quantization (#29774) 
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
 2025-12-01 12:00:52 +00:00
Omer Ullman Argov
39d28108f4
[Feat] Support non-gated activations in NVFP4 modelopt path (#29004) 2025-11-30 11:02:40 -05:00
Isotr0py
47539cfd3e
[Bugfix] Fix mismatched nvfp4 gemm output shape (#29742) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-30 09:15:01 +00:00
Isotr0py
e1464c3a08
[Quantization] Enable compressed-tensors AWQ for Turing GPU (#29732) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-30 06:04:28 +00:00
Xin Yang
a491b0911b
[LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#29708) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Signed-off-by: Xin Yang <105740670+xyang16@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-30 10:37:25 +08:00
Jinzhen Lin
1656ad3704
[Kernel][Quantization] add w4a8 support for marlin kernel (#24722) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
 2025-11-29 07:19:33 -08:00
Didier Durand
04a797cd0e
[Doc]: fixing typos in various files. (#29717) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-11-29 01:15:39 -08:00
Huamin Li
3fd1fb0b60
Revert "[LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#28971)" (#29697) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
 2025-11-28 15:26:52 -08:00
Xin Yang
745a3bae1a
[LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#28971) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-28 10:48:28 +08:00
Matthew Bonanni
fc1d8be3dc
[Attention] Update attention imports (#29540) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-27 11:19:09 -05:00
HDCharles
df01eda4dc
[Bugfix] Make compressed-tensors MoEs respect ignored layers (#28878) 
Signed-off-by: HDCharles <charlesdavidhernandez@gmail.com>
 2025-11-26 21:35:13 -05:00
HDCharles
e603129505
[refactor] CTConfig methods to static/class methods (#28870) 
Signed-off-by: HDCharles <charlesdavidhernandez@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-26 17:21:58 +00:00
Michael Goin
7df0289782
Change warning logs to debug for unimplemented MXFP4 Linear/Attention (#29441) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-25 22:52:31 +00:00
Michael Goin
e502098643
[Kernel] Add NVFP4 MoE CUTLASS support for SM120 (#29242) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2025-11-25 06:59:07 -08:00
Michael Goin
6f1355a1b7
[Perf] Disable DeepGEMM MoE by default when TP=8 is used (#29346) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-24 19:01:40 -07:00
Michael Goin
c17610e2ba
[Bugfix] Only use triton_kernels for MXFP4 on SM90 and SM100 (#29339) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-24 18:22:46 -05:00
bnellnm
8f066146c3
[MoE][Refactor] Make select_experts a non-static method (#29067) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-11-24 13:38:04 -05:00
jiahanc
5f96c00c55
[Fix] Add SM check to flashinfer MOE backend (#29144) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-11-23 00:39:30 +00:00
Lukas Geiger
d045e22dfe
[Model][Qwen3VL] Tune Triton w8a8 block fp8 kernel for L40s (#29217) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-11-21 17:30:55 -08:00
Mingyuan Ma
b4c8fbaae2
Add TRTLLM MoE NVFP4 kernel to CompressedTensorsW4A4MoeMethod (#28892) 
Signed-off-by: mingyuanm <mingyuanm@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-11-21 09:54:11 -07:00
Wentao Ye
a42ab317ac
[Log] Optimize startup log (#28948) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-21 08:46:20 -08:00
Aleksandr Malyshev
b7f1f490a6
Upstream triton fp4 weight preshuffle (#28888) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
 2025-11-21 11:34:46 -05:00
Hongxia Yang
3f5f36da3f
[ROCm] Fix for import when building with upstream triton for gfx1100 for gpt-oss serving (#29127) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
 2025-11-21 03:30:07 +00:00