xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-11-03 06:07:58 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,467 Commits 155 Branches 89 Tags
Commit Graph

1777 Commits

Author SHA1 Message Date
vllmellm
40de1ef455
[FEAT] [ROCm]: Add AITER Block-Scaled GEMM Feature (#14968) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-13 19:08:20 -07:00
Harry Mellor
6223dd8114
Update deprecated type hinting in model_executor/layers (#18056) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 04:17:23 -07:00
Michael Goin
ea6ae8cb45
[Bugfix] Fix marlin moe fallback logic for llama4 (#18042) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-13 07:53:28 +00:00
Michael Goin
1df491c522
[Bugfix] Fixes for new marlin moe usage (#18017) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-13 03:50:04 +00:00
Jee Jee Li
c06af9a959
[Misc] Slight spelling modification (#18039) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-12 20:36:27 -07:00
Tao He
60f7624334
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844) 2025-05-12 19:52:47 -07:00
Michael Goin
307939f299
Use NVFP4 Marlin for CompressedTensorsW4A16Fp4 (#18000) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika <dipikasikka1@gmail.com>
 2025-05-12 18:07:34 -06:00
Harry Mellor
9d7ea9dbbf
Update some more deprecated type hinting (#17998) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-12 23:49:33 +00:00
bwshen-mi
acee8f48aa
[Model] Support MiMo-7B inference with MTP (#17433) 
Signed-off-by: wp-alpha <wangpeng66@xiaomi.com>
Co-authored-by: wangpeng66 <wangpeng66@xiaomi.com>
 2025-05-12 23:25:33 +00:00
Michael Goin
f065de4e88
Fix FBGEMM integration (#18002) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-12 23:02:07 +00:00
Isotr0py
021c16c7ca
[Model] Broadcast Ovis2 implementation to fit Ovis1.6 (#17861) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-11 17:56:30 -07:00
TJian
a810b5b088
[BugFix] [ROCm]: Bugfix and handle addition case of input for rocm_aiter_rms_norm (#17857) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-11 04:17:11 -07:00
wang.yuqi
e4b8713380
[New Model]: nomic-embed-text-v2-moe (#17785) 2025-05-11 00:59:43 -07:00
Dipika Sikka
cd3edfc908
[Misc] Add compressed-tensors NVFP4A16 emulation support (#17914) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
 2025-05-11 15:58:38 +08:00
Jinzhen Lin
d74e5f37bc
[Kernel] fp4 marlin kernel (#17687) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
 2025-05-10 19:58:49 -07:00
Pavani Majety
0c0fdae84f
[Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) 2025-05-09 16:24:41 -07:00
Michael Goin
22481fbfa3
Update CT WNA16MarlinMoE integration (#16666) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-09 13:19:45 -04:00
Michael Goin
85b72cb7b1
Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) 2025-05-09 08:58:18 -07:00
qli88
9f64e93415
[BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2025-05-09 08:59:36 -06:00
Harry Mellor
c6798baa9c
Change top_k to be disabled with 0 (still accept -1 for now) (#17773) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-09 10:01:49 +00:00
Lucas Wilkinson
5e6f939484
[Attention] MLA move rotary embedding to cuda-graph region (#17668) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-09 11:14:42 +08:00
vllmellm
3c9396a64f
[FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
 2025-05-09 10:42:05 +08:00
Shu Wang
376786fac1
Add cutlass support for blackwell fp8 blockwise gemm (#14383) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
 2025-05-08 15:09:55 -07:00
Michael Goin
4f605a6de5
Fix noisy warning for uncalibrated q_scale/p_scale (#17414) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-08 15:56:59 -04:00
Jee Jee Li
a944f8ede7
[Misc] Delete LoRA-related redundancy code (#17841) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-08 06:02:21 -07:00
Cyrus Leung
015815fe01
[Bugfix] use_fast failing to be propagated to Qwen2-VL image processor (#17838) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-08 05:39:21 -07:00
fxmarty-amd
bb239a730f
[Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) 
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
 2025-05-08 02:53:53 -07:00
xsank
0a9bbaa104
[Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) 
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
 2025-05-08 07:50:22 +00:00
Ximingwang-09
597051e56f
[Qwen3]add qwen3-235b-bf16 fused moe config on A100 (#17715) 2025-05-07 23:09:32 -07:00
Hashem Hashemi
5a499e70d5
[Kernel][Hardware][AMD] Bf16 mfma opt for ROCm skinny GEMMs (#17071) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
 2025-05-07 22:34:49 -07:00
Bowen Bao
db593aa67f
[Quantization] Quark MXFP4 format loading (#16943) 2025-05-07 15:05:05 -04:00
Yong Hoon Shin
98c89e16ff
Make key optional for rotary embedding (#17566) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-05-07 00:11:46 -07:00
Satyajith Chilappagari
043e4c4955
Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) 
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>
 2025-05-07 00:07:30 -07:00
Jee Jee Li
ba7703e659
[Misc] Remove qlora_adapter_name_or_path (#17699) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-06 23:10:37 -07:00
Wanrui Dai
f80ae5bdcf
[Kernel] Use fused rmsnorm for some models like qwen3 series (#17735) 
Signed-off-by: evian <eviantai@u.nus.edu>
Co-authored-by: evian <eviantai@u.nus.edu>
 2025-05-06 23:10:02 -07:00
Szymon Ożóg
1a45a61387
[Kernel] GGUF MoeVec kernel (#16780) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-05-06 23:07:23 -07:00
Isotr0py
c3e9d5060e
[Misc] Use apply_rotary_emb from vllm_flash_attn for Qwen2-VL vision RoPE (#17726) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-07 04:51:33 +00:00
Jee Jee Li
822de7fb94
[Misc] Split model loader (#17712) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-07 12:42:26 +08:00
Michael Goin
e50a1f1a9c
[TPU] Add kernel test for moe_pallas (#17496) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2025-05-06 17:59:57 -07:00
Chih-Chieh Yang
18dd5e01f2
[Model] Mamba2 causal conv1d Refactor to Split Prefill and Decode Requests for Corresponding Kernels (#17146) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-05-06 17:59:30 -07:00
Hongxia Yang
ed3a1d2106
[ROCm] fix num_stages for default moe config to avoid triton OutOfResource error (#17744) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
 2025-05-07 00:39:48 +00:00
Mengqing Cao
f9bc5a0693
[Bugfix] Fix triton import with local TritonPlaceholder (#17446) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-05-06 17:53:09 +08:00
Stan Wozniak
999328be0d
[Model] Add GraniteMoeHybrid 4.0 model (#17497) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Thomas Ortner <boh@zurich.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-05-06 12:00:31 +08:00
Jinzhen Lin
1d0c9d6b2d
[Kernel] some optimizations for dense marlin and moe marlin (#16850) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
 2025-05-05 09:39:30 -07:00
Harry Mellor
d6484ef3c3
Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
rasmith
e3d0a1d190
[Quantizaton] [AMD] Add support for running DeepSeek int8 w8a8 MoE on ROCm (#17558) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-05-02 21:41:10 -07:00
Eric Hartford
9b103a1d76
fix typo in logging (#17605) 2025-05-02 18:04:40 -07:00
Xiaodong Wang
9352cdb56d
[Hardware][AMD] Improve OAM device ID + llama4 Maverick MOE tuning (#16263) 
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Lu Fang <lufang@fb.com>
 2025-05-02 19:44:19 +00:00
Caleb_Du
3e887d2e0c
permute/unpermute kernel for moe optimization (#14568) 
Signed-off-by: Caleb_Du <Caleb_Du@zju.edu.cn>
 2025-05-02 11:31:55 -07:00
Michael Goin
868c546da4
Support W8A8 INT8 MoE for compressed-tensors (#16745) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-02 10:03:32 -04:00