xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-26 20:01:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,312 Commits 155 Branches 89 Tags
Commit Graph

2271 Commits

Author SHA1 Message Date
Wentao Ye
0d4891cd03
[Bug] Fix DeepGemm for EP low latency case (#20833) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-11 23:05:12 -07:00
Nicolò Lucchesi
3c7d942da8
[Frontend] Abstract prompt and SpeechToTextConfig for transcriptions models (#20637) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-11 21:33:26 -07:00
Varun Sundar Rabindranath
890323dc1b
[Bugfix] : Fix typo - logger.warn_once -> logger.warning_once (#20852) 2025-07-11 20:56:24 -07:00
Isotr0py
01cae37713
[CI/Build] Ensure compatability with Transformers v4.53 (#20541) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-11 20:53:07 -07:00
yurhett
11c0198615
[Bugfix] Fix tensor parallel issue in Qwen3 reranker weight loading (#20682) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-07-11 20:52:43 -07:00
Li, Jiang
b1235c3e10
[Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break not-cuda-alike devices (#20822) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-11 20:52:05 -07:00
Jee Jee Li
44d02f54db
[Misc] Restrict deep_gemm's log output (#20827) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-11 20:50:42 -07:00
Varun Sundar Rabindranath
53fa457391
[Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-11 07:51:46 -07:00
Jee Jee Li
8020e98c9f
[Quantization][1/N] MoE support BNB-Inflight Quantization (#20061) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-11 08:01:13 +00:00
nopperl
5d09152ff1
[V1] Enable Mamba2 layers other than MambaMixer2 in the v1 engine (#20660) 
Signed-off-by: nopperl <54780682+nopperl@users.noreply.github.com>
 2025-07-11 05:53:31 +00:00
Luka Govedič
31d5c1797f
[Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 04:56:28 +00:00
Wentao Ye
e2de455c34
[Feature] Integrate SM100 DeepGEMM support (#20087) 2025-07-10 20:18:05 -07:00
Michael Goin
922f316441
[Model] Support HF format of minimax (#20211) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-11 02:55:21 +00:00
Duncan Moss
5923ab9524
[fix]: disable cutlass block scaled group gemm for EP (#20781) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
 2025-07-11 02:39:18 +00:00
Simon Mo
b854321ffe
[Docs] Lazy import gguf (#20785) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-07-10 16:06:37 -07:00
Varun Sundar Rabindranath
f0c98cae27
[Misc] MoE ModularKernel : Introduce TopKWeightAndReduce (#20648) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-10 14:40:38 -07:00
Varun Sundar Rabindranath
fdadb6f43a
[Bugfix] Fused MoE Modular Kernel chunking loop (#20392) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-10 20:31:10 +00:00
Ming Yang
3de2ed767f
[Bugfix] Remove assertion of expert_map being None (#20714) 
Signed-off-by: Ming Yang <yming@meta.com>
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-07-10 19:55:22 +00:00
Nathan Hoos
d6902ce79f
[V0][V1][Core] Add outlines integration for V1, and update V0 integration. (#15975) 
Signed-off-by: Nathan Hoos <thwackyy.y@gmail.com>
 2025-07-10 15:30:26 -04:00
Sanger Steel
5e53c89a74
[Bugfix] [CI] Fix Tensorizer LoRA test (#20760) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
 2025-07-10 19:07:06 +00:00
shineran96
4bed167768
[Model][VLM] Support JinaVL Reranker (#20260) 
Signed-off-by: shineran96 <shinewang96@gmail.com>
 2025-07-10 10:43:43 -07:00
Harry Mellor
3482fd7e4e
[Doc] Add engine args back in to the docs (#20674) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-10 08:02:40 -07:00
Seiji Eicher
ad6c2e1a0b
Correct PPMissingLayer handling in Deepseek-V2-Lite PP deployment (#20665) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-07-09 20:34:40 -07:00
Varun Sundar Rabindranath
805d62ca88
[Misc] DP : Add ExpertTokensMetadata (#20332) 
Signed-off-by: Varun <vsundarr@redhat.com>
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-07-10 00:33:14 +00:00
fxmarty-amd
332d4cb17b
[Feature][Quantization] MXFP4 support for MOE models (#17888) 
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>
 2025-07-09 13:19:02 -07:00
Jacob Manning
bf03ff3575
[Kernel] Add Conch backend for mixed-precision linear layer (#19818) 
Signed-off-by: Jacob Manning <jmanning+oss@stackav.com>
 2025-07-09 13:17:55 -07:00
Tuan, Hoang-Trong
47043eb678
[Kernel] Triton implementation of causal-conv1d for Mamba-based models (#18218) 
Signed-off-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-07-09 12:53:55 -07:00
Michael Goin
31b96d1c64
Support Llama 4 for cutlass_moe_fp4 (#20453) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-09 15:53:38 -04:00
Sanger Steel
4ac9c33f78
[Bugfix] Fix handling of Tensorizer arguments for LoadConfig (#20643) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
 2025-07-09 15:36:37 +00:00
Duncan Moss
97abeb1daa
[feat] enable SM100 CUTLASS block scaled group gemm for smaller batch sizes (#20640) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
 2025-07-09 11:03:35 +08:00
Isotr0py
b9fca83256
[Bugfix] Fix GLM-4.1-V video prompt update (#20635) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-07-08 23:13:58 +00:00
Cyrus Leung
32dffc2772
[Core] Rename get_max_tokens_per_item for backward compatibility (#20630) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-08 23:11:30 +00:00
Ming Yang
c438183e99
[Bugfix] Fix topk_ids indices_type for CUTLASS w8a8 FP8 MoE (#20166) 
Signed-off-by: Ming Yang <yming@meta.com>
 2025-07-08 23:10:57 +00:00
viravera
c6c22f16d3
Revert invalid spellchecker fix on deepseek_vl2 (#20618) 2025-07-08 15:07:14 +00:00
Cyrus Leung
dd382e0fe3
[Model] Implement missing get_language_model for Keye-VL (#20631) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-08 07:47:46 -07:00
Yan Ma
a4c23314c0
[xpu]feat: support multi-lora on xpu (#20616) 
Signed-off-by: yan <yan.ma@intel.com>
 2025-07-08 22:07:10 +08:00
Sanger Steel
72d14d0eed
[Frontend] [Core] Integrate Tensorizer in to S3 loading machinery, allow passing arbitrary arguments during save/load (#19619) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Eta <esyra@coreweave.com>
 2025-07-07 22:47:43 -07:00
Ming Yang
afb7cff1b9
[Bugfix] Fix Maverick correctness by filling zero to cache space in cutlass_moe (#20167) 
Signed-off-by: Ming Yang <yming@meta.com>
 2025-07-08 01:07:22 +00:00
Patrick von Platen
14601f5fba
[Config] Refactor mistral configs (#20570) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-07-07 15:25:10 -07:00
Harry Mellor
042d131f39
Fix links in multi-modal model contributing page (#18615) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-07 21:13:52 +00:00
wang.yuqi
110df74332
[Model][Last/4] Automatic conversion of CrossEncoding model (#19675) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-07-07 14:46:04 +00:00
Cyrus Leung
c18b3b8e8b
[Bugfix] Add use_cross_encoder flag to use correct activation in ClassifierPooler (#20527) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-06 14:01:48 -07:00
Brayden Zhong
cede942b87
[Benchmark] Add support for multiple batch size benchmark through CLI in benchmark_moe.py (#20516) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-07-06 09:20:11 +00:00
Chengji Yao
4548c03c50
[TPU][Bugfix] fix the MoE OOM issue (#20339) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-07-05 21:19:09 -07:00
Lucia Fang
432870829d
[Bugfix] Fix missing per_act_token parameter in compressed_tensors_moe (#20509) 
Signed-off-by: Lu Fang <fanglu@fb.com>
 2025-07-06 12:08:30 +08:00
Isotr0py
32c9be2200
[v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-05 09:41:10 +00:00
Lucia Fang
8aeaa910a2
Fix unknown attribute of topk_indices_dtype in CompressedTensorsW8A8Fp8MoECutlassMethod (#20507) 
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
 2025-07-05 14:03:20 +08:00
Duncan Moss
3d184b95b8
[feat]: CUTLASS block scaled group gemm for SM100 (#19757) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Co-authored-by: Duncan Moss <dmoss@nvidia.com>
 2025-07-04 12:58:04 -06:00
Thomas Parnell
2f35a022e6
Enable V1 for Hybrid SSM/Attention Models (#20016) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-07-04 17:46:53 +00:00
wang.yuqi
2e26f9156a
[Model][3/N] Automatic conversion of CrossEncoding model (#20168) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-07-04 05:47:39 -07:00