xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 01:24:54 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,306 Commits 155 Branches 89 Tags
Commit Graph

9306 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Lifans
d6069887c6
[rocm] enable torchao quantization for rocm (#24400) 
Signed-off-by: Lifan Shen <lifans@meta.com>
 2025-09-10 06:16:21 -07:00
Ye (Charlotte) Qi
492196ed0e
[CI/Build] split true unit tests to Entrypoints Unit Tests (#24418) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-10 06:16:07 -07:00
Nick Hill
f4f1a8df22
[BugFix] Ensure integrity of reused CPU tensors during async scheduling (#24527) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: guoze.lin <guozelin@tencent.com>
 2025-09-10 21:15:14 +08:00
lacora
0b9a612fa3
[BugFix][easy] Fix flaky test test_gpt_oss_multi_turn_chat (#24549) 
Signed-off-by: lacora2017 <yehu@meta.com>
Co-authored-by: lacora2017 <yehu@meta.com>
 2025-09-10 21:14:55 +08:00
Wenlong Wang
4c04eef706
[BugFix][Multi Modal] Fix TensorSchema shape mismatch in Molmo (#24559) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-10 06:14:27 -07:00
Harry Mellor
f36355abfd
Move LoadConfig from config/__init__.py to config/load.py (#24566) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-10 06:14:18 -07:00
Yash Pratap Singh
9e3c3a7df2
[LoRA]: Add LoRA support to Mistral's Voxtral models (#24517) 
Signed-off-by: Yash Pratap Singh <yashsingh20001@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-10 06:12:03 -07:00
baonudesifeizhai
6cbd41909e
Feature/vit attention unification# 23880 (#23978) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-10 06:10:14 -07:00
danielafrimi
72d30108a0
Support for NemotronH Nano VLM (#23644) 
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
 2025-09-10 06:10:06 -07:00
Tyler Michael Smith
8b83b93739
[Docs] Document the extra memory footprint overhead when using EPLB (#24537) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-10 06:09:49 -07:00
Harry Mellor
9dbefd88e9
[Docs] Improve organisation of API Reference nav (#24569) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-10 06:08:21 -07:00
vllmellm
7c195d43da
[ROCm][Bugfix] Fix Aiter RMSNorm (#23412) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-09-10 21:08:03 +08:00
Lucas Wilkinson
0ae43dbf8c
[Attention] add DCP support for FLASH_ATTN_MLA backend (#24453) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-10 17:19:26 +08:00
li-jinpeng
267c80d31f
[Model] Limit CPU threads for image transformations in InternVL to reduce cpu contention. (#24519) 
Signed-off-by: li-jinpeng <3332126450@qq.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-10 16:45:44 +08:00
Flora Feng
77f62613f9
Consolidate rendering parameters into RenderConfig dataclass (#24543) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 08:44:47 +00:00
Remy
feaf202e93
[Bugfix] Guard _may_reorder_batch for encoder-only models on CPU (#24319) (#24348) 
Signed-off-by: Remy <eunhwan.shin@dtonic.io>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2025-09-10 14:24:42 +08:00
Simon Mo
91130ae376
[docs] promo pytorch conf and ray summit (#24562) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-09-09 23:24:20 -07:00
Harry Mellor
e40827280b
[Docs] Enable relative links in examples to function when rendered in the docs (#24041) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-09 21:40:45 -07:00
pwschuurman
4377b1ae3b
[Bugfix] Update Run:AI Model Streamer Loading Integration (#23845) 
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Signed-off-by: Peter Schuurman <psch@google.com>
Co-authored-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-09 21:37:17 -07:00
Chenheli Hua
009d689b0c
[Core] Simplify and unify mm uuid handling & auto-generated mm hash overrides processing. (#24271) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-09-09 21:36:09 -07:00
Wei
0efdb5c3ba
[gpt-oss] Cache permute indices for faster MXFP4 MoE layer loading (#24154) 
Signed-off-by: Wei Wei <wwei6@meta.com>
 2025-09-10 04:27:53 +00:00
Wenlong Wang
53b42f4102
[BugFix][Spec Decode] Fix out-of-range index triggered by eagle3; re-enable test for LlamaForCausalLMEagle3 (#24392) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-09 21:24:23 -07:00
Chauncey
309d7aa401
[P/D] MultiConnector supports shutdown (#24425) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-09 21:24:11 -07:00
Yihua Cheng
b4a01aaf95
[KV Connector] More async support for get_num_new_matched_tokens (#23620) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
 2025-09-09 21:23:37 -07:00
Nick Hill
83dd28aae4
[CI] Adjust threshold for flaky ngram spec decoding test (#24528) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-09 21:07:33 -07:00
Nick Hill
f88e84016f
[BugFix] Fix async core engine client finalizer (#24540) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-09 21:07:13 -07:00
Ignacio Sica
3c2156b3af
[Hardware][Apple-CPU] Enable native bfloat16 on Apple Silicon (M2 and later) (#24129) 
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
 2025-09-10 03:50:21 +00:00
Nick Hill
7e7db04310
[CI] Retry flaky fp8 cutlass mla tests (#24536) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-09 20:33:10 -07:00
Chen Zhang
41f160b974
Add @heheda12345 to CODEOWNERS of KVCacheManager related code (#24546) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-10 03:30:32 +00:00
Yong Hoon Shin
dc625ea6b8
[Perf] Convert np array to torch tensor to index into block table for attn chunking (#24474) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-09-09 20:01:06 -07:00
bnellnm
b23fb78623
[Bugfix] Fix for 24530. Fix naive all2all shared expert overlap. (#24538) 2025-09-09 17:53:53 -07:00
Tyler Michael Smith
561f38dc3c
[Bugfix] Improve EPLB config validation error message (#24524) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-10 00:32:36 +00:00
Charlie Fu
73e688cb79
[ROCm][Feature] Enable Pipeline Parallelism with Ray Compiled Graph on ROCm (#24275) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-09-09 23:27:35 +00:00
Ekagra Ranjan
fb1a8f932a
[Benchmark] Add option to skip oversampling in benchmark (#24457) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2025-09-09 22:00:17 +00:00
Ekagra Ranjan
0dc9cbb527
[Benchmark] Update bench doc with mtbench, blazedit, spec bench (#24450) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2025-09-09 21:15:41 +00:00
Jiangyun Zhu
b5fb3005a8
[Log] Use a relative path in debug-level logs to distinguish files with identical names (#23846) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-09-09 16:46:35 -04:00
Wentao Ye
15de5ff9ea
[Feature] Disallow FlashMLA on Blackwell (#24521) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-09 14:59:34 -04:00
Jiangyun Zhu
b8a93076d3
[CI] execute all piecewise compilation tests together (#24502) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
v0.10.2rc1 		2025-09-09 11:05:25 -07:00
Chenyaaang
c3f9773b2c
[TPU] Fix tpu structured decoding in mixed batches (#24458) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-09-09 11:04:25 -07:00
Nicolò Lucchesi
3707cb2505
[Docs] Gemma3n transcriptions endpoint support (#24512) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-09 11:03:32 -07:00
Kazuhiro Serizawa
920ed46b09
[Misc] bump outlines_core to fix the version conflicts with outlines >= 1.2.0 (#24368) 
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-09-09 10:59:46 -07:00
Flora Feng
15cb047e25
Extend renderer with embedding support and integrate completion endpoint (#24405) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 01:46:46 +08:00
Jee Jee Li
9ad0688e43
[Bugfix] Fix hidden_size for multimodal classification model (#24501) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-09 10:37:25 -07:00
Gregory Shtrasberg
b9a1c4c8a2
[ROCm][CI/Build] Sync ROCm dockerfiles with the ROCm fork (#24279) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-09 12:21:56 -04:00
youkaichao
1aa427fdc1
[Kernels] Add Flash Linear Attention Kernels (#24518) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-10 00:04:41 +08:00
Micah Williamson
1c63a16b65
[Core] Run garbage collector after CUDA graph capture to fix throughput regression (#24128) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-09 10:38:10 -04:00
d.transposed
922d3b401b
[Bugfix] Handle the edge case in detokenizer where processed tokens contain both stop str and eos token (#23938) 
Signed-off-by: dtransposed <damian.bogunowicz@gmail.com>
 2025-09-09 07:30:24 -07:00
wang.yuqi
19332c0479
[Model] Systematic support for fp32 head, pooling models part (#23810) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-09 07:29:50 -07:00
Wentao Ye
a55cf41a09
[Compilation][WideEP] Enable Piecewise CUDAGraph for DeepEPHT (#24123) 2025-09-09 10:21:10 -04:00
Ye (Charlotte) Qi
6fb2788163
[CI/Build][Doc] Fully deprecate old bench scripts for serving / throughput / latency (#24411) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-09 10:02:35 +00:00