xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-20 23:55:59 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,065 Commits 155 Branches 89 Tags
Commit Graph

10065 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Cyrus Leung
00c0b25e82 [Model] Use merge_by_field_config for MM models (A-C) (#26073) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Lucas Wilkinson
0655b90d80 [FA/Chore] Bump vllm-flash-attention (#25537) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Thomas Parnell
83fa298682 Change size of single CUDA graph for CI to 4 (#26089) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Huy Do
5a083ce2ea Update base image to 22.04 (jammy) (#26065) 
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
pwschuurman
115019045d Run:ai model streamer add GCS package support (#24909) 
Signed-off-by: Peter Schuurman <psch@google.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Nick Hill
93d2be10b6 [Misc] Make handling of SamplingParams clearer in n>1 case (#26032) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Gregory Shtrasberg
91e10c725c [ROCm][Bugfix] Add missing parameter to ROCm backend (#26029) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Jerry Zhang
2ae74a80af Support RL online quantization with torchao (#23014) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Lucas Wilkinson
ac1598d166 [BugFix] ChunkedLocalAttention is currently not CG compatible (#26034) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Wentao Ye
ce8ee3d9e7 [Bug] Fix Negative Cuda Memory Usage (#25683) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Hosang
d4a83e01bb [ROCm][Build] Add support for AMD Ryzen AI MAX / AI 300 Series (#25908) 
Signed-off-by: Hosang Yoon <hosang.yoon@amd.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Lucas Wilkinson
90529cec41 [BugFix][DP/EP] Fix CUTLASS MLA hang under load (#26026) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Michael Goin
bba7623426 [CI] Tweaks to GPT-OSS Eval (Blackwell) for stability (#26030) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Huamin Li
d2f544018f Fix test_mamba_ssm_ssd.py due to missing _query_start_loc_to_chunk_indices_offsets (#25995) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Johnny
ed7eb771a3 [NVIDIA] Blackwell Family (#24673) 
Signed-off-by: Johnny <johnnynuca14@gmail.com>
Signed-off-by: johnnynunez <johnnynuca14@gmail.com>
Signed-off-by: Johnny <johnnync13@gmail.com>
Signed-off-by: Salvatore Cena <cena@cenas.it>
Co-authored-by: Aidyn-A <31858918+Aidyn-A@users.noreply.github.com>
Co-authored-by: Salvatore Cena <cena@cenas.it>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Kenichi Maehashi
0944358a90 [Bugfix] Apply same sampling parameters for both n=1 and n>1 (#26005) 
Signed-off-by: Kenichi Maehashi <maehashi@preferred.jp>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Nathan Scott
aeff0604bb [Benchmark] Finish documented v0.11.0 deprecation of --endpoint-type (#26007) 
Signed-off-by: Nathan Scott <nathans@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
billishyahao
a561b9832d [MISC] Fix misleading batch_size_capture_list when cuda_graph_sizes < 4 (#25829) 
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Luka Govedic <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Harry Mellor
e8773e620f [CI] Only capture a single CUDA graph size in CI by default (#25951) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Cyrus Leung
63c56cbb25 [Misc] Factor out common _apply_feature_select_strategy (#26003) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Wenlong Wang
25e5b9ccec [BugFix][MM] Fix Nonetype error when video is cache in qwen2.5-omni-thinker (#26004) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
nadathurv
b9ed8c9679 [Doc] updating torch.compile doc link (#25989) 
Signed-off-by: nadathurv <work.vnadathur@gmail.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Co-authored-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Lucia Fang
9506409fc6 [Misc]allow disable pynccl (#25421) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Harry Mellor
fda819837e Update to Transformers v4.56.2 (#24638) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Lucas Wilkinson
7c795fdf41 [BugFix] Fix default kv-cache-dtype default for DeepseekV3.2 (#25988) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Zhewen Li
6444f65a2b [Bugfix] Fix __syncwarp on ROCM (#25996) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Roger Wang
4c094b339e [MM] Add text-only mode for Qwen3-VL (#26000) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Param
cd0bbf5de2 Fix INT8 quantization error on Blackwell GPUs (SM100+) (#25935) 
Signed-off-by: padg9912 <phone.and.desktop@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Wentao Ye
2b6b859916 [Log] Optimize Log for FP8MOE (#25709) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Salvatore Cena
04cb503fda Update launch_bounds_utils.h for correct compile on Multiple Cuda Arch - PTXAS out of range Warning (#25843) 
Signed-off-by: Salvatore Cena <cena@cenas.it>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Lucia Fang
d437ba32fd [Model] MTP fallback to eager for DeepSeek v32 (#25982) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Lucas Wilkinson
e734a2a085 [Misc] Make EP kernels install script support uv (#25785) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Andrew Xia
fd56f2e644 [gpt-oss] use vLLM instead of openai types for streaming (#25186) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Harry Mellor
1690954497 [Docs] Remove API Reference from search index (#25949) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Harry Mellor
b3e1846da6 Add explicit pooling classes for the Transformers backend (#25322) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
David Ben-David
8328d39d40 [V1] [P/D] Add Support for KV Load Failure Recovery (#19330) 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Jee Jee Li
ef318228e7 [Bench] Add DeepSeekV32 to MoE benchmark (#25962) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
cjackal
8ecccdd15f [Llama4] [multimodal] Fix misplaced dtype cast of cos_sin_cache in Llama4VisionRotaryEmbedding (#25889) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Or Ozeri
bb2e04e41e OffloadingConnector: Fix GPU block tracking bug (#25856) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
bnellnm
6083b4d926 [Docs] Add moe kernel features doc (#25297) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Cyrus Leung
493acdb7e2 [Doc] Improve MM Pooling model documentation (#25966) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Wentao Ye
3c75d3b00c [Bug] Fix AttributeError: 'QKVParallelLinear' object has no attribute 'orig_dtype' (#25958) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
youkaichao
206ab1f0df [bugfix][deepseek] fix flashmla kernel selection (#25956) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Cyrus Leung
e33579cd96 [Bugfix] Token type and position embeddings fail to be applied to inputs_embeds (#25922) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Pavani Majety
8c52fccb1a [Bugfix] Fix accuracy issue of TRTLLM FP8 MOE and improve logging (#25895) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Anion
ea6144a019 [Bugfix][Model] Fix inference for Hunyuan dense models (#25354) 
Signed-off-by: anion <1005128408@qq.com>
Signed-off-by: Anion <123177548+Anionex@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Sergio Paniego Blanco
b6ea29b721 Add Hugging Face Inference Endpoints guide to Deployment docs (#25886) 
Signed-off-by: sergiopaniego <sergiopaniegoblanco@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Asaf Joseph Gardin
d9f8ded136 [Kernel][Moe Configs] Add more tuned triton configs for ExpertsInt8 and FP8 (#25858) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
ihb2032
02776c0386 [Fix] Improve CPU backend compatibility for RISC-V (#25816) 
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00
Reza Barazesh
8914d52869 [CI] Move applicable tests to CPU (#24080) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:57 -07:00