xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-21 04:45:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,065 Commits 155 Branches 89 Tags
Commit Graph

10065 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Juan Villamizar
81ee45298d [ROCm] Small functional changes for gptoss (#25201) 
Signed-off-by: jpvillam <jpvillam@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Thomas Parnell
d12433adfc [Kernel] [Mamba] Remove BLOCK_H=1 from list of tuneable configurations for _chunk_cumsum_fwd_kernel (#25197) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Corey Lowman
4ebc513fc1 Add VLLM_NVTX_SCOPES_FOR_PROFILING=1 to enable nvtx.annotate scopes (#25501) 
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
kourosh hakhamaneshi
7a8f0a3548 [BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Michael Goin
907bbca7b7 Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Andrew Xia
eb1f43bc82 [gpt-oss][bugfix] remove logic to require resp_ in ResponseAPI (#25428) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Michael Goin
99eaeebe66 Fix triton_reshape_and_cache_flash.py triton import (#25522) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
rouchenzi
715e24e1b3 Add VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE & VLLM_ENABLE_INDUCTOR_COORDINA… (#25493) 
Signed-off-by: rouchenzi <ruochenwen@gmail.com>
Signed-off-by: rouchenzi <40842833+rouchenzi@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Thomas Parnell
cf0e250200 [V0 Deprecation] Remove placeholder attn (#25510) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Jialin Ouyang
0c11617ff1 [Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Isotr0py
930e691c65 [CI/Build] Fix and re-enable v1 PP test on CI (#25496) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Gregory Shtrasberg
c0f11557e1 [Bugfix] Fix for the import error from #24588 (#25481) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Chauncey
0438c65376 [Build] Update Xgrammar to 0.1.25 (#25467) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Alec S
d8fda7420a [Bugfix] gpt-oss container tool output bug (#25485) 
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Wentao Ye
86e5b73d71 [CI] Fix Pre-commit Issue (#25497) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Ilya Markov
e49561cd91 Enable symmetric memory all reduce by default only enabling for TP (#25070) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Michael Goin
0e30643147 [Bugfix] Lower gpt-oss max cudagraph size to 992 to be compatible with FA3 (#25508) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
jiahanc
8ba3b17cc1 [Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Michael Goin
8222e2651d [Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
ElizaWszola
b672b8c3b8 [Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Amir Samani
56201cfb01 [core] add nccl symmetric memory for all reduce (#24532) 
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Hashem Hashemi
9689be1e8e [ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Jee Jee Li
65c4513ad8 [Core] Ensure LoRA linear respect the base_layer's tp_size and tp_rank (#25487) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Ekagra Ranjan
5acda4cc71 [Spec Decode][CI] Add e2e test for examples/spec_decode.py and prevent breaking Acceptance Length (#24531) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Weida Hong
78f892c373 [Misc] Reduce initialization time of auto_tune (#23682) 
Signed-off-by: Weida Hong <wdhongtw@google.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Burkhard Ringlein
26da2c6244 [V1][Kernel] Add triton implementation for reshape_and_cache_flash (#24503) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Ming Yang
0081c6956a Use macro guard CUDA functions for back compatibility in grouped_topk_kernel.cu (#25346) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Wentao Ye
6462feef65 [Log] Optimize kv cache memory log from Bytes to GiB (#25204) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
rivos-shreeasish
e9a74500e5 [BugFix] Fix UB in per_token_group_quant.cu (#24913) 
Signed-off-by: Shreeasish Kumar <shreeasish@rivosinc.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
bnellnm
02a3ce2230 [Kernels] Support blocked fp8 quantization for compressed tensors MoE (#25219) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Harry Mellor
9cae377a16 Add backward compatibility for GuidedDecodingParams (#25422) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Lucas Wilkinson
8c5c35c027 [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Thomas Parnell
f97da2c732 [V1] Remove V0 code paths for Hybrid models (#25400) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Michael Goin
02134245a9 [UX] Change kv-cache-memory log level to debug (#25479) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Fanli Lin
2ab27b70f5 [XPU] Fix MOE DP accuracy issue on XPU (#25465) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Peter Pan
a500f7cc09 [Docs] NixlConnector quickstart guide (#24249) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Chauncey
1b75f784b8 [P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Joel
0eddd2b528 [BugFix] Register expert_map as named buffer for wake_up and sleep (#25458) 
Signed-off-by: wuxibin <wuxibin@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Isotr0py
030774abcf [CI/Build] Fix disabled v1 attention backend selection test (#25471) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
vllmellm
77389d87b2 [docs] Benchmark Serving Incorrect Arg (#25474) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Jee Jee Li
59659b74c4 [Core] Optimize LoRA weight loading (#25403) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Isotr0py
3b96eafdb0 [Bugfix] Fix idefics3 tie_word_embeddings (#25454) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Andreas Hartel
fb64e67533 [Test]: Hermes tool parser stream output error in Qwen3 case (#25203) 
Signed-off-by: Andreas Hartel <andreas.hartel@aleph-alpha.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Cyrus Leung
215da8510d [Misc] Move DP for ViT code inside model executor dir (#25459) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Zhikaiiii
c4a15ee240 [Frontend] Add a new xml-based tool parser for qwen3-coder (#25028) 
Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Ming Yang
3a640b8f74 Handle triton kernel import exception (#25319) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Cyrus Leung
0a1397c7df [Model] Enable DP for ViT in Qwen2-VL (#25445) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Chendi.Xue
921945c81e [NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Varun Sundar Rabindranath
675fc471bf [DP/EP][GPTOSS] Use triton matmul-ogs kernels for GPTOSS DP/EP (#24588) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Michael Yao
b0ae0ad935 [Docs] Fix griffe warnings in vllm/lora/ops (#25369) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00