xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-29 10:40:54 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,933 Commits 155 Branches 89 Tags
Commit Graph

3052 Commits

Author SHA1 Message Date
Seiji Eicher
41174e2803 [ray][metrics] Replace ':' with '_' for OpenTelemetry compatibility in Ray (#25439) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Cyrus Leung
129a643b4c [CI/Build] Fix some V1 tests not being run (#25569) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Cyrus Leung
d3c732e985 [CI/Build] Split up Distributed Tests (#25572) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
wang.yuqi
fb0eece290 [Bugfix] Properly abort pooling request. (#25734) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Chauncey
515e30b023 [CI] Fix test_shared_storage_connector_hashes (#25748) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Chih-Chieh Yang
62ae26c870 [Model] Mamba2 varlen refactor (#21467) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: RishiAstra <40644327+RishiAstra@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Eugene Khvedchenya
9e6628ccfc EVS Support (Video tokens pruning) (#22980) 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
tomeras91
1d1436c3f7 [Model] rename NemotronH_Nano_VL -> NemotronH_Nano_VL_V2 (#25708) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Ekagra Ranjan
f3a478b55e [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Cyrus Leung
b0e9f04bbd [Misc] Simplify test_argsort_mm_positions (#25690) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Matthew Bonanni
a355561291 [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Cyrus Leung
34e6a31e40 [Model] Define merge_by_field_config MM interface (#25676) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Nicolò Lucchesi
91d4299774 [Misc] Remove cruft file in repo (#25678) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Isotr0py
f7f76a8668 [Bugfix] Fix InternS1 video processing after Transformers v4.56 (#25644) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Cyrus Leung
af10a37c6c [mypy] Fix wrong type annotations related to tuple (#25660) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Cyrus Leung
222411313d [CI/Build] Fix flaky entrypoints test (#25663) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
wang.yuqi
f3d9099b44 [V0 deprecation] Remove unreachable model_config.supported_tasks (#25642) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Agata Dobrzyniewicz
034c0152db [Bugfix] Add triton.language.tensor placeholder (#25649) 
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Tyler Michael Smith
6c6e553644 Revert "[Performance] Move apply_w8a8_block_fp8_linear to an op class… (#25607) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Jacob Kahn
004eed39ff Map CwmForCausalLM to llama and LlamaForCausalLM (#25611) 
Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
XuruiYang
c26e7b14d7 [Model] Add LongCat-Flash (#23991) 
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Wei Wei
b839194931 [Kernel] Support DCP for Triton backend (#25132) 
Signed-off-by: Wei Wei <wwei6@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Wentao Ye
b3fd4ed80c [Refactor] Use DeepGEMM Col Major TMA Aligned Tensor (#25517) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Shu Wang
72a5101c7a Support mnnvl all2allv from Flashinfer (#21003) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Tao Hui
71566e8afc [Bugfix] Fix DeepSeekV31ToolParser to correctly parse multiple tools in non-streaming output (#25405) 
Signed-off-by: taohui <taohui3@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:55 -07:00
Jonas M. Kübler
f98d4d38c0 [Bug] fix import and unit test (#25558) 
Signed-off-by: Jonas M. Kübler <44084297+jmkuebler@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Woosuk Kwon
9914857f2b [V0 Deprecation] Remove max_seq_len_to_capture (#25543) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Cyrus Leung
4ca175ea0b [Misc]] Move processing context to multimodal directory (#25548) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Isotr0py
c39befcead [CI/Build] Fix v1 OOT registration test (#25547) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Chengji Yao
5b4ba2e1e1 [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Ben Browning
d7fb5a4ae8 [Bugfix] [Frontend] Cleanup gpt-oss non-streaming chat tool calls (#25514) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Benjamin Chislett
177c37e960 [Spec Decode] Enable FlashInfer Spec Decoding (#25196) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
0xNullPath
3c62d28bb9 [Model] Support SeedOss Reason Parser (#24263) 
Signed-off-by: Yan Lu <luyan@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
ahao-anyscale
9596fbd6e5 [BUG] Allows for RunAI Streamer and Torch.compile cache to be used together (#24922) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Doug Smith
b50fa00537 Improve output when failing json.loads() on structured output test (#25483) 
Signed-off-by: dougbtv <dosmith@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
kourosh hakhamaneshi
7a8f0a3548 [BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Andrew Xia
eb1f43bc82 [gpt-oss][bugfix] remove logic to require resp_ in ResponseAPI (#25428) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Thomas Parnell
cf0e250200 [V0 Deprecation] Remove placeholder attn (#25510) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Jialin Ouyang
0c11617ff1 [Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Isotr0py
930e691c65 [CI/Build] Fix and re-enable v1 PP test on CI (#25496) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Ilya Markov
e49561cd91 Enable symmetric memory all reduce by default only enabling for TP (#25070) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
jiahanc
8ba3b17cc1 [Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
ElizaWszola
b672b8c3b8 [Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Amir Samani
56201cfb01 [core] add nccl symmetric memory for all reduce (#24532) 
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Hashem Hashemi
9689be1e8e [ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Burkhard Ringlein
26da2c6244 [V1][Kernel] Add triton implementation for reshape_and_cache_flash (#24503) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Harry Mellor
9cae377a16 Add backward compatibility for GuidedDecodingParams (#25422) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Lucas Wilkinson
8c5c35c027 [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Thomas Parnell
f97da2c732 [V1] Remove V0 code paths for Hybrid models (#25400) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00
Peter Pan
a500f7cc09 [Docs] NixlConnector quickstart guide (#24249) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-03 13:35:54 -07:00