xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-09 18:27:09 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,793 Commits 155 Branches 89 Tags
Commit Graph

233 Commits

Author SHA1 Message Date
Lucas Kabela
3da17c2cc2
[Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2025-09-19 20:27:21 -04:00
Jee Jee Li
2821986450
[Core] Modify the initialization parameters of the lora manager (#25249) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-19 18:01:28 +00:00
Aziz
38db529f66
[feat]: Create interface for model-specific M-RoPE (#24194) 
Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-18 19:18:56 +00:00
Boyuan Feng
94e6b2d55f
Allow users to specify kv cache memory size (#21489) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 13:41:07 +00:00
Maximilien de Bayser
2554b27baa
[V0 Deprecation] Remove pooling model support in V0 (#23434) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-29 00:04:02 -07:00
Woosuk Kwon
1c859a1387
[V0 Deprecation] Remove advance_step (#22969) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-15 08:22:31 -07:00
Woosuk Kwon
71683ca6f6
[V0 Deprecation] Remove multi-step scheduling (#22138) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-08-12 20:18:39 -07:00
Richard Zou
04e38500ee
[Bugfix] VLLM_V1 supports passing other compilation levels (#19340) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-07-29 09:35:58 -04:00
KazusatoOoko
fd48d99ffd
[BugFix]: Batch generation from prompt_embeds fails for long prompts (#21390) 
Signed-off-by: KazusatoOko <kazusto.oko@sakana.ai>
Co-authored-by: KazusatoOko <kazusto.oko@sakana.ai>
 2025-07-23 20:43:17 -07:00
Michael Goin
82ec66f514
[V0 Deprecation] Remove Prompt Adapters (#20588) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-23 16:36:48 -07:00
Congcong Chen
2c11a738b3
[Model] New model support for microsoft/Phi-4-mini-flash-reasoning (#20702) 
Signed-off-by: Congcong Chen <congcongchen@microsoft.com>
 2025-07-12 06:02:10 -07:00
lkchen
f56d2996ca
[Misc] Respect no_use_tqdm_on_load flag while capturing CUDA graph (#20834) 
Signed-off-by: Linkun <github@lkchen.net>
 2025-07-11 23:04:45 -07:00
Sanger Steel
72d14d0eed
[Frontend] [Core] Integrate Tensorizer in to S3 loading machinery, allow passing arbitrary arguments during save/load (#19619) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Eta <esyra@coreweave.com>
 2025-07-07 22:47:43 -07:00
Lukas Geiger
1409ef9134
[Core] Cast multimodal input in hf processor (#18862) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-06-03 20:24:56 -07:00
Simon Mo
02f0c7b220
[Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Cyrus Leung
696259ca01
[Core] Automatically cast multi-modal input dtype (#18756) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-27 23:45:48 +08:00
燃
f6037d1907
[Bugfix] Fix MRoPE Errors in the Qwen-VL Model When Processing Pure Text (#18526) 
Co-authored-by: 松灵 <wpf272043@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-22 05:22:53 -07:00
Cyrus Leung
ad0012a0ac
Revert "[Bugfix] Fix MRoPE Errors in the Qwen-VL Model When Processing Pure Text (#18407)" (#18456) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-20 22:39:22 -07:00
燃
be48360c1f
[Bugfix] Fix MRoPE Errors in the Qwen-VL Model When Processing Pure Text (#18407) 
Co-authored-by: 松灵 <wpf272043@alibaba-inc.com>
 2025-05-20 06:59:48 -07:00
Nan Qin
9609327fa4
[Core] [Bugfix]: tensor parallel with prompt embeds (#18171) 
Signed-off-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
 2025-05-19 20:21:27 -07:00
Tao He
60f7624334
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844) 2025-05-12 19:52:47 -07:00
Jee Jee Li
822de7fb94
[Misc] Split model loader (#17712) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-07 12:42:26 +08:00
Cyrus Leung
887d7af882
[Core] Gate prompt_embeds behind a feature flag (#17607) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-04 00:19:20 +08:00
Andrew Sansom
cc2a77d7f1
[Core] [Bugfix] Add Input Embeddings (#15428) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-02 01:06:39 -07:00
Cyrus Leung
aec9674dbe
[Core] Remove legacy input mapper/processor from V0 (#15686) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 15:38:48 +08:00
Harry Mellor
423e9f1cbe
Use Transformers helper get_text_config() instead of checking for text_config (#17105) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-25 08:47:35 -07:00
Woosuk Kwon
b411418ff0
[Chore] Remove Sampler from Model Code (#17084) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-24 02:49:33 -07:00
Yang Fan
2c1bd848a6
[Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) 
Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>
 2025-04-18 23:14:36 -07:00
Yihua Cheng
3408e47159
[P/D][V1] KV Connector API V1 (#15960) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-04-17 13:22:40 -07:00
Benjamin Kitor
82eb61dd4c
[misc] use tqdm.auto where appropriate (#16290) 
Signed-off-by: Benjamin Kitor <bkitor@gigaio.com>
 2025-04-09 21:54:54 -07:00
yihong
2de4118243
fix: change GB to GiB in logging close #14979 (#15807) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-03-31 10:00:50 -07:00
Varun Sundar Rabindranath
6c663dfd5e
[misc] LoRA - Skip LoRA kernels when not required (#15152) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-26 11:33:45 +08:00
Cyrus Leung
f6137adbcb
Revert "[Bugfix] Limit profiling run sequence length by max_model_len (#14785) (#14892) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-16 09:13:46 -07:00
Kyle Sayers
d30aa7e9e6
[Bugfix] Limit profiling run sequence length by max_model_len (#14785) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-03-16 07:44:19 -07:00
Jee Jee Li
b8b0ccbd2d
[Bugfix] Make the deviceprofiler include LoRA memory. (#14469) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-08 07:12:22 +00:00
Jun Duan
82fbeae92b
[Misc] Accurately capture the time of loading weights (#14063) 
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
 2025-03-01 17:20:30 -08:00
Benjamin Chislett
9804145cac
[Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
 2025-02-27 15:28:08 -08:00
Joe Runde
3f808cc044
[Bugfix] Do not crash V0 engine on input errors (#13101) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-02-26 19:07:29 +08:00
Harry Mellor
cdc1fa12eb
Remove unused kwargs from model definitions (#13555) 2025-02-24 17:13:52 -08:00
Jun Duan
68d535ef44
[Misc] Capture and log the time of loading weights (#13666) 2025-02-21 22:06:34 -08:00
Lucia Fang
f525c0be8b
[Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-02-19 17:06:23 +08:00
Zhe Zhang
fdc5df6f54
use device param in load_model method (#13037) 2025-02-19 16:05:02 +08:00
shangmingc
913df14da3
[Bugfix] Remove unused seq_group_metadata_list from ModelInputForGPU (#12935) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-02-08 14:46:19 +00:00
Roger Wang
bf3b79efb8
[VLM] Qwen2.5-VL 2025-02-05 13:31:38 -08:00
Cody Yu
cf58b9c4ca
[MISC] Remove model input dumping when exception (#12582) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-02-03 13:34:16 -08:00
Russell Bryant
e489ad7a21
[Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
fade_away
cb3e73e4c8
[BugFix] fix wrong output when using lora and num_scheduler_steps=8 (#11161) 
FIX issue https://github.com/vllm-project/vllm/issues/9688
https://github.com/vllm-project/vllm/issues/11086 #12487

---------

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: weilong.yu <weilong.yu@shopee.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-02-01 12:52:07 +08:00
Lucas Wilkinson
cabaf4eff3
[Attention] MLA decode optimizations (#12528) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
 2025-01-30 23:49:37 -08:00
youkaichao
6dd94dbe94
[perf] fix perf regression from #12253 (#12380) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-24 11:34:27 +08:00
Gregory Shtrasberg
e97f802b2d
[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2025-01-23 18:04:03 +00:00