xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-07 01:47:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,064 Commits 155 Branches 89 Tags
Commit Graph

473 Commits

Author SHA1 Message Date
Sage Moore
642bf2dd8b Merge branch 'main' of https://github.com/neuralmagic/vllm into lwilkinson/attn-slicing 2025-06-08 18:02:06 +00:00
Reid
122cdca5f6
[Misc] refactor context extension (#19246) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-07 05:13:21 +00:00
Sage Moore
f8848bb201 misc fixes. lm_eval still gets a wrong answer but it no longer hangs 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-06-04 22:46:18 +00:00
jmswen
c8dcc15921
Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) 
Signed-off-by: Jon Swenson <jmswen@gmail.com>
 2025-06-04 08:26:47 -07:00
Xu Wenqing
02658c2dfe
Add DeepSeek-R1-0528 function call chat template (#18874) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-06-04 13:24:18 +00:00
汪志鹏
3336c8cfbe
Fix #19130 (#19132) 
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-06-04 01:42:06 -07:00
Calvin Chen
8d646c2e53
[Cleanup][v1]:remote guided-decoding-backend for example (#19059) 
Signed-off-by: calvin chen <120380290@qq.com>
 2025-06-04 04:23:26 +00:00
Jiaxin Shan
abd7df2fca
[Misc] Fix path and python alias errors in disagg_prefill exmaples (#18919) 2025-06-03 17:15:18 -07:00
Sage Moore
2e3484c237 debugging 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-06-03 19:25:01 +00:00
Simon Mo
02f0c7b220
[Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
汪志鹏
1282bd812e
Add tarsier model support (#18985) 
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-06-03 13:13:13 +08:00
Sage Moore
18e7d6c7b8 Merge branch 'main' of https://github.com/neuralmagic/vllm into lwilkinson/attn-slicing 2025-06-03 00:52:39 +00:00
Siyuan Liu
9112b443a0
[Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-06-03 00:06:20 +00:00
Calvin Chen
c57d577e8d
add an absolute path for run.sh (#18258) 
Signed-off-by: calvin chen <120380290@qq.com>
 2025-06-02 19:38:23 +00:00
Sage Moore
8332924320 dp format 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-06-02 19:15:23 +00:00
Sage Moore
8ea80fca4a revert offline_inference/basic.py 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-06-02 18:05:48 +00:00
Sage Moore
21d9529a79 revert offline_inference/basic.py 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-06-02 18:05:26 +00:00
Nick Hill
9a1b9b99d7
[BugFix] Fix multi-node offline data-parallel (#18981) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Yizhou Liu <liu_yizhou@outlook.com>
 2025-05-31 08:34:52 -07:00
Satyajith Chilappagari
2a50ef5760
[Neuron] Add Multi-Modal model support for Neuron (#18921) 
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Ashraf Mahgoub <ashymahg@amazon.com>
Co-authored-by: Rohith Nallamaddi <nalrohit@amazon.com>
Co-authored-by: FeliciaLuo <luof@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
 2025-05-31 10:39:11 +00:00
Sage Moore
62da375465 more fixes 2025-05-30 21:17:06 +00:00
Mark McLoughlin
0e98964e94
[V1][Metrics] Remove metrics that were deprecated in 0.8 (#18837) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-28 18:54:12 +00:00
Reid
435fa95444
[Frontend] add run batch to CLI (#18804) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-28 07:08:57 -07:00
wang.yuqi
3e9ce609bd
[Bugfix] Fix nomic max_model_len (#18755) 2025-05-27 20:29:53 -07:00
Mark McLoughlin
06a0338015
[V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-27 09:37:06 +00:00
Reid
fc6d0c290f
[Misc] improve docs (#18734) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-27 07:07:01 +00:00
Cyrus Leung
753944fa9b
[Doc] Update reproducibility doc and example (#18741) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-27 07:03:13 +00:00
Harry Mellor
27bebcd897
Convert examples to ruff-format (#18400) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-26 16:57:54 +00:00
Cyrus Leung
82e2339b06
[Doc] Move examples and further reorganize user guide (#18666) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-26 07:38:04 -07:00
AlexZhao
8820821b59
[Misc] Fixed the abnormally high TTFT issue in the PD disaggregation example (#18644) 
Signed-off-by: zhaohaidao <zhaohaidao2008@hotmail.com>
Signed-off-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>
Co-authored-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>
 2025-05-26 13:51:27 +08:00
Isotr0py
75f81750f3
[VLM] Initialize video input support for InternVL models (#18499) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-05-25 04:51:25 +00:00
Feng XiaoLong
4fc1bf813a
[Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) 
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
 2025-05-23 16:16:26 -07:00
Lucas Wilkinson
18bf91e6a8 wip 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-23 03:31:49 +00:00
Chenheli Hua
04eb88dc80
Re-submit: Fix: Proper RGBA -> RGB conversion for PIL images. (#18569) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-05-23 01:59:18 +00:00
Sanger Steel
c32e249a23
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
 2025-05-22 18:44:18 -07:00
Kai Wu
c91fe7b1b9
[Frontend][Bug Fix] Update llama4 pythonic jinja template and llama4_pythonic parser (#17917) 
Signed-off-by: Kai Wu <kaiwu@meta.com>
 2025-05-22 16:44:08 -07:00
Lucas Wilkinson
9c60a6299d tp1 working multistream tp > 1 broken 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-22 20:51:36 +00:00
Lucas Wilkinson
04f11d97a0 working but only on the same stream 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-22 20:51:36 +00:00
Lucas Wilkinson
ffb740ae95 manually manage stream 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-22 20:51:36 +00:00
Sage Moore
020269c4c5 added multhreading support 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-05-22 20:51:36 +00:00
Lucas Wilkinson
9ccfd094ff fix dummy mode 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-22 20:51:35 +00:00
Lucas Wilkinson
f93bdd3151 support more args in dp example 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-22 20:51:35 +00:00
Lucas Wilkinson
df8f889f37 support MLA 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-22 20:51:35 +00:00
Lucas Wilkinson
37c9babaa0 enable naive microbatching 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-05-22 20:51:35 +00:00
Reid
cb506ecb5a
[Misc] improve Automatic Prefix Caching example (#18554) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-22 14:50:46 +00:00
Calvin Chen
3f505233fd
[Doc] Add stream flag for chat completion example (#18524) 
Signed-off-by: calvin chen <120380290@qq.com>
 2025-05-22 14:07:10 +00:00
CYJiang
71075029f2
[Doc] Support --stream arg in openai_completion_client.py script (#18388) 
Signed-off-by: googs1025 <googs1025@gmail.com>
 2025-05-22 13:20:17 +00:00
Reid
107f5fc4cb
[Misc] refactor disaggregated-prefill-v1 example (#18474) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-21 11:10:14 +00:00
Reid
8f55962a7f
[Misc] refactor prompt embedding examples (#18405) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-20 15:26:12 +00:00
Gong Shufan
8171221834
[Misc] Fix typo (#18330) 2025-05-19 09:51:01 -07:00
Reid
27d0952600
[Misc] extract parser.parse_args() (#18323) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-19 04:06:26 +00:00