xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-08 10:53:39 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,610 Commits 155 Branches 89 Tags
Commit Graph

62 Commits

Author SHA1 Message Date
Tyler Michael Smith
ab153be252 take 2 
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-07-11 14:42:44 +00:00
Or Ozeri
cc876d0f29
[KVConnector] Aggregate finished requests on the scheduler (#19555) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-07-10 09:22:18 +01:00
Nick Hill
59389c927b
[BugFix][CPU] Fix CPU worker dependency on cumem_allocator (#20696) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-10 14:24:20 +08:00
Kunshang Ji
0b407479ef
[misc]refactor Platform.set_device method (#20262) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-09 01:39:47 +00:00
Yang Yang
6e2c19ce22
[Refactor]Abstract Platform Interface for Distributed Backend and Add xccl Support for Intel XPU (#19410) 
Signed-off-by: dbyoung18 <yang5.yang@intel.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-07 04:32:32 +00:00
Bowen Wang
e9fd658a73
[Feature] Expert Parallelism Load Balancer (EPLB) (#18343) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
 2025-06-26 15:30:21 -07:00
Maximilien de Bayser
799397ee4f
Support embedding models in V1 (#16188) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-18 21:36:33 -07:00
Isotr0py
1173804dca
[Bugfix] Fix TP inference for Flex attention backend (#19657) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-16 11:21:37 +00:00
Ye (Charlotte) Qi
cc867be19c
[V1] Reuse V0's memory_profiling util for gpu worker memory profiling (#19312) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-06-10 08:40:01 +08:00
Luka Govedič
2d8476e465
[BugFix][V1] Fix memory profiling bug (#18974) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-07 10:34:51 -07:00
Li, Jiang
4555143ea7
[CPU] V1 support for the CPU backend (#16441) 2025-06-03 18:43:01 -07:00
Simon Mo
02f0c7b220
[Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Divakar Verma
774c5fde30
[V1] fix torch profiling for V1 offline scenarios (#18445) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-05-28 04:16:30 +00:00
youkaichao
6a7988c55b
Refactor pplx init logic to make it modular (prepare for deepep) (#18200) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-05-23 23:43:43 +08:00
Harry Mellor
a1fe24d961
Migrate docs from Sphinx to MkDocs (#18145) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 02:09:53 -07:00
Sanger Steel
c32e249a23
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
 2025-05-22 18:44:18 -07:00
Lucia Fang
3d2779c29a
[Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) 
Signed-off-by: Lucia Fang <fanglu@fb.com>
 2025-05-15 22:28:27 -07:00
bnellnm
f9c069c85e
Modularize fused experts and integrate PPLX kernels (#15956) 2025-05-14 13:11:54 -07:00
Jee Jee Li
822de7fb94
[Misc] Split model loader (#17712) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-07 12:42:26 +08:00
Li, Jiang
a6fed02068
[V1][PP] Support PP for MultiprocExecutor (#14219) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: jiang.li <jiang1.li@intel.com>
 2025-05-06 07:58:05 -07:00
Harry Mellor
d6484ef3c3
Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
Daniel Li
48cb2109b6
[V1] Move usage stats to worker and start logging TPU hardware (#16211) 2025-04-25 14:06:01 -06:00
Han Zhang
d41faaf9df
Restore buffers when wake up from level 2 sleep (#16564) (#16889) 
Signed-off-by: Han <zh950713@gmail.com>
 2025-04-21 20:18:28 +08:00
Yihua Cheng
3408e47159
[P/D][V1] KV Connector API V1 (#15960) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-04-17 13:22:40 -07:00
wwl2755
463bbb1835
[Bugfix][V1] Fix bug from putting llm_engine.model_executor in a background process (#15367) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-04-03 07:32:10 +00:00
Eric Tang
ddb94c2605
[core] Add tags parameter to wake_up() (#15500) 
Signed-off-by: Eric <erictang000@gmail.com>
 2025-04-02 01:59:27 -07:00
Chen Zhang
93a00d7dde
[v1] Refactor KVCacheConfig (#14079) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-03-21 04:56:27 -07:00
Woosuk Kwon
0c6f5023c3
[V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-03-20 17:50:43 -07:00
Roger Wang
ad19c8a003
[V1] Move OOM check into sampler run (#14728) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-03-13 20:40:23 -07:00
Cody Yu
b706d898af
[Bugfix][V1][PP] Only warmup sampler at last PP rank (#14643) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-03-11 23:40:07 +00:00
Roger Wang
1fc973c0b5
[V1][Core] Fix memory issue with logits & sampling (#14508) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Varun Sundar Rabindranath <3337719+varun-sundar-rabindranath@users.noreply.github.com>
 2025-03-11 04:03:41 +00:00
Robert Shaw
5f0b53c6ea
Revert "[V1][Core] Fix memory issue with logits & sampling" (#14504) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-03-08 17:43:37 -08:00
Roger Wang
8d5aa466fb
[V1][Core] Fix memory issue with logits & sampling (#13776) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-08 06:11:04 -08:00
Harry Mellor
cf069aa8aa
Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Varun Sundar Rabindranath
03f48b3db6
[Core] LoRA V1 - Add add/pin/list/remove_lora functions (#13705) 2025-02-25 00:18:02 -08:00
cjackal
51010a1807
[Misc] set single whitespace between log sentences (#13771) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-02-25 10:26:12 +08:00
Roger Wang
227578480d
Revert "[V1][Core] Fix memory issue with logits & sampling" (#13775) 2025-02-24 09:16:05 -08:00
Roger Wang
437b76ff59
[V1][Core] Fix memory issue with logits & sampling (#13721) 2025-02-24 06:10:06 -08:00
youkaichao
eb24dc4a45
[v1] torchrun compatibility (#13642) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-23 22:47:24 +08:00
youkaichao
3e472d882a
[core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Varun Sundar Rabindranath
cbc40128eb
[V1] LoRA - Enable Serving Usecase (#12883) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-02-14 14:21:12 +08:00
Aoyu
2092a6fa7d
[V1][Core] Add worker_base for v1 worker (#12816) 
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-02-13 20:35:18 +08:00
Rui Qiao
9605c1256e
[V1][core] Implement pipeline parallel on Ray (#12996) 2025-02-13 08:02:46 +00:00
Woosuk Kwon
2ff4857678
[V1][Minor] Move scheduler outputs to a separate file (#13062) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-11 02:10:06 +00:00
Russell Bryant
e489ad7a21
[Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
youkaichao
6e650f56a1
[torch.compile] decouple compile sizes and cudagraph sizes (#12243) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-24 02:01:30 +08:00
youkaichao
68ad4e3a8d
[Core] Support fully transparent sleep mode (#11743) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-22 14:39:32 +08:00
Chen Zhang
5f0ec3935a
[V1] Remove _get_cache_block_size (#12214) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-01-20 21:54:16 +08:00
Cyrus Leung
59a0192fb9
[Core] Interface for accessing model from VllmRunner (#10353) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-20 15:00:59 +08:00
Chen Zhang
69d765f5a5
[V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-01-17 07:39:35 +00:00