xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-20 10:47:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,970 Commits 155 Branches 89 Tags
Commit Graph

39 Commits

Author SHA1 Message Date
Nick Hill
5175b884f7
[BugFix] Remove default multiproc executor collective_rpc timeout (#17000) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-22 23:27:14 +00:00
Robert Shaw
2b05b8ce69
[V1][Frontend] Improve Shutdown And Logs (#11737) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-04-16 19:48:34 -07:00
yihong
04149cce27
[BugFix] fix some typos found by typos. (#16314) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-09 03:43:59 -07:00
youkaichao
a865bc1ca6
[core] do not send error across process (#16174) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-04-07 19:09:03 -07:00
Nick Hill
15dac210f0
[V1] AsyncLLM data parallel (#13923) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-27 16:14:41 -07:00
Chen Zhang
93a00d7dde
[v1] Refactor KVCacheConfig (#14079) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-03-21 04:56:27 -07:00
Mickaël Seznec
a597a57595
[Attention] Flash Attention 3 - fp8 (#14570) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
 2025-03-20 01:14:20 -04:00
Harry Mellor
cf069aa8aa
Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Li, Jiang
02296f420d
[Bugfix][V1][Minor] Fix shutting_down flag checking in V1 MultiprocExecutor (#14053) 2025-02-28 22:31:01 -08:00
youkaichao
eb24dc4a45
[v1] torchrun compatibility (#13642) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-23 22:47:24 +08:00
youkaichao
3e472d882a
[core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Woosuk Kwon
4fb8142a0e
[V1][PP] Enable true PP with Ray executor (#13472) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-18 09:15:32 -08:00
Cody Yu
9206b3d7ec
[V1][PP] Run engine busy loop with batch queue (#13064) 2025-02-15 03:59:01 -08:00
Rui Qiao
9605c1256e
[V1][core] Implement pipeline parallel on Ray (#12996) 2025-02-13 08:02:46 +00:00
youkaichao
91dd8f7aa6
[bugfix] respect distributed_executor_backend in world_size=1 (#12934) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-08 16:17:08 +08:00
Russell Bryant
e489ad7a21
[Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Cyrus Leung
59a0192fb9
[Core] Interface for accessing model from VllmRunner (#10353) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-20 15:00:59 +08:00
youkaichao
6d0e3d3724
[core] clean up executor class hierarchy between v1 and v0 (#12171) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-18 14:35:15 +08:00
youkaichao
87a0c076af
[core] allow callable in collective_rpc (#12151) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-17 20:47:01 +08:00
Chen Zhang
69d765f5a5
[V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-01-17 07:39:35 +00:00
youkaichao
bf53e0c70b
Support torchrun and SPMD-style offline inference (#12071) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-16 19:58:53 +08:00
Keyun Tong
ad388d25a8
Type-fix: make execute_model output type optional (#12020) 2025-01-15 09:44:56 +00:00
youkaichao
ad34c0df0f
[core] platform agnostic executor via collective_rpc (#11256) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-15 13:45:21 +08:00
youkaichao
89ce62a316
[platform] add ray_device_key (#11948) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-13 16:20:52 +08:00
Rui Qiao
022c5c6944
[V1] Refactor get_executor_cls (#11754) 2025-01-06 07:59:16 +00:00
WangErXiao
e5d7ed0c53
[V1] log GPU blocks num for MultiprocExecutor (#11656) 2025-01-04 00:13:12 +00:00
Robert Shaw
1543914c04
[V1] Improve TP>1 Error Handling + Stack Trace (#11721) 
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-01-03 21:29:11 +00:00
Robert Shaw
df04dffade
[V1] [4/N] API Server: ZMQ/MP Utilities (#11541) 2024-12-28 01:45:08 +00:00
Rui Qiao
9832e5572a
[V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) 2024-12-24 19:49:46 -08:00
Rui Qiao
a491d6f535
[V1] TP Ray executor (#11107) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2024-12-23 23:00:12 +00:00
Tyler Michael Smith
2bfdbf2a36
[V1][Core] Use weakref.finalize instead of atexit (#11242) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-16 22:11:33 -08:00
Mark McLoughlin
6d917d0eeb
Enable mypy checking on V1 code (#11105) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2024-12-14 09:54:04 -08:00
Russell Bryant
4863e5fba5
[Core] V1: Use multiprocessing by default (#11074) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-12-13 16:27:32 -08:00
Russell Bryant
ccede2b264
[Core] cleanup zmq ipc sockets on exit (#11115) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-12-11 19:12:24 -08:00
Tyler Michael Smith
28b3a1c7e5
[V1] Multiprocessing Tensor Parallel Support for v1 (#9856) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-10 06:28:14 +00:00
Robert Shaw
91c9ebbb1b
[V1] Fix Configs (#9971) 2024-11-04 00:24:40 +00:00
Robert Shaw
d6459b4516
[V1] Fix EngineArgs refactor on V1 (#9954) 2024-11-02 07:44:38 -07:00
youkaichao
e893795443
[2/N] executor pass the complete config to worker/modelrunner (#9938) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2024-11-02 07:35:05 -07:00
Woosuk Kwon
6c5af09b39
[V1] Implement vLLM V1 [1/N] (#9289) 2024-10-22 01:24:07 -07:00