xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-01-07 05:49:38 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,835 Commits 155 Branches 89 Tags
Commit Graph

161 Commits

Author SHA1 Message Date
shangmingc
1dd23386ec
[Misc] Update usage with mooncake lib for kv transfer (#16523) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-04-14 11:31:37 +00:00
Huazhong Ji
68bb122eb4
[MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) 
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
 2025-04-12 09:20:25 +00:00
Kebe
b4ac449a83
[Misc] Merge the logs of pp layers partitions (#16225) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-04-08 00:18:15 -07:00
Chengji Yao
01b6113659
[TPU] optimize the all-reduce performance (#15903) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-04-03 00:25:14 +00:00
Li, Jiang
550b2801ad
[CPU][Bugfix] Using custom allreduce for CPU backend (#15934) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-04-02 07:46:47 -07:00
Ilya Markov
b7b7676d67
[Distributed] Add custom allreduce support for ROCM (#14125) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-03-31 22:49:12 -07:00
shangmingc
6fa7cd3dbc
[Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore (#12957) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-03-29 04:01:46 -07:00
Nick Hill
6d531ad7b8
[Misc][V1] Misc code streamlining (#15723) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-28 20:59:47 -07:00
Kebe
4e0f6076be
[Bugfix] Fix failure to launch in Tensor Parallel TP mode on macOS. (#14948) 
Signed-off-by: Kebe <mail@kebe7jun.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-03-28 10:13:41 +08:00
Robert Shaw
bd45912b99
[TPU] Lazy Import (#15656) 
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
 2025-03-28 09:57:01 +08:00
Nick Hill
15dac210f0
[V1] AsyncLLM data parallel (#13923) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-27 16:14:41 -07:00
Yi Liu
9cc645141d
[MISC] Refine no available block debug msg (#15076) 
Signed-off-by: Yi Liu <yiliu4@habana.ai>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Yi Liu <yiliu4@habana.ai>
 2025-03-25 00:01:10 +08:00
youkaichao
9606d572ed
[distributed] fix dp group (#15355) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-24 14:54:27 +00:00
Russell Bryant
038de04d7b
Fix zmq IPv6 URL format error (#15341) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-24 09:30:41 -04:00
billishyahao
742369d35a
[Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) 
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
 2025-03-20 00:00:33 -07:00
Alexander Matveev
cfbca8a2f2
[V1] TPU - Tensor parallel MP support (#15059) 2025-03-20 00:55:18 +00:00
hoshi-hiyouga
414919138b
[Bugfix] torchrun compatibility (#14899) 
Signed-off-by: hiyouga <hiyouga@buaa.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-03-18 05:49:27 -07:00
Nick Hill
b82662d952
[BugFix] Fix torch distributed stateless PG backend init (#14870) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-15 20:26:19 -07:00
yasu52
3fb17d26c8
[Doc] Fix typo in documentation (#14783) 
Signed-off-by: yasu52 <tsuguro4649@gmail.com>
 2025-03-13 20:33:09 -07:00
Mathis Felardos
1bd32bc8dd
[Config][Disaggregated] Add timeout configuration for the torch.store and add KVTransferConfig.kv_connector_extra_config (#14367) 
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
 2025-03-12 20:15:20 -07:00
gnovack
d6123170d5
[Neuron] Add Neuron device communicator for vLLM v1 (#14085) 2025-03-10 18:37:04 -07:00
Jiayi Yao
6d7f037748
[Feat] Support chunked prefill for LMCache connector (#14505) 
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
 2025-03-08 19:30:06 -08:00
Mathis Felardos
980385f8c1
[Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and fix the reshape of the KVCache (#14369) 
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
 2025-03-07 22:39:31 -08:00
Kuntai Du
288ca110f6
[Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-03-04 21:10:32 +00:00
Mathis Felardos
b9e41734c5
[Bugfix][Disaggregated] patch the inflight batching on the decode node in SimpleConnector to avoid hangs in SimpleBuffer (nccl based) (#13987) 
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
 2025-02-28 07:53:45 +00:00
Harry Mellor
145944cb94
Improve pipeline partitioning (#13839) 2025-02-25 18:53:56 -08:00
Jiayi Yao
2f42a4888c
[Feature] Support KV cache offloading and disagg prefill with LMCache connector. (#12953) 2025-02-25 00:38:42 -08:00
cjackal
51010a1807
[Misc] set single whitespace between log sentences (#13771) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-02-25 10:26:12 +08:00
Nick Hill
5a2ba16f5c
[Core][Distributed] Use IPC (domain socket) ZMQ socket for local comms (#13688) 2025-02-23 02:54:29 -08:00
youkaichao
3e472d882a
[core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Isotr0py
b2c3fc5d65
[Bugfix][CPU] Fix cpu all-reduce using native pytorch implementation (#13586) 2025-02-20 22:24:17 -08:00
Yan Ma
30513d1cb6
[Bugfix] fix xpu communicator (#13368) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2025-02-17 20:59:18 +08:00
youkaichao
a0231b7c25
[platform] add base class for communicators (#13208) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-16 22:14:22 +08:00
Lu Fang
009439caeb
Simplify logic of locating CUDART so file path (#13203) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-02-13 13:52:41 +08:00
Lu Fang
042c3419fa
Introduce VLLM_CUDART_SO_PATH to allow users specify the .so path (#12998) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-02-12 09:06:13 -08:00
Cyrus Leung
8a69e0e20e
[CI/Build] Auto-fix Markdown files (#12941) 2025-02-08 04:25:15 -08:00
Lu Fang
45cbc4991d
[Bugfix] Fix disagg hang caused by the prefill and decode communication issues (#12723) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-02-07 16:39:50 -08:00
ZSL98
433c4a4923
Make vllm compatible with verl (#12824) 
Co-authored-by: zhangshulai <zhangshulai@bytedance.com>
 2025-02-07 11:54:20 +08:00
Akash kaothalkar
022bcc701a
[Bugfix] Fix 'ModuleNotFoundError: No module named 'intel_extension_for_pytorch'' for --tensor-parallel-size more than 1 (#12546) 2025-02-04 23:11:02 -08:00
Russell Bryant
e489ad7a21
[Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Harry Mellor
823ab79633
Update pre-commit hooks (#12475) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-27 17:23:08 -07:00
Jani Monoses
9c485d9e25
[Core] Free CPU pinned memory on environment cleanup (#10477) 2025-01-21 11:56:41 -08:00
shangmingc
df450aa567
[Bugfix] Fix num_heads value for simple connector when tp enabled (#12074) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-01-20 02:56:43 +00:00
youkaichao
ad34c0df0f
[core] platform agnostic executor via collective_rpc (#11256) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-15 13:45:21 +08:00
youkaichao
310aca88c9
[perf]fix current stream (#11870) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-09 07:18:21 +00:00
Harry Mellor
aba8d6ee00
[Doc] Move examples into categories (#11840) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-08 13:09:53 +00:00
XiaobingZhang
e512f76a89
fix init error for MessageQueue when n_local_reader is zero (#11768) 2025-01-07 06:12:48 +00:00
cennn
9e764e7b10
[distributed] remove pynccl's redundant change_state (#11749) 2025-01-06 09:05:48 +08:00
cennn
635b897246
[distributed] remove pynccl's redundant stream (#11744) 2025-01-05 23:09:11 +08:00
Yan Burman
300acb8347
[Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) 
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
 2025-01-04 14:50:16 +08:00