xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-08 09:57:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,204 Commits 155 Branches 89 Tags
Commit Graph

312 Commits

Author SHA1 Message Date
Woosuk Kwon
4172235ab7
[V0 deprecation] Deprecate V0 Neuron backend (#21159) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-06 16:15:18 -07:00
yzds
ac201a0eaf
[Feature] Support Decode Context Parallel (DCP) for MLA (#23734) 
Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-09-06 13:24:05 +08:00
Didier Durand
35bf193864
[Doc]: fix typos in Python comments (#24294) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-05 19:41:12 -07:00
liuzhenwei
e599e2c65e
[XPU][P/D] Add XPU support in NixlConnector (#22436) 
Signed-off-by: zhenwei <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-04 21:03:12 -07:00
WeiQing Chen
6adaed42f4
[Feature][P/D]: Optimize NIXL Connector xfer Launch (#23887) 
Signed-off-by: ycyaw66 <497410282@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
 2025-09-03 19:14:30 +00:00
bnellnm
e9b92dcd89
[Kernels] Overlap shared experts with send/recv (#23273) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-03 12:35:18 -04:00
Nicolò Lucchesi
f0c503f66e
[Nixl] Heterogeneous TP support FlashInfer (#20189) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-03 15:19:54 +08:00
co63oc
1bd007f234
fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
Didier Durand
0235103cbb
[Doc]: fix typos in Python comments (#24042) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-01 19:07:45 -07:00
Didier Durand
107284959a
[Doc]: fix typos in Python comments (#24026) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-01 09:38:20 +00:00
Or Ozeri
14b4326b94
v1: Support KV events from connectors (#19737) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-09-01 01:13:21 +00:00
Ning Xie
68a349114f
[Misc] enhance type hint for rearrange return value (#23519) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-30 06:43:33 -07:00
Chaojun Zhang
235c9db8a7
[XPU] support data parallel for MoE models on XPU (#22887) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2025-08-29 09:23:04 +08:00
Yongye Zhu
082cc07ef8
DP/EP Support for gpt-oss with deepep-ht comm kernel on SM100 (#23608) 2025-08-27 17:33:21 -04:00
yzds
c7c80af084
fix pynccl reduce_scatter (#23648) 
Co-authored-by: hongchao <hongchao@msh.team>
 2025-08-26 18:21:11 -07:00
Zhonghua Deng
c3b0fd1ee6
[V1][P/D]P2pNcclConnector supports flashinfer (#23536) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-08-26 22:56:16 +00:00
Didier Durand
7c04779afa
[Doc]: fix various spelling issues in multiple files (#23636) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-26 14:05:29 +00:00
Zhonghua Deng
9188ae7cb5
[Bugfix][V1][P/D]Fix the issue where repeated requests for the same input produce abnormal outputs for P2pNcclConnector (#23403) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
 2025-08-25 12:57:08 -07:00
Ilya Markov
0313cf854d
[PERF] PyTorch Symmetric Memory All-Reduce (#20759) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-22 15:39:08 -06:00
Flora Feng
53415653ff
[P/D][Nixl] Make kv cache register compatible with hybrid memory allocator (#23079) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-08-21 22:30:48 -07:00
rongfu.leng
4fbda0b20c
[Feature] use --eplb_config to set eplb param (#20562) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-20 14:07:28 -07:00
Chengji Yao
e9d6a3db69
[TPU] make ptxla not imported when using tpu_commons (#23081) 
Signed-off-by: Chengji Yao <chengjiyao@gmail.com>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@gmail.com>
 2025-08-19 11:46:42 +08:00
bnellnm
8ad7285ea2
[Kernels] Clean up FusedMoeMethodBase and modular kernel setup. Remove extra arguments from modular kernel methods. (#22035) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-15 14:46:00 -04:00
Nick Hill
0933f9d518
[BugFix][KVConn] Fix use of get_required_kvcache_layout (#22734) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-15 01:39:43 +00:00
Ilya Markov
1d20c34717
[CI] Fix tests/distributed/test_ca_buffer_sharing.py (#22849) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-13 20:09:30 -07:00
Nicolò Lucchesi
422f22e012
[CI][Nixl] Check kv cache layout during handshake (#22745) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-08-12 12:53:52 -07:00
Nicolò Lucchesi
d030b01548
[BugFix][Nixl][PD] Fix heterogenous TP (#22663) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-08-12 05:37:30 -07:00
GuanLuo
16fb668b61
fix: NIXL connector transfers partial block to pass full multi-modal context (#21074) 
Signed-off-by: GuanLuo <gluo@nvidia.com>
 2025-08-11 09:40:55 -07:00
Cyrus Leung
d411df0296
[Misc] Further refine type annotations in parallel state (#22499) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-10 05:49:48 -07:00
Or Ozeri
7ad7adb67f
v1: Pass KVConnectorOutput to scheduler-side (#22157) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-08-08 23:09:51 -07:00
Cyrus Leung
43c4f3d77c
[Misc] Begin deprecation of get_tensor_model_*_group (#22494) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-08 01:11:54 -07:00
Shu Wang
b2c8ce57c6
Fix Flashinfer CUTLASS MOE Allgather (#21963) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
 2025-08-07 19:18:25 -07:00
WeiQing Chen
4be02a3776
[Bugfix] EPLB load statistics problem (#22167) 
Signed-off-by: ycyaw66 <497410282@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
 2025-08-07 04:07:54 +00:00
Ning Xie
74333ae2f6
[Misc] correct static type check for GroupCoordinator (#21946) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-05 03:17:46 -07:00
Ning Xie
bd3db7f469
[Misc] log more detailed message for ensure_model_parallel_initialized (#22144) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-04 19:36:55 -07:00
Ning Xie
29b97c0995
[Doc] add backend to doc string of initialize_model_parallel (#22142) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-04 19:36:20 -07:00
lkchen
f4f4e7ef27
[V0 deprecation][P/D] Deprecate v0 KVConnectorBase code (1/2) (#21785) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-08-04 19:11:33 -07:00
Ning Xie
c2e75b3c11
remove duplicate code within cleanup_dist_env_and_memory (#22147) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-03 20:03:58 -07:00
David Ben-David
aefeea0fde
[V1] [P/D] Refactor KV Connector Path (#21980) 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
 2025-08-03 04:03:40 -07:00
Ning Xie
7de45db9a5
[Misc] update doc comment for send (#22026) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-03 00:55:20 -07:00
Rui Qiao
d331759488
Introduce RayPPCommunicator for ray-based PP (#21660) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-08-01 11:50:58 -07:00
wxsm
f4135232b9
feat(distributed): add get_required_kvcache_layout class method to kv connector api (#20433) 
Signed-off-by: wxsm <wxsms@foxmail.com>
 2025-07-30 16:41:51 +00:00
Chenguang Zheng
4904e53c32
[Bugfix] SharedStorage Connector for V1 PD multimodal (#21611) 
Signed-off-by: fake0fan <645327136@qq.com>
Signed-off-by: herotai214 <herotai214@gmail.com>
Co-authored-by: herotai214 <herotai214@gmail.com>
 2025-07-30 09:18:37 -07:00
Calvin Chen
e18f085103
skip fusedmoe layer for start_load_kv (#21378) 
Signed-off-by: calvin chen <wen.chen@dynamia.ai>
 2025-07-28 18:59:44 -07:00
Kuntai Du
b18b417fbf
Revert "[V1] Exception Handling when Loading KV Cache from Remote Store" (#21778) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-07-28 20:15:18 +00:00
Nick Hill
7d44c691b0
[P/D] Log warnings related to prefill KV expiry (#21753) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-28 18:40:53 +00:00
Adeline
15a72ac478
[V1] Exception Handling when Loading KV Cache from Remote Store (#21534) 
Signed-off-by: liuyumoye <adeline_ly2023@outlook.com>
Co-authored-by: liuyumoye <adeline_ly2023@outlook.com>
 2025-07-27 20:34:17 -07:00
WeiQing Chen
97d6c30cc9
[BugFix] Fix shared storage connector load kv only load attention layer (#21428) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-07-26 07:07:40 -07:00
Juncheng Gu
6066284914
[P/D] Support CPU Transfer in NixlConnector (#18293) 
Signed-off-by: Juncheng Gu <juncgu@gmail.com>
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Richard Liu <39319471+richardsliu@users.noreply.github.com>
Co-authored-by: Richard Liu <ricliu@google.com>
 2025-07-24 17:58:42 +01:00
Rui Qiao
1e9ea8e69d
[P/D] Move FakeNixlWrapper to test dir (#21328) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-24 08:53:45 -07:00