xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-22 04:15:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,757 Commits 155 Branches 89 Tags
Commit Graph

104 Commits

Author SHA1 Message Date
Matthew Bonanni
fc1d8be3dc
[Attention] Update attention imports (#29540) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-27 11:19:09 -05:00
rasmith
71362ffab4
[CI/Build][AMD] Skip test_multi_shared_storage_connector_consistency in test_multi_connector.py due to hipErrorLaunchFailure when calling .cpu() (#29253) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-11-23 04:42:49 +00:00
rasmith
a4fdf2405c
[CI/Build] Skip tests that require libcudart in test_lmcache_integration.py (#29228) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-11-22 10:59:39 +00:00
Jialin Ouyang
30b9c67743
Revert "[Redo] #26368 (#28771)" (#29121) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-11-20 21:27:45 -08:00
Or Ozeri
647464719b
[KVConnector][Core] Support cross-layer KV blocks (#27743) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-11-20 19:09:59 +01:00
Chendi.Xue
c3e2978620
[NIXL] fix cpu PD after physical <> logical block_size PR (#28904) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-11-18 14:03:23 -05:00
Nicolò Lucchesi
f226a3f0c1
[CI][NIXL] Change default block_size for tests (#28927) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-11-18 09:22:30 -08:00
Cyrus Leung
98b4d389ed
[Redo] #26368 (#28771) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-11-14 22:47:41 -08:00
Chendi.Xue
c9e665852a
[NIXL] heterogeneous block_size support (#26759) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2025-11-14 21:51:32 -08:00
Nick Hill
ac86bff8cb
Revert "[Core] Performance: Use list[np.ndarray] instead of list[list… (#28773) 2025-11-14 20:24:00 -08:00
Jialin Ouyang
186352b270
[Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output tokens for GC optimization (#26368) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-11-14 16:04:04 -08:00
Mark McLoughlin
6e25b1cddf
[KV Connector] Test async mode in scheduler tests (#28550) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-11-13 18:30:59 -05:00
Nicolò Lucchesi
a7ef3eb0cd
[NIXL] Generalize block-first backend layouts (FlashInfer-like) (#28282) 2025-11-11 16:57:43 +00:00
Nick Hill
289eb6c537
[Core] Simplify async KV output aggregation (#28327) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-09 09:44:13 -08:00
Nick Hill
67a2da890e
[PerfFix] Avoid separate thread for MP executor shm spin (take 2) (#28319) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-07 22:11:03 +00:00
Nicolò Lucchesi
68a72a5cc1
Revert "[PerfFix] Avoid separate thread for MP executor shm spin (#28012)" (#28289) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-11-07 15:07:01 +00:00
Samuel Shen
40db194446
[CI]: Add LMCacheConnector Unit Tests (#27852) 
Signed-off-by: Samuel Shen <slshen@uchciago.edu>
Co-authored-by: Samuel Shen <slshen@uchciago.edu>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
 2025-11-05 09:45:57 -08:00
Kuntai Du
86dca07d9b
[Hybrid allocator + kv connector] revert connector test changes related to hybrid allocator (#28011) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-11-05 10:36:31 +00:00
Nick Hill
c9f66da8fd
[PerfFix] Avoid separate thread for MP executor shm spin (#28012) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-04 08:33:55 -08:00
Mark McLoughlin
58279c60b5
[KV Connector] Make KVCacheConfig an explicit constructor argument (#27887) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-11-03 23:00:49 -08:00
Yihua Cheng
e675118849
[Add] cmdline argument parsing for KV cache offloading modules (#27621) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-01 07:17:07 +00:00
Nick Hill
0cdbe7b744
[Core] Async scheduling + structured outputs compatibility (#26866) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-01 00:35:04 +00:00
GuanLuo
d6517be3cd
[Bugfix] Missing NIXL metadata for handshake initialization if instance spans multi-node (#26338) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-10-31 10:16:00 -07:00
Kuntai Du
b853540388
[Core][Hybrid allocator + kv connector 1/n] Enable hybrid allocator + KV cache connector (#25712) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
 2025-10-24 23:34:18 -07:00
kourosh hakhamaneshi
7e1d697b56
[Bugfix] Fix MultiConnector stats reconstruction across process boundaries (#27366) 
Signed-off-by: Kourosh Hakhamaneshi <Kourosh@anyscale.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-10-24 17:08:05 +00:00
Nicolò Lucchesi
4dfdb821c8
[P/D] Dynamic kv_output_aggregator collect size (#26734) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-22 18:07:58 +02:00
Mark McLoughlin
4ca13a8667
[NIXL] Terminate handshake listener thread in shutdown (#26404) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-22 16:59:53 +02:00
Nicolò Lucchesi
bfa59be8f1
[CI] Nixl integration tests DP-EP (#27199) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-22 11:17:48 +08:00
Tyler Michael Smith
6c2eef5a5d
[P/D] KVConnector for decode benchmarking (#25986) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-10-21 16:30:47 -07:00
Nicolò Lucchesi
72f431e709
[Nixl] Minor refactor to handshake related metadata (#26410) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-21 09:07:47 +02:00
dongbo910220
8a297115e2
[Chore] Separate out hashing utilities from vllm.utils (#27151) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-10-19 11:09:38 +08:00
Nicolò Lucchesi
2ba60ec7fe
[CI] Nixl integration tests (#27010) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-17 07:13:31 -07:00
Nick Hill
4aed506b65
[Core] Streamline some structured output related code (#26737) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-14 23:27:44 +00:00
Qier Li
720394de43
[KVConnector][Metrics] Aggregate scheduler-side KVConnectorStats (#26046) 
Signed-off-by: Qier Li <kevin44036@gmail.com>
 2025-10-14 14:38:07 +00:00
Chendi.Xue
7e6edb1469
[NIXL][HeteroTP] Enable KV transfer from HND prefill to NHD decode (#26556) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-10-14 09:46:05 +00:00
Will Eaton
53c9a7cee2
[P/D] [NixlConnector] kv load recovery integration (#26171) 
Signed-off-by: Will Eaton <weaton@redhat.com>
 2025-10-13 08:48:04 -07:00
Harry Mellor
8fcaaf6a16
Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Chendi.Xue
9bb38130cb
[Bugfix] Fix GPU_ID issue in test script (#26442) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-10-12 11:39:05 +00:00
Mark McLoughlin
784c231151
[NIXL] Ignore abort on already-finished request (#25067) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-10 12:21:56 +02:00
Chen Zhang
606b00e80f
[bugfix][DCP] fix block_size of hash in DCP prefix caching (#26296) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-10 03:02:49 -07:00
Cyrus Leung
1e4ecca1d0
[V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:42:31 +00:00
Harry Mellor
6c04638214
Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Nicolò Lucchesi
2a6dc67eb5
[Bugfix] Fix _reqs_to_process leak on abort (#26012) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-04 11:39:31 +00:00
Nicolò Lucchesi
48f309029a
[NIXL][Misc] Expose metrics from NIXL for logging to CLI (#25388) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-03 10:47:59 +00:00
David Ben-David
9a9f48dff7
[V1] [P/D] Add Support for KV Load Failure Recovery (#19330) 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
 2025-09-30 14:57:08 -07:00
Reza Barazesh
bc546f76a1
[CI] Move applicable tests to CPU (#24080) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-30 14:45:20 +01:00
Nicolò Lucchesi
80608ba5af
[NIXL] Add support for MLA caches with different latent dim (#25902) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-30 12:18:29 +00:00
Chenxi Yang
d0d138bc55
[Nixl][P/D] Add cuda2cpu support (HD->DH transfer) (#24690) 
Signed-off-by: Chenxi Yang <cxyang@fb.com>
Co-authored-by: Chenxi Yang <cxyang@fb.com>
 2025-09-29 14:31:51 +00:00
Chauncey
2827b3f4a3
[CI] Fix test_shared_storage_connector_hashes (#25748) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-26 20:46:17 +08:00