xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 14:25:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,212 Commits 155 Branches 89 Tags
Commit Graph

64 Commits

Author SHA1 Message Date
Cyrus Leung
1e4ecca1d0
[V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:42:31 +00:00
Harry Mellor
6c04638214
Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Nicolò Lucchesi
2a6dc67eb5
[Bugfix] Fix _reqs_to_process leak on abort (#26012) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-04 11:39:31 +00:00
Nicolò Lucchesi
48f309029a
[NIXL][Misc] Expose metrics from NIXL for logging to CLI (#25388) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-03 10:47:59 +00:00
David Ben-David
9a9f48dff7
[V1] [P/D] Add Support for KV Load Failure Recovery (#19330) 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
 2025-09-30 14:57:08 -07:00
Reza Barazesh
bc546f76a1
[CI] Move applicable tests to CPU (#24080) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-30 14:45:20 +01:00
Nicolò Lucchesi
80608ba5af
[NIXL] Add support for MLA caches with different latent dim (#25902) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-30 12:18:29 +00:00
Chenxi Yang
d0d138bc55
[Nixl][P/D] Add cuda2cpu support (HD->DH transfer) (#24690) 
Signed-off-by: Chenxi Yang <cxyang@fb.com>
Co-authored-by: Chenxi Yang <cxyang@fb.com>
 2025-09-29 14:31:51 +00:00
Chauncey
2827b3f4a3
[CI] Fix test_shared_storage_connector_hashes (#25748) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-26 20:46:17 +08:00
Peter Pan
da5e7e4329
[Docs] NixlConnector quickstart guide (#24249) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2025-09-23 14:23:22 +00:00
Chauncey
f05a4f0e34
[P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2025-09-23 16:08:02 +02:00
Chendi.Xue
5774b0a1da
[NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
 2025-09-23 04:17:42 +00:00
Nicolò Lucchesi
090197034f
[Bugfix] Fix missing clear_connector_metadata (#25397) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-23 08:10:59 +08:00
Or Ozeri
c59a0eca42
[KV offload][4/N] Offloading KV connector (#22595) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-09-19 19:07:17 +00:00
Nicolò Lucchesi
a3d087adec
[P/D][Nixl] Introduce KVTransferMetrics and aggregation strategy (#22188) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-19 11:09:14 +00:00
Zebing Lin
82dfb12e52
[Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-09-08 21:34:37 -07:00
Didier Durand
d7e1e59972
[Doc]: fix typos in Python comments (#24093) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-02 21:05:45 -07:00
co63oc
1bd007f234
fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
Flora Feng
69f46359dd
[Multimodal] Consolidate mm inputs into MultiModalFeatureSpec (#23779) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-08-29 18:36:57 +08:00
Flora Feng
53415653ff
[P/D][Nixl] Make kv cache register compatible with hybrid memory allocator (#23079) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-08-21 22:30:48 -07:00
Woosuk Kwon
c9b38be8aa
[Spec Decode] Make propose_draft_token_ids non-blocking for lower TTFT (#23041) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-18 17:20:38 -07:00
Or Ozeri
c280066f9d
[v1] Move block_hashes from KVCacheManager to Request.block_hashes (#19728) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-08-15 16:52:52 -07:00
Cyrus Leung
19b927e52d
[Core] Use individual MM items in P0/P1 cache and model runner (#22570) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-13 07:18:07 -07:00
Nicolò Lucchesi
6b794c756c
[Nixl][CI] Fix tests (#22806) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-08-13 06:03:53 -07:00
Nicolò Lucchesi
422f22e012
[CI][Nixl] Check kv cache layout during handshake (#22745) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-08-12 12:53:52 -07:00
Nicolò Lucchesi
3d9d40efde
[Bugfix][CI] Fix test_remote_decode_lifecycle.py::test_short_prompt_lifecycle (#22727) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-08-12 07:30:17 -07:00
GuanLuo
16fb668b61
fix: NIXL connector transfers partial block to pass full multi-modal context (#21074) 
Signed-off-by: GuanLuo <gluo@nvidia.com>
 2025-08-11 09:40:55 -07:00
Nick Hill
5898b135ab
[BugFix] Fix KVConnectorOutput TPU breakage (#22598) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-10 19:33:48 -07:00
Tyler Michael Smith
8ecb3e9e93
[CI Bugfix] Fix wNa16 kernel not found for test_shared_storage_connector_hashes (#22163) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-08-03 22:19:04 -07:00
Abirdcfly
0d7db16a92
[PD] add test for chat completions endpoint (#21925) 
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
 2025-08-03 19:57:03 -07:00
David Ben-David
aefeea0fde
[V1] [P/D] Refactor KV Connector Path (#21980) 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
 2025-08-03 04:03:40 -07:00
Roger Wang
067c34a155
docs: remove deprecated disable-log-requests flag (#22113) 
Signed-off-by: Roger Wang <hey@rogerw.me>
 2025-08-02 00:19:48 -07:00
Michael Goin
055bd3978e
[CI Bugfix] Fix CI OOM for test_shared_storage_connector_hashes (#21973) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-31 11:45:29 +08:00
Chenguang Zheng
4904e53c32
[Bugfix] SharedStorage Connector for V1 PD multimodal (#21611) 
Signed-off-by: fake0fan <645327136@qq.com>
Signed-off-by: herotai214 <herotai214@gmail.com>
Co-authored-by: herotai214 <herotai214@gmail.com>
 2025-07-30 09:18:37 -07:00
Kuntai Du
b18b417fbf
Revert "[V1] Exception Handling when Loading KV Cache from Remote Store" (#21778) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-07-28 20:15:18 +00:00
Adeline
15a72ac478
[V1] Exception Handling when Loading KV Cache from Remote Store (#21534) 
Signed-off-by: liuyumoye <adeline_ly2023@outlook.com>
Co-authored-by: liuyumoye <adeline_ly2023@outlook.com>
 2025-07-27 20:34:17 -07:00
Cyrus Leung
86ae693f20
[Deprecation][2/N] Replace --task with --runner and --convert (#21470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-27 19:42:40 -07:00
Juncheng Gu
6066284914
[P/D] Support CPU Transfer in NixlConnector (#18293) 
Signed-off-by: Juncheng Gu <juncgu@gmail.com>
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Richard Liu <39319471+richardsliu@users.noreply.github.com>
Co-authored-by: Richard Liu <ricliu@google.com>
 2025-07-24 17:58:42 +01:00
Rui Qiao
1e9ea8e69d
[P/D] Move FakeNixlWrapper to test dir (#21328) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-24 08:53:45 -07:00
Jiayi Yan
7ba34b1241
[bugfix] fix syntax warning caused by backslash (#21251) 2025-07-20 17:12:10 +00:00
kourosh hakhamaneshi
9f414a12ad
[BugFix] Make PD work with Ray (#21072) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
 2025-07-19 08:46:50 -07:00
Alexander Matveev
5b032352cc
[Attention] MLA - Flashinfer Ragged Prefill (#20034) 2025-07-10 20:17:47 -07:00
Yiming
cd587c93ef
[BugFix]: Properly set engine_id when using multi connector (#19487) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: leiyiming <leiyiming@kingsoft.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-07-09 20:32:44 +00:00
Nicolò Lucchesi
71d1d75b7a
[PD][Nixl] Remote consumer READ timeout for clearing request blocks (#20139) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-08 08:56:40 +01:00
Woosuk Kwon
2863befce3
[Optimization] Use Shared CachedRequestData Instance Across All Requests (#20232) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 09:07:50 -07:00
Nicolò Lucchesi
2582683566
[PD] Skip tp_size exchange with rank0 (#19413) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-25 20:04:39 -07:00
lkchen
4734704b30
[PD] let toy proxy handle /chat/completions (#19730) 
Signed-off-by: Linkun <github@lkchen.net>
 2025-06-25 15:17:45 -04:00
lkchen
91f7d9d0b6
[P/D] Asynchronously do _nixl_handshake (#19836) 
Signed-off-by: Linkun Chen <github@lkchen.net>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-24 12:46:10 -07:00
lkchen
1bcd15edc7
[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when *all* transfer done (#19874) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-06-22 22:41:53 -07:00