xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-11-01 15:55:44 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,575 Commits 155 Branches 89 Tags
Commit Graph

970 Commits

Author SHA1 Message Date
22quinn
54de71d0df
[Sampler] Support returning all logprobs or logits (#21792) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-08-04 03:04:12 -07:00
ZiTian.Zhao
49bcd893e7
[refactor] improve ConstantList exception specificity (#22156) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
 2025-08-03 22:14:49 -07:00
Giancarlo Delfin
aa7012eb6d
Add tree attention backend for v1 (part 1) (#20401) 
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
 2025-08-03 22:13:26 -07:00
ZiTian.Zhao
e27d25a0dc
[fix] fix correct assertion syntax error in attention utils. (#22154) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
 2025-08-03 19:24:02 -07:00
David Ben-David
aefeea0fde
[V1] [P/D] Refactor KV Connector Path (#21980) 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
 2025-08-03 04:03:40 -07:00
H
24d1dffbeb
[executor] feat: add supports_pp attr to executors (#21786) 
Signed-off-by: Haibin Lin <haibin.lin@bytedance.com>
 2025-08-03 18:04:45 +08:00
Thomas Parnell
4abfd8796f
[V1] [Hybrid] Validate compatibility of attention backend batch reordering at init time (#21557) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-02 05:29:40 -07:00
Sage Moore
0edaf752d7
[Attention][DBO] Add support for "splitting" the CommonAttentionMetadata (#21153) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-08-01 19:47:53 -07:00
Nick Hill
8d524ce79f
[BugFix] Improve internal DP load balancing (#21617) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-01 19:45:27 -07:00
fhl2000
23322431c8
[V1][CUDA] Full cudagraph support for FlashInfer (#21367) 2025-08-01 21:49:34 -04:00
Nick Hill
881e1af43a
[BugFix] Harden distributed DP startup (#21538) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-01 21:40:45 +00:00
rongfu.leng
b879ecd6e2
[Bugfix] fix when skip tokenizer init (#21922) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-08-01 10:09:36 -07:00
Harry Mellor
2d7b09b998
Deprecate --disable-log-requests and replace with --enable-log-requests (#21739) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-01 17:16:37 +01:00
Michael Goin
f81c1bb055
[Bugfix] Check NVIDIA artifactory is accessible before using flashinfer cubin kernels (#21893) 2025-08-01 08:28:45 -04:00
wuhang
e6680f9e25
[Bugfix] Add log prefix in non-dp mode engine core (#21889) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2025-08-01 09:04:16 +00:00
Zebing Lin
e0f63e4a35
[Core] Avoid repeated len(block_token_ids) check in hash_request_tokens (#21781) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-08-01 00:23:29 -07:00
Mickaël Seznec
e1a7fe4af5
[BugFix] fix: aot passes kvcache dtype information (#19750) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
 2025-08-01 05:45:02 +00:00
zhiweiz
9e0726e5bf
[Meta] Official Eagle mm support, first enablement on llama4 (#20788) 
Signed-off-by: morgendave <morgendave@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-07-31 10:35:07 -07:00
Nick Hill
5daffe7cf6
[BugFix] Fix case where collective_rpc returns None (#22006) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-31 12:51:37 +00:00
Michael Goin
61445453df
[UX] Rename CUTLASS_MLA_VLLM_V1 to CUTLASS_MLA (#21966) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-30 20:40:34 -07:00
Zebing Lin
ca9e2be3ed
[Core] Move EngineCoreRequest to Request conversion out of EngineCore (#21627) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-07-30 15:00:54 -07:00
Nick Hill
56bd537dde
[Misc] Support more collective_rpc return types (#21845) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-30 10:20:20 -07:00
633WHU
5c765aec65
[Bugfix] Fix TypeError in scheduler when comparing mixed request_id types (#21816) 
Signed-off-by: chiliu <chiliu@paypal.com>
Co-authored-by: chiliu <chiliu@paypal.com>
 2025-07-30 08:54:44 -07:00
Yong Hoon Shin
ad510309ee
Override attention metadata for fast prefill in some KV sharing setups (#21590) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-07-30 08:54:15 -07:00
Ruixiang Tan
8f4a1c9a04
[Misc] Improve code readability of KVCacheManager (#21673) 
Signed-off-by: tanruixiang <tanruixiang0104@gmail.com>
Signed-off-by: Ruixiang Tan <819464715@qq.com>
Signed-off-by: GitHub <noreply@github.com>
 2025-07-30 07:20:43 -07:00
MingzhenHan
b7b23da4d2
[Bugfix] Fix comment typo of get_num_common_prefix_blocks() (#21827) 
Signed-off-by: MingzhenHan <hanmingzhen2002@outlook.com>
 2025-07-29 20:35:33 -07:00
Csrayz
b917da442b
Expose PyTorch profiler configuration to environment variables (#21803) 
Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>
 2025-07-29 19:46:31 -07:00
Chen Zhang
555e7225bc
[v1][attention] Support Hybrid Allocator + FlashInfer (#21412) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-07-30 01:45:29 +00:00
elvischenv
58b11b24a6
[Bugfix] Fix workspace buffer None issue for Flashinfer TRTLLM Backend (#21525) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-07-29 10:34:00 -04:00
Richard Zou
04e38500ee
[Bugfix] VLLM_V1 supports passing other compilation levels (#19340) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-07-29 09:35:58 -04:00
Chen Zhang
755fa8b657
[KVCache] Make KVCacheSpec hashable (#21791) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-07-29 19:58:29 +08:00
Nick Hill
7234fe2685
[Misc] Rework process titles (#21780) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-29 05:14:47 +00:00
Kuntai Du
b18b417fbf
Revert "[V1] Exception Handling when Loading KV Cache from Remote Store" (#21778) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-07-28 20:15:18 +00:00
Michael Goin
34a20c49b3
[Logs] Change flashinfer sampler logs to once (#21759) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-28 06:59:51 -07:00
wuhang
bccc43c033
[Bugfix]check health for engine core process exiting unexpectedly (#21728) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2025-07-28 06:17:31 -07:00
Asaf Joseph Gardin
a6c050286a
[v1][mamba] Added mamba_type into MambaSpec (#21715) 
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-07-28 08:15:55 +00:00
Lucas Wilkinson
139a7f07bd
[BugFix] Fix ChunkedLocalAttention when the hybrid kv-cache is disabled (#21707) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-28 07:18:47 +00:00
Adeline
15a72ac478
[V1] Exception Handling when Loading KV Cache from Remote Store (#21534) 
Signed-off-by: liuyumoye <adeline_ly2023@outlook.com>
Co-authored-by: liuyumoye <adeline_ly2023@outlook.com>
 2025-07-27 20:34:17 -07:00
Cyrus Leung
86ae693f20
[Deprecation][2/N] Replace --task with --runner and --convert (#21470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-27 19:42:40 -07:00
ZiTian.Zhao
a8936e5193
Refactor: Remove numpy dependency from LoggingStatLogger (#20529) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
 2025-07-27 04:06:21 +00:00
Ye (Charlotte) Qi
a40a8506df
[Misc] Improve memory profiling debug message (#21429) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-07-26 07:07:21 -07:00
Maximilien de Bayser
1cd6eaba54
Support encoder-only models without KV-Cache (#21270) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-07-26 21:09:52 +08:00
Rui Qiao
cea96a0156
[Bugfix] Fix sync_and_slice_intermediate_tensors (#21537) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-25 17:07:58 -07:00
who who who
b3caeb82e7
[ROCm][AITER] Enable fp8 kv cache on rocm aiter backend. (#20295) 
Signed-off-by: fsx950223 <fsx950223@outlook.com>
Signed-off-by: amd-ruitang3 <Rui.Tang2@amd.com>
Co-authored-by: amd-ruitang3 <Rui.Tang2@amd.com>
 2025-07-25 06:50:21 -07:00
Mengqing Cao
f3a683b7c9
[Bugfix][Logprobs] Fix logprobs op to support more backend (#21591) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-07-25 05:53:07 -07:00
Cyrus Leung
46d81d6951
[V1] Get supported tasks from model runner instead of model config (#21585) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-25 05:36:45 -07:00
weiliang
2dd72d23d9
update flashinfer to v0.2.9rc1 (#21485) 
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
 2025-07-24 14:06:11 -07:00
Juncheng Gu
6066284914
[P/D] Support CPU Transfer in NixlConnector (#18293) 
Signed-off-by: Juncheng Gu <juncgu@gmail.com>
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Richard Liu <39319471+richardsliu@users.noreply.github.com>
Co-authored-by: Richard Liu <ricliu@google.com>
 2025-07-24 17:58:42 +01:00
Lucas Wilkinson
61b8cea3b4
[Attention] Optimize FlashInfer MetadataBuilder Build call (#21137) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-24 03:21:46 -07:00
Chauncey
6da0078523
[Feat] Allow custom naming of vLLM processes (#21445) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-24 03:15:23 -07:00