xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-18 19:07:11 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,124 Commits 155 Branches 89 Tags
Commit Graph

538 Commits

Author SHA1 Message Date
youkaichao
2382ad29d1
[ci] fix linter (#13701) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 20:28:59 +08:00
youkaichao
3e472d882a
[core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Cyrus Leung
7f6bae561c
[CI/Build] Fix pre-commit errors (#13696) 2025-02-22 00:31:26 -08:00
Mark McLoughlin
2cb8c1540e
[Metrics] Add --show-hidden-metrics-for-version CLI arg (#13295) 2025-02-22 00:20:45 -08:00
Mark McLoughlin
1cd981da4f
[V1][Metrics] Support vllm:cache_config_info (#13299) 2025-02-22 00:20:00 -08:00
Jennifer Zhao
da31b5333e
[Bugfix] V1 Memory Profiling: V0 Sampler Integration without Rejection Sampler (#13594) 
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-02-22 00:08:29 -08:00
Lu Fang
bb78fb318e
[v1] Support allowed_token_ids in v1 Sampler (#13210) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-02-22 14:13:05 +08:00
Jun Duan
68d535ef44
[Misc] Capture and log the time of loading weights (#13666) 2025-02-21 22:06:34 -08:00
Lucas Wilkinson
288cc6c234
[Attention] MLA with chunked prefill (#12639) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-02-21 15:30:12 -08:00
Nick Hill
31aa045c11
[V1][Sampler] Avoid an operation during temperature application (#13587) 2025-02-20 22:05:56 -08:00
Woosuk Kwon
d3ea50113c
[V1][Minor] Print KV cache size in token counts (#13596) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-20 09:24:31 -08:00
youkaichao
ba81163997
[core] add sleep and wake up endpoint and v1 support (#12987) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
 2025-02-20 12:41:17 +08:00
Nick Hill
a4c402a756
[BugFix] Avoid error traceback in logs when V1 LLM terminates (#13565) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-20 00:49:01 +00:00
Nick Hill
caf7ff4456
[V1][Core] Generic mechanism for handling engine utility (#13060) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-19 17:09:22 +08:00
Woosuk Kwon
4c82229898
[V1][Spec Decode] Optimize N-gram matching with Numba (#13365) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-18 13:19:58 -08:00
Nick Hill
30172b4947
[V1] Optimize handling of sampling metadata and req_ids list (#13244) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-18 12:15:33 -08:00
Woosuk Kwon
4fb8142a0e
[V1][PP] Enable true PP with Ray executor (#13472) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-18 09:15:32 -08:00
youkaichao
932b51cedd
[v1] fix parallel config rank (#13445) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-18 12:33:45 +08:00
Woosuk Kwon
cd4a72a28d
[V1][Spec decode] Move drafter to model runner (#13363) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-17 15:40:12 -08:00
Cody Yu
6ac485a953
[V1][PP] Fix intermediate tensor values (#13417) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-02-17 13:37:45 -08:00
Woosuk Kwon
4c21ce9eba
[V1] Get input tokens from scheduler (#13339) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-17 11:01:07 -08:00
Jee Jee Li
2010f04c17
[V1][Misc] Avoid unnecessary log output (#13289) 2025-02-16 19:26:24 -08:00
Woosuk Kwon
69e1d23e1e
[V1][BugFix] Clean up rejection sampler & Fix warning msg (#13362) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-16 12:25:29 -08:00
Woosuk Kwon
e18227b04a
[V1][PP] Cache Intermediate Tensors (#13353) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-16 10:02:27 -08:00
Woosuk Kwon
7b89386553
[V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-16 09:39:08 -08:00
Lily Liu
80f63a3966
[V1][Spec Decode] Ngram Spec Decode (#12193) 
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-02-15 18:05:11 -08:00
Cody Yu
9206b3d7ec
[V1][PP] Run engine busy loop with batch queue (#13064) 2025-02-15 03:59:01 -08:00
Mark McLoughlin
2ad1bc7afe
[V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) 2025-02-15 03:56:19 -08:00
Woosuk Kwon
0c73026844
[V1][PP] Fix memory profiling in PP (#13315) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-14 20:17:25 -08:00
Nick Hill
6a854c7a2b
[V1][Sampler] Don't apply temp for greedy-only (#13311) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-14 18:10:53 -08:00
Aoyu
a12934d3ec
[V1][Core] min_p sampling support (#13191) 
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
 2025-02-14 15:50:05 -08:00
Lu Fang
6224a9f620
Support logit_bias in v1 Sampler (#13079) 2025-02-14 04:34:59 -08:00
Nick Hill
085b7b2d6c
[V1] Simplify GPUModelRunner._update_states check (#13265) 2025-02-14 04:33:43 -08:00
Alexander Matveev
45f90bcbba
[WIP] TPU V1 Support Refactored (#13049) 2025-02-14 00:21:53 -08:00
Kero Liang
b0ccfc565a
[Bugfix][V1] GPUModelRunner._update_states should return True when there is a finished request in batch (#13126) 2025-02-13 22:39:20 -08:00
Sage Moore
ba59b78a9c
[ROCm][V1] Add intial ROCm support to V1 (#12790) 2025-02-13 22:21:50 -08:00
Varun Sundar Rabindranath
cbc40128eb
[V1] LoRA - Enable Serving Usecase (#12883) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-02-14 14:21:12 +08:00
Roger Wang
dd5ede4440
[V1] Consolidate MM cache size to vllm.envs (#13239) 2025-02-13 20:19:03 -08:00
Aoyu
2092a6fa7d
[V1][Core] Add worker_base for v1 worker (#12816) 
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-02-13 20:35:18 +08:00
Roger Wang
fdcf64d3c6
[V1] Clarify input processing and multimodal feature caching logic (#13211) 2025-02-13 03:43:24 -08:00
Rui Qiao
9605c1256e
[V1][core] Implement pipeline parallel on Ray (#12996) 2025-02-13 08:02:46 +00:00
Murali Andoorveedu
4c0d93f4b2
[V1][Bugfix] Copy encoder input ids to fix set iteration issue during VLM abort (#13173) 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
 2025-02-12 12:58:11 -08:00
bnellnm
f4d97e4fc2
[Bug] [V1] Try fetching stop_reason from EngineOutput before checking the request (#13108) 2025-02-12 02:39:16 -08:00
Mark McLoughlin
75e6e14516
[V1][Metrics] Add several request timing histograms (#12644) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-02-11 10:14:00 -05:00
Cody Yu
41c5dd45b9
[V1][Metrics] Add GPU prefix cache hit rate % gauge (#12592) 2025-02-11 08:27:25 +00:00
Woosuk Kwon
2ff4857678
[V1][Minor] Move scheduler outputs to a separate file (#13062) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-11 02:10:06 +00:00
Nick Hill
67c4637ccf
[V1] Use msgpack for core request serialization (#12918) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-10 11:35:56 +08:00
Woosuk Kwon
24700c346b
[V1] Cache uses_mrope in GPUModelRunner (#12969) 2025-02-08 15:32:32 -08:00
Woosuk Kwon
870c37481e
[V1][Minor] Remove outdated comment (#12968) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-08 12:48:30 -08:00
youkaichao
fe743b798d
[bugfix] fix early import of flash attention (#12959) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-09 00:06:56 +08:00