xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-06 14:47:05 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,126 Commits 155 Branches 89 Tags
Commit Graph

5126 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Woosuk Kwon
a7facf98d9 minor 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 23:44:46 -07:00
Woosuk Kwon
1e7bf7970a minor 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 23:38:12 -07:00
Woosuk Kwon
24ce0a7638 Add simple scheduler 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 23:25:10 -07:00
Woosuk Kwon
e484ecb947 Add logging 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 22:09:30 -07:00
Woosuk Kwon
da07067215 Add common states 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 22:04:57 -07:00
Woosuk Kwon
6cd1b1a18c Merge branch 'main' into v1-sched-interface-2 2025-03-12 21:45:44 -07:00
Siyuan Liu
1bc3b739c4
[V1][TPU] Add assertion on multi-step-scheduler (#14707) 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
 2025-03-12 21:37:58 -07:00
Mathis Felardos
1bd32bc8dd
[Config][Disaggregated] Add timeout configuration for the torch.store and add KVTransferConfig.kv_connector_extra_config (#14367) 
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
 2025-03-12 20:15:20 -07:00
TY-AMD
128bf75283
[BugFix][TritonMLA] Process weights after model loading for GGUF (#14555) 
Signed-off-by: TianyuanWu <Tianyuan.Wu@amd.com>
 2025-03-12 20:14:36 -07:00
Gregory Shtrasberg
a94a699c3f
[ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-03-12 20:14:04 -07:00
Richard Liu
ab426ec9c0
Add ray[data] as tpu dependency (#14691) 
Signed-off-by: <ricliu@google.com>
Signed-off-by: Richard Liu <ricliu@google.com>
 2025-03-12 20:13:48 -07:00
Joe Runde
165290d357
[bugfix] fixup warning message for plugged schedulers for v1 (#14700) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-03-12 20:12:13 -07:00
Woosuk Kwon
8730469cfa mv to interface 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 18:23:39 -07:00
Woosuk Kwon
5b38e984b3 Merge branch 'main' into v1-sched-interface-2 2025-03-12 18:12:39 -07:00
Kevin H. Luu
ce20124671
[release] Add force remove for TPU logs (#14697) 2025-03-12 22:35:18 +00:00
Woosuk Kwon
53be4a8634
[V1] Allow sliding window + prefix caching (#13069) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 11:21:19 -07:00
Nick Hill
f5d3acd474
[BugFix][V1] Fix parallel sampling finishing/aborts (#14512) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-12 10:29:48 -07:00
TJian
916836bbfb
[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-12 09:31:19 -07:00
Sage Moore
d9f83d6206
[ROCm] Enable chunked prefill/paged attention in MLA on ROCm (#14316) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-03-12 15:51:20 +00:00
ameyanjarlekar
4a754fcf15
[Bugfix] Missing thumbnail from NVLM-D processor (#14633) 
Signed-off-by: ameyanjarlekar <aanjarlekar@nvidia.com>
 2025-03-12 08:50:49 -07:00
Woosuk Kwon
c0c25e25fa
[Model] Add support for Gemma 3 (#14660) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-12 08:36:33 -07:00
Sage Moore
45f3f3f59e
[ROCm][Bugfix] Ensure that the moe_wna16_gemm kernel is not built on ROCm platforms. (#14629) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-03-12 08:00:28 -04:00
Li, Jiang
ff47aab056
[CPU] Upgrade CPU backend to torch-2.6 (#13381) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-03-12 10:41:13 +00:00
Woosuk Kwon
06e22ba44c interface 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-12 00:00:03 -07:00
Woosuk Kwon
8d46d5d11d Minor 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-11 23:43:04 -07:00
Woosuk Kwon
f198d7d07a utils 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-11 23:36:42 -07:00
Woosuk Kwon
0bf6e97493 sched 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-11 23:35:10 -07:00
Woosuk Kwon
6e7209347d mv 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-11 23:27:38 -07:00
Pavani Majety
debd6bbf09
[Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-03-12 05:13:11 +00:00
Benjamin Chislett
5c538c37b2
[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
 2025-03-11 22:12:41 -07:00
Szymon Ożóg
e22ee1e7a2
[Kernel] GGUF MoE kernel (#14613) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
 2025-03-12 03:33:27 +00:00
Isotr0py
e392d85831
[Core] Refactor QKVCrossParallelLinear implementation to support BNB 4-bit quantization (#14545) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-11 20:12:52 -07:00
Aaron Pham
77a318bd01
[V1][Core] Support MistralTokenizer for Structured Output (#14625) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-12 10:40:09 +08:00
Farzad Abdolhosseini
80e78d02ac
[Model] Extend Ultravox to accept audio longer than 30s (#13631) 
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
 2025-03-12 10:27:10 +08:00
Jennifer Zhao
4a42b9f5d6
[Doc] Update benchmarks README (#14646) 
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2025-03-11 19:23:04 -07:00
Joe Runde
47532cd9f4
[core][V1] pluggable scheduler (#14466) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-03-12 01:15:15 +00:00
Randy Chen
36e0c8f7da
[Feature] Add vllm bench CLI (#13993) 
Signed-off-by: Randy Chen <acad.randyjhc@gmail.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-03-12 00:31:48 +00:00
Kevin H. Luu
9f583e360c
[release] Add commands to clean up logs on TPU release node (#14642) 2025-03-12 00:14:50 +00:00
Cody Yu
b706d898af
[Bugfix][V1][PP] Only warmup sampler at last PP rank (#14643) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-03-11 23:40:07 +00:00
iefgnoix
863d315c86
[V1][TPU] Pad the block_table.shape[1] so the ragged paged attention can handle correctly (#14597) 2025-03-11 19:12:26 -04:00
Richard Liu
d374f04a33
Fix run_tpu_test (#14641) 
Signed-off-by: <ricliu@google.com>
Signed-off-by: Richard Liu <ricliu@google.com>
 2025-03-11 21:14:33 +00:00
Russell Bryant
61a01b27a7
[V1] Delay all xgrammar usage until needed (#14616) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-11 20:21:33 +00:00
Yang.Tao
53056731fd
fix some typos : supported_head_sizes (#14627) 2025-03-11 10:38:24 -07:00
Russell Bryant
4cbf286794
[V1] Remove cache from StructuredOutputManager (#14622) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-11 10:36:07 -07:00
Kunshang Ji
c6e14a61ab
[Hardware][Intel GPU] upgrade IPEX dependency to 2.6.10. (#14564) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-03-11 17:11:47 +00:00
Lucas Wilkinson
07b4b7a37f
[BugFix/Build] Fix sparse kernels not getting built on hopper (#14572) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-11 17:09:03 +00:00
Dilip Gowda Bhagavan
07964e2f30
docs: Add documentation for s390x cpu implementation (#14198) 
Signed-off-by: Dilip Gowda Bhagavan <dilip.bhagavan@ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-11 17:02:17 +00:00
Russell Bryant
4bf82d4b90
[V1] Add regex structured output support with xgrammar (#14590) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-11 23:03:44 +08:00
Richard Liu
9ab326713f
Uninstall dependencies before installing requirements/tpu.txt (#14586) 
Signed-off-by: <ricliu@google.com>
Signed-off-by: Richard Liu <ricliu@google.com>
 2025-03-11 08:01:35 -07:00
Cyrus Leung
af295e9b01
[Bugfix] Update --hf-overrides for Alibaba-NLP/gte-Qwen2 (#14609) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-11 07:59:43 -07:00