xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-22 20:15:46 +08:00
Code Issues Packages Projects Releases Wiki Activity
12,170 Commits 155 Branches 89 Tags
Commit Graph

85 Commits

Author SHA1 Message Date
Harry Mellor
8781cd6b88
Add Eagle and Eagle3 support to Transformers modeling backend (#30340) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-11 17:02:10 +00:00
Wentao Ye
d6464f2679
[Chore] Fix torch precision warning (#30428) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-11 04:05:56 +00:00
Andreas Karatzas
ed7af3178a
[ROCm][CI] Attempt to fix the failures under a subgroup of the e2e the test group (#29358) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2025-12-10 05:33:13 +00:00
Micah Williamson
7d80c73d42
[CI] Reduce Flakiness For test_spec_decode.py::test_suffix_decoding_acceptance (#30367) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2025-12-10 02:35:49 +00:00
Lucas Wilkinson
abe93bce59
[Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable true async spec-decode (#29624) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-12-09 17:18:10 -08:00
Wentao Ye
83319b44c2
[Compile] Fix torch warning TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled (#29897) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-09 10:40:37 -05:00
Charlie Fu
6af70e11a0
[ROCm][CI] Fix test_max_len.py for Rocm (#29916) 
Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Charlie Fu <Charlie.Fu@amd.com>
 2025-12-08 16:58:30 -05:00
Divakar Verma
962d703818
[Bugfix][llama4_eagle] Fix missing 'lm_head' attribute (#29926) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-12-05 19:57:26 +00:00
Divakar Verma
afb1e5b380
[CI][ROCm][tests/v1/e2e] Fix multiprocessing launch for the test (#29123) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-12-02 20:46:10 +00:00
Divakar Verma
e2fbfc955e
[CI][AMD] spec_decode:eagle skip FLASH_ATTN for deepseek on ROCm (#29827) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-12-02 05:27:46 +00:00
Divakar Verma
a690fb5bd6
[CI][ROCm] Fix test_correctness_sliding_window (#29243) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-02 04:53:27 +00:00
Nick Hill
44822d7ff2
[BugFix] Preserve spec decoding uniform decode when scheduling (#29759) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-12-01 17:15:52 -08:00
EanWang211123
37b15e97e8
[Multimodal][Speculative Decoding]Eagle3 mm support, enablement on qwen3vl (#29594) 
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: EanWang211123 <wangyiheng@sangfor.com.cn>
Co-authored-by: Louie Tsai <louie.tsai@intel.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-11-27 22:05:45 -08:00
Nick Hill
4e57c6587f
[Core] Support logprobs with spec decode + async scheduling (#29223) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-25 12:55:24 -08:00
WeiQing Chen
b34129bf8e
[Misc] remove useless v1 env (#29164) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-11-21 01:41:20 -08:00
Nick Hill
5bdd155277
[CI] Fix async scheduling + spec decoding test flake (#28902) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-18 05:26:32 +00:00
Ronald
d8874c61a5
[Core] Async Scheduling X Spec Decoding Compatibility (#24799) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-11-17 12:16:20 -08:00
Nick Hill
80b6080ddc
[BugFix] Fix async scheduling + chunked prefill + preemption (#28787) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-17 06:46:46 +08:00
Nick Hill
58e61e56b7
[Test] Rework e2e async scheduling tests (#28744) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-14 16:01:09 -08:00
Laith Sakka
2e0ad629b0
Avoid bytecode hook and simplify TorchCompileWrapperWithCustomDipatch (#25110) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-11-14 14:11:10 -08:00
Cyrus Leung
e2741f6cbc
[Chore] Rename SchedulerConfig.chunked_prefill_enabled (#28735) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-14 18:39:57 +00:00
Yong Hoon Shin
9324e10275
Fix KV sharing fast prefill with cudagraph enabled (#28537) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-14 11:53:42 +00:00
Yannick Schnider
119c4927b3
[Bugfix] Fix validate model input for decoder models (#27099) 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-13 10:18:47 -08:00
Nicolò Lucchesi
19d91ece4b
[CI] Fix flaky test_eagle_correctness test (#28364) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-11-09 16:04:59 +00:00
Xiaohong (Sean) Chen
d0c7792004
[Bugfix][LoRA][Spec Decode] Support LoRA with speculative decoding (#21068) 
Signed-off-by: Sean Chen <xiaohong_chen1991@hotmail.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Danielle Robinson <dcmaddix@gmail.com>
Co-authored-by: Haipeng Li <li2haipeng@gmail.com>
Co-authored-by: li2haipeng <44383182+li2haipeng@users.noreply.github.com>
 2025-11-08 01:58:22 +00:00
Nick Hill
938a81692e
[AsyncScheduling] Don't schedule past request max_tokens (#27922) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-04 17:06:28 +00:00
Aurick Qiao
2c19d96777
[Spec Decode] Integrate Suffix Decoding from Arctic Inference (#25784) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-11-03 09:23:31 -08:00
Rémi Delacourt
cec7c28833
[Bugfix] Padded Eagle Specdec with Chunked Prefill (#26263) 
Signed-off-by: Rémi Delacourt <remi@mistral.ai>
Signed-off-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-11-03 02:22:46 -05:00
Nick Hill
0cdbe7b744
[Core] Async scheduling + structured outputs compatibility (#26866) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-01 00:35:04 +00:00
Dipika Sikka
413ef7a3b4
[Speculators] Move tests + fix integration (#27308) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Signed-off-by: rahul-tuli <rtuli@redhat.com>
Co-authored-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-10-29 00:54:21 -07:00
Nick Hill
4fe5895361
[AsyncScheduling] Make async overlap work with logprobs (#27615) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-28 22:35:54 +00:00
Huy Do
becb7de40b
Update PyTorch to 2.9.0+cu129 (#24994) 
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-21 17:20:18 -04:00
Bram Wasti
b2f78cbad4
[small][batch invariance] Rename the env and internal flags to simplify usage (#26855) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-10-16 21:40:25 +00:00
Morrison Turnansky
96b9aa5aa0
[Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change compilation level to compilation mode, deprecation compilation level (#26355) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-15 02:51:16 +00:00
Maximilien de Bayser
d8bebb008a
Add tests for chunked prefill and prefix cache with causal pooling models (#26526) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Ayush Singh <ayush1009208@gmail.com>
 2025-10-14 07:45:04 +08:00
Harry Mellor
8fcaaf6a16
Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Nick Hill
5bc26c438d
[BugFix] Make penalties and bad_words work with async scheduling (#26467) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-10 23:27:04 +00:00
Nick Hill
949cb0170d
[BugFix] Fix async scheduling + request preemption (#26385) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-10 20:29:57 +00:00
Thomas Parnell
31a4b3e6c4
Revert #24446 and #26168 (#26332) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-07 16:38:19 -06:00
Cyrus Leung
1e4ecca1d0
[V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:42:31 +00:00
Yannick Schnider
6431be808f
[Tests] conftest: Extending VllmRunner and HfRunner to accept token_ids as input (#26295) 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-06 17:19:34 +00:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Yannick Schnider
f05fea1f5e
[Core] Enable decode of context length equal to max model length (#26168) 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
 2025-10-04 09:59:26 +00:00
Yannick Schnider
8ee846c27c
[Bugfix] Re-enable prefill of max model length (#24446) 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
 2025-10-03 14:13:34 +02:00
WeiQing Chen
f1d53d150c
[Multimodal][Speculative Decoding]Eagle Eagle3 mm support, enablement on qwen2.5vl (#22872) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: LJH-LBJ <98734602+LJH-LBJ@users.noreply.github.com>
 2025-09-27 03:35:47 +00:00
Jonas M. Kübler
6f5c0931c1
[Spec decode] automatically disable mm for text-only draft models (#25667) 
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
 2025-09-27 08:10:21 +08:00
qizixi
c70ac4b8ff
[spec decode] Consolidate speculative decode method name for MTP (#25232) 
Signed-off-by: zixi-qi <qizixi@meta.com>
 2025-09-26 22:27:05 +00:00
Matthew Bonanni
3468f17ebe
[V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-25 17:37:50 +00:00
Woosuk Kwon
eb68c2dcd9
[CI] Revert back prepare_prompts and check_answers (#25087) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-17 11:03:16 -07:00
Wenlong Wang
cfa3234a5b
[CI][Spec Decode] Adjust threshold for flaky ngram spec decoding test again (#24771) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-13 15:45:11 +08:00