xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-22 22:45:44 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,122 Commits 155 Branches 89 Tags
Commit Graph

6122 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wes
7a888271f5
Use Cache Hinting for fused_moe kernel (#15511) 2025-03-26 23:21:34 +00:00
Alexander Matveev
9d119a86ae
[V1] TPU CI - Fix test_compilation.py (#15570) 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
 2025-03-26 21:51:54 +00:00
Alexander Matveev
b2e85e26f4
[V1] TPU - Revert to exponential padding by default (#15565) 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
 2025-03-26 21:35:05 +00:00
Alexei-V-Ivanov-AMD
dd8a29da99
Applying some fixes for K8s agents in CI (#15493) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2025-03-26 20:35:11 +00:00
marko
27df5199d9
Support SHA256 as hash function in prefix caching (#15297) 
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
 2025-03-26 11:11:28 -07:00
Nick Hill
35fad35a48
[V1][Sampler] Faster top-k only implementation (#15478) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-26 10:56:47 -07:00
Aaron Pham
733e7c9e95
[Refactor] Remove unnecessary backend parameter in structured output interface (#15317) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-26 17:51:56 +00:00
Harry Mellor
0af4d764d6
Fix weight loading for some models in Transformers backend (#15544) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-26 10:17:53 -07:00
youkaichao
e64afa455c
multi-node offline DP+EP example (#15484) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-26 23:54:24 +08:00
Alex Brooks
1711b929b6
[Model] Add Reasoning Parser for Granite Models (#14202) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
 2025-03-26 14:28:07 +00:00
Harry Mellor
c091c0a588
Improve validation of TP in Transformers backend (#15540) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-26 07:26:48 -07:00
cyyever
1aa162e030
Apply torchfix (#15532) 
Signed-off-by: cyy <cyyever@outlook.com>
 2025-03-26 12:09:06 +00:00
Harry Mellor
cf5c8f1686
Separate base model from TransformersModel (#15467) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-03-26 18:13:38 +08:00
Reid
4ec2cee000
[Misc] improve example script output (#15528) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-03-26 10:12:47 +00:00
wwl2755
99f536f830
[Misc] Enhance warning information to user-defined chat template (#15408) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-03-26 02:21:15 -07:00
vllmellm
5ebf66748b
[FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-26 16:30:30 +08:00
Bryan Lu
781d056280
[Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990) 
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-26 08:24:07 +00:00
daniel-salib
5aefd6ac31
Fix raw_request extraction in load_aware_call decorator (#15382) 
Signed-off-by: Daniel Salib <danielsalib@meta.com>
 2025-03-25 22:29:54 -07:00
Varun Sundar Rabindranath
6c663dfd5e
[misc] LoRA - Skip LoRA kernels when not required (#15152) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-26 11:33:45 +08:00
Lucas Wilkinson
33437bc6e7
[BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1, producing bad results) (#15492) 
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
 2025-03-25 20:33:22 -07:00
Tyler Michael Smith
23114d3364
[Misc] Warn about v0 in benchmark_paged_attn.py (#15495) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-25 20:31:04 -07:00
Cyrus Leung
997c8811d6
[Model] Support multi-image for Molmo (#15438) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-26 11:26:33 +08:00
Harry Mellor
e42389f9d7
Transformers backend already supports V1 (#15463) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-25 20:26:16 -07:00
Varun Sundar Rabindranath
ff38f0a32c
[CI/Build] LoRA: Delete long context tests (#15503) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-25 17:18:34 -07:00
Varun Sundar Rabindranath
a5cfbab3c8
[Core] LoRA: V1 Scheduler optimization (#15422) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-25 22:50:09 +00:00
Chenyaaang
ac3cd6e83c
[core] add bucket padding to tpu_model_runner (#14995) 
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
 2025-03-25 17:27:22 -04:00
Lu Fang
082ab86f5f
[V1] Support long_prefill_token_threshold in v1 scheduler (#15419) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-03-25 14:22:26 -07:00
Nick Hill
6aa196c8dc
[V1][Minor] Use SchedulerInterface type for engine scheduler field (#15499) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-25 14:21:36 -07:00
Nicolò Lucchesi
a0dd7dcd49
[TPU][V1] Fix Sampler recompilation (#15309) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-03-25 16:43:54 -04:00
Maximilien de Bayser
e977c11111
Add workaround for shared field_names in pydantic model class (#13925) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-03-25 20:31:08 +00:00
Joe Runde
5f063a80bd
[bugfix] add supports_v1 platform interface (#15417) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-03-25 15:00:32 -04:00
Antonio Gómez
5d8e1c9279
[Bugfix] Support triton==3.3.0+git95326d9f for RTX 5090 (Unsloth + vLLM compatibility) (#15471) 
Co-authored-by: ServerAI <ai@exc-mad-ai.com>
 2025-03-25 17:59:25 +00:00
yarongmu-google
0a049c7d86
[CI/Build] Add tests for the V1 tpu_model_runner. (#14843) 
Signed-off-by: Yarong Mu <ymu@google.com>
 2025-03-25 12:27:16 -04:00
youkaichao
d0cfec7ab9
[bugfix] fix inductor cache on max_position_embeddings (#15436) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-25 07:05:39 -07:00
Szymon Ożóg
a608160027
[Kernel] Fix conflicting macro names for gguf kernels (#15456) 
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-03-25 13:50:49 +00:00
Cyrus Leung
3f04a7fbf2
[Doc] Update V1 user guide for multi-modality (#15460) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-25 11:01:58 +00:00
Cyrus Leung
5994430b84
[Misc] Remove redundant num_embeds (#15443) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-25 18:27:57 +08:00
Cyrus Leung
a9e879b316
[Misc] Clean up MiniCPM-V/O code (#15337) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-25 10:22:52 +00:00
Md. Shafi Hussain
3e2f37a69a
Dockerfile.ppc64le changes to move to UBI (#15402) 
Signed-off-by: Md. Shafi Hussain <Md.Shafi.Hussain@ibm.com>
 2025-03-25 10:15:14 +00:00
Thien Tran
4f044b1d67
[Kernel][CPU] CPU MLA (#14744) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-03-25 09:34:59 +00:00
Siyuan Liu
4157f563b4
[Hardware][TPU][Bugfix] Fix v1 mp profiler (#15409) 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
 2025-03-25 01:43:00 -07:00
Lu Fang
051da7efe3
Fix CUDA kernel index data type in vllm/csrc/quantization/gptq_marlin/awq_marlin_repack.cu +10 (#15160) 
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Richard Barnes <rbarnes@meta.com>
 2025-03-25 15:36:45 +08:00
Woosuk Kwon
25f560a62c
[V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
v0.8.2 		2025-03-24 21:04:41 -07:00
Russell Bryant
a09ad90a72
[V1] guidance backend for structured output + auto fallback mode (#14779) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
 2025-03-24 21:02:33 -07:00
Chauncey
10b34e36b9
[Bugfix] Fixed the issue of not being able to input video and image simultaneously (#15387) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-03-25 03:48:08 +00:00
Tyler Michael Smith
b5269db959
Revert "Fix non-contiguous input passed to Marlin kernel (#15319)" (#15398) 2025-03-24 20:43:51 -07:00
Jee Jee Li
6db94571d7
[Misc] Remove LoRA log (#15388) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-24 20:43:48 -07:00
Harry Mellor
97cfa65df7
Add pipeline parallel support to TransformersModel (#12832) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-03-25 10:41:45 +08:00
Woosuk Kwon
911c8eb000
[Minor][Spec Decode] Remove compiled_softmax (#15416) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-24 19:09:04 -07:00
Woosuk Kwon
ebcebeeb6b
[V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-24 17:16:46 -07:00