xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-17 15:45:59 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,817 Commits 155 Branches 89 Tags
Commit Graph

240 Commits

Author SHA1 Message Date
Jie Fu (傅杰)
70363bccfa
Fix syntaxWarning: invalid escape sequence '\s' (#16532) 
Signed-off-by: Jie Fu <jiefu@tencent.com>
 2025-04-12 14:39:42 +00:00
Harry Mellor
cd77382ac1
Improve configs - LoadConfig (#16422) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-11 20:27:27 +00:00
Lu Fang
7678fcd5b6
Fix the torch version parsing logic (#15857) 2025-04-10 07:37:47 -07:00
TJian
2976dc27e9
[Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and Triton Fused MoE bugs (#16198) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
 2025-04-08 19:12:34 -07:00
Nick Hill
651cf0fec1
[V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (#15906) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-04 12:56:43 -07:00
Wei Zeng
30d6a015e0
[Feature] specify model in config.yaml (#15798) 
Signed-off-by: weizeng <weizeng@roblox.com>
 2025-04-01 01:20:06 -07:00
youkaichao
555aa21905
[V1] Fully Transparent Implementation of CPU Offloading (#15354) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-31 20:22:34 +08:00
Reid
26df46ee59
[Misc] cli auto show default value (#15582) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-03-28 22:23:00 +00:00
Nick Hill
15dac210f0
[V1] AsyncLLM data parallel (#13923) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-27 16:14:41 -07:00
Hiroaki Sugiyama
8958217ad5
[Bugfix] Fix use_cascade_attention handling for Alibi-based models on vllm/v1 (#15211) 
Signed-off-by: h-sugi <h.sugi@ieee.org>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-27 22:29:29 +08:00
Bella kira
f4c98b4d4c
[Misc] Consolidate LRUCache implementations (#15481) 
Signed-off-by: Bella kira <2374035698@qq.com>
 2025-03-27 06:43:43 +00:00
ElizaWszola
9239bf718e
[Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
 2025-03-27 00:54:44 +00:00
marko
27df5199d9
Support SHA256 as hash function in prefix caching (#15297) 
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
 2025-03-26 11:11:28 -07:00
Gregory Shtrasberg
8279201ce6
[Build] Cython compilation support fix (#14296) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-03-24 23:37:54 +00:00
hijkzzz
0661cfef7a
Fix v1 supported oracle for worker-cls and worker-extension-cls (#15324) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-03-23 10:23:35 +08:00
Cyrus Leung
baec0d4de9
Revert "[Feature] specify model in config.yaml (#14855)" (#15293) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-21 08:30:23 -07:00
Russell Bryant
61e8c18350
[Misc] Add cProfile helpers (#15074) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-21 04:56:09 -07:00
Shanshan Shen
a989ca2bf6
[Bugfix] Add int8 torch dtype for KVCache (#15260) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-03-21 08:58:28 +00:00
Wei Zeng
0fa3970deb
[Feature] specify model in config.yaml (#14855) 
Signed-off-by: weizeng <weizeng@roblox.com>
 2025-03-21 00:26:03 -07:00
Cody Yu
5df2da5b97
[Misc] Better RayExecutor and multiprocessing compatibility (#14705) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-03-20 19:27:46 -07:00
Nick Hill
22d33baca2
[FrontEnd][Perf] merge_async_iterators fast-path for single-prompt requests (#15150) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-19 21:04:41 +00:00
Rémi Delacourt
61c6a5a796
[VLM] Merged multi-modal processor for Pixtral (#12211) 
Signed-off-by: remi <remi@mistral.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 06:28:27 -07:00
Cyrus Leung
3556a41434
[VLM] Limit multimodal input cache by memory (#14805) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 02:52:05 -07:00
Lucas Wilkinson
5952d8ab61
[Attention] Get rid of mla cache alignment (#14842) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-15 05:08:25 +00:00
Cyrus Leung
382403921f
[VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-03-13 02:23:12 -07:00
22quinn
eb8b5eb183
[V1] Support bad_words in sampler (#13376) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-03-08 14:50:26 -08:00
Aaron Pham
80e9afb5bc
[V1][Core] Support for Structured Outputs (#12388) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-03-07 07:19:11 -08:00
மனோஜ்குமார் பழனிச்சாமி
cc10281498
[Misc] Set default value of seed to None (#14274) 
Signed-off-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
 2025-03-07 10:40:01 +00:00
Tyler Michael Smith
72c62eae5f
[V1] EP/TP MoE + DP Attention (#13931) 2025-03-04 21:27:26 -08:00
Nick Hill
5db6b2c961
[V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (#13869) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-04 15:06:47 +00:00
Harry Mellor
cf069aa8aa
Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
cjackal
51010a1807
[Misc] set single whitespace between log sentences (#13771) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-02-25 10:26:12 +08:00
youkaichao
2382ad29d1
[ci] fix linter (#13701) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 20:28:59 +08:00
youkaichao
3e472d882a
[core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Cyrus Leung
7f6bae561c
[CI/Build] Fix pre-commit errors (#13696) 2025-02-22 00:31:26 -08:00
Robin
c6ed93860f
[Bugfix][API Server] Fix invalid usage of 'ge' and 'le' in port valid… (#13672) 2025-02-21 22:05:28 -08:00
Lucas Wilkinson
288cc6c234
[Attention] MLA with chunked prefill (#12639) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-02-21 15:30:12 -08:00
Gregory Shtrasberg
410886950a
[ROCm] Avoid using the default stream on ROCm (#13238) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-02-14 09:29:26 +08:00
Aoyu
2092a6fa7d
[V1][Core] Add worker_base for v1 worker (#12816) 
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-02-13 20:35:18 +08:00
Cyrus Leung
c9d3ecf016
[VLM] Merged multi-modal processor for Molmo (#12966) 2025-02-13 04:34:00 -08:00
MoonRide303
21f5d50fa5
[Bugfix] Do not use resource module on Windows (#12858) (#13029) 2025-02-11 08:21:18 -08:00
youkaichao
cf797aa856
[core] port pynvml into vllm codebase (#12963) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-09 15:00:00 +08:00
Lucas Wilkinson
75e94309e8
[Perf] Mem align KV caches for CUDA devices (MLA perf improvement) (#12676) 
Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Lucas Wilkinson <lcwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
 2025-02-04 18:22:24 -08:00
youkaichao
ad4a9dc817
[cuda] manually import the correct pynvml module (#12679) 
fixes problems like https://github.com/vllm-project/vllm/pull/12635 and
https://github.com/vllm-project/vllm/pull/12636 and
https://github.com/vllm-project/vllm/pull/12565

---------

Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-03 15:58:21 +08:00
Russell Bryant
e489ad7a21
[Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Harry Mellor
823ab79633
Update pre-commit hooks (#12475) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-27 17:23:08 -07:00
youkaichao
da02cb4b27
[core] further polish memory profiling (#12126) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-18 12:25:08 +08:00
youkaichao
87a0c076af
[core] allow callable in collective_rpc (#12151) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-17 20:47:01 +08:00
Shanshan Shen
9ddac56311
[Platform] move current_memory_usage() into platform (#11369) 
Signed-off-by: Shanshan Shen <467638484@qq.com>
 2025-01-15 03:38:25 +00:00
Chen Zhang
0f8cafe2d1
[Kernel] unified_attention for Attention.forward (#11967) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-01-13 19:28:53 +08:00