xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-21 23:15:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,806 Commits 155 Branches 89 Tags
Commit Graph

7806 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mac Misiura
18bdcf4113
feat - add a new endpoint get_tokenizer_info to provide tokenizer/chat-template information (#20575) 
Signed-off-by: m-misiura <mmisiura@redhat.com>
 2025-07-16 21:52:14 +08:00
Cyrus Leung
1c3198b6c4
[Model] Consolidate pooler implementations (#20927) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-16 13:39:13 +00:00
Michael Yao
260127ea54
[Docs] Add intro and fix 1-2-3 list in frameworks/open-webui.md (#19199) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2025-07-16 06:11:38 -07:00
Seiji Eicher
d0dc4cfca4
Fix inadvertently silenced PP tests for mp, add DeepSeek V2/V3 model family to PP tests (#20831) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-07-16 00:14:49 -07:00
Lucas Wilkinson
d31a647124
[BugFix] Fix import error on non-blackwell machines (#21020) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-15 22:27:29 -07:00
Chengji Yao
85431bd9ad
[TPU] fix kv_cache_update kernel block size choosing logic (#21007) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-07-16 04:39:48 +00:00
zhiweiz
c11013db8b
[Meta] Llama4 EAGLE Support (#20591) 
Signed-off-by: qizixi <qizixi@meta.com>
Co-authored-by: qizixi <qizixi@meta.com>
 2025-07-15 21:14:15 -07:00
Peter Pan
1eb2b9c102
[CI] update typos config for CI pre-commit and fix some spells (#20919) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-15 21:12:40 -07:00
Maximilien de Bayser
6ebf313790
Avoid direct comparison of floating point numbers (#21002) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-07-15 21:12:14 -07:00
Patrick von Platen
cfbcb9ed87
[Voxtral] Add more tests (#21010) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-07-15 21:11:49 -07:00
Wentao Ye
76ddeff293
[Doc] Remove duplicate docstring (#21012) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-15 20:09:13 -07:00
Michael Goin
f46098335b
[Bugfix] Fix Mistral3 support on SM100/SM120 (#20998) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-15 20:08:41 -07:00
Chendi.Xue
e9534c7202
[CI][HPU] update for v0 deprecate by switching to VLLM_TARGET_DEVICE=empty (#21006) 
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
 2025-07-15 20:07:05 -07:00
Doug Smith
7976446015
Add Dockerfile argument for VLLM_USE_PRECOMPILED environment (#20943) 
Signed-off-by: dougbtv <dosmith@redhat.com>
 2025-07-15 19:53:57 -07:00
Ming Yang
fcb9f879c1
[Bugfix] Correct per_act_token in CompressedTensorsW8A8Fp8MoECutlassM… (#20937) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-07-15 19:53:42 -07:00
Ricardo Decal
3ed94f9d0a
[Docs] Enhance Anyscale documentation, add quickstart links for vLLM (#21018) 
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
 2025-07-15 19:46:56 -07:00
Reid
fa839565f2
[Misc] Refactor: Improve argument handling for conda command (#20481) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-15 19:43:19 -07:00
Brayden Zhong
75a99b98bf
[Chore] Remove outdated transformers check (#20989) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-07-15 19:42:40 -07:00
Chauncey
b5c3b68359
[Misc] bump xgrammar version to v0.1.21 (#20992) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-15 19:42:16 -07:00
Thomas Parnell
6cbc4d4bea
[Model] Add ModelConfig class for GraniteMoeHybrid to override default max_seq_len_to_capture (#20923) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-15 19:19:10 -07:00
Michael Goin
153c6f1e61
[Frontend] Remove print left in FrontendArgs.add_cli_args (#21004) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-15 19:18:41 -07:00
Chauncey
34cda778a0
[Frontend] OpenAI Responses API supports input image (#20975) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-15 18:59:36 -06:00
Elfie Guo
30800b01c2
[Nvidia] Integrate SM100 cudnn prefill API to MLA prefill (#20411) 
Signed-off-by: Elfie Guo <elfieg@nvidia.com>
Co-authored-by: Elfie Guo <eflieg@nvidia.com>
 2025-07-15 17:56:45 -07:00
Chen LI
10be209493
[Bug Fix] get_distributed_init_method should get the ip from get_ip i… (#20889) 
Signed-off-by: Chen Li <lcpingping@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-07-15 21:23:52 +00:00
Marko Rosenmueller
19c863068b
[Frontend] Support cache_salt in /v1/completions and /v1/responses (#20981) 
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
 2025-07-15 21:01:04 +00:00
Tuan, Hoang-Trong
f29fd8a7f8
[BugFix] fix 3 issues: (1) using metadata for causal-conv1d, (2) indexing overflow in v1 vLLM, and (3) init_states in v0 (#20838) 
Signed-off-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
 2025-07-15 16:08:26 -04:00
Gregory Shtrasberg
ed10f3cea1
[ROCm] warpSize is being made non constexpr in ROCm 7.0 (#20330) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-07-15 14:01:44 -04:00
Harry Mellor
b637e9dcb8
Add full serve CLI reference back to docs (#20978) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-15 17:42:30 +00:00
Harry Mellor
1e36c8687e
[Deprecation] Remove nullable_kvs (#20969) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-15 17:21:50 +00:00
Harry Mellor
5bac61362b
Configure Gemini (#20971) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-15 09:37:05 -07:00
Harry Mellor
313ae8c16a
[Deprecation] Remove everything scheduled for removal in v0.10.0 (#20979) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-15 15:57:53 +00:00
Cyrus Leung
c847e34b39
[CI/Build] Fix wrong path in Transformers Nightly Models Test (#20994) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-15 08:53:16 -07:00
Patrick von Platen
e7e3e6d263
Voxtral (#20970) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-07-15 07:35:30 -07:00
Christian Pinto
4ffd963fa0
[v1][core] Support for attention free models (#20811) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2025-07-15 14:20:01 +00:00
Harry Mellor
56fe4bedd6
[Deprecation] Remove TokenizerPoolConfig (#20968) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-15 14:00:50 +00:00
Rui Qiao
d91278181d
[doc] Add more details for Ray-based DP (#20948) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-15 05:37:12 -07:00
Li Wang
20149d84d9
[MISC] Add init files for python package (#20908) 
Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-07-15 12:16:33 +00:00
Thomas Parnell
3534c39a20
[V1] [Hybrid] Refactor mamba state shape calculation; enable V1 via cli (#20840) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-15 04:04:35 -07:00
Yifei Teng
c586b55667
[TPU] Optimize kv cache update kernel (#20415) 
Signed-off-by: Yifei Teng <tengyifei88@gmail.com>
 2025-07-15 03:56:43 -07:00
Ricardo Decal
33d560001e
[Docs] Improve documentation for ray cluster launcher helper script (#20602) 
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
 2025-07-15 03:55:45 -07:00
kourosh hakhamaneshi
f148c44c6a
[frontend] Refactor CLI Args for a better modular integration (#20206) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
 2025-07-15 02:23:42 -07:00
Ricardo Decal
235bfd5dfe
[Docs] Improve documentation for RLHF example (#20598) 
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
 2025-07-15 01:54:10 -07:00
Reid
68d28e37b0
[frontend] Add --help=page option for paginated help output (#20961) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-15 00:42:00 -07:00
Ilya Markov
37a7d5d74a
[Misc] Refactor AllReduceFusionPass. Remove parameter (#20918) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-07-15 06:57:40 +00:00
Woosuk Kwon
d4d309409f
Implement Async Scheduling (#19970) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-14 23:01:46 -07:00
Jennifer He
85bd6599e4
[Model] Add AutoWeightsLoader support for BERT, RoBERTa (#20534) 
Signed-off-by: Jennifer He <islandhe@gmail.com>
Signed-off-by: <islandhe@gmail.com>
Signed-off-by: Jen H <islandhe@gmail.com>
 2025-07-15 13:34:24 +08:00
Boyuan Feng
91b3d190ae
[cold start] replace VLLM_COMPILE_DEPYF with debug_dump_dir (#20940) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-07-15 13:02:17 +08:00
Isotr0py
fc017915f5
[Doc] Clearer mistral3 and pixtral model support description (#20926) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-07-14 21:56:53 -07:00
Pavani Majety
9ad0a4588b
[Bugfix] Switch bailout logic for kv-cache-dtype with SM100 Flashinfer (#20934) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-07-15 03:27:50 +00:00
Ruheena Suhani Shaik
016b8d1b7f
Enabled BnB NF4 inference on Gaudi (#20172) 
Signed-off-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
 2025-07-14 20:26:08 -07:00