xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-23 07:55:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,734 Commits 155 Branches 89 Tags
Commit Graph

6734 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chenyaaang
9b70e2b4c1
[Misc][Tools][Benchmark] Publish script to auto tune server parameters (#17207) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-05-01 19:53:03 +00:00
Chen Xia
173daac19d
[Bug]change the position of cuda_graph_sizes in dataclasses (#17548) 
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
 2025-05-01 11:52:37 -07:00
sstamenk
04f2cfc894
Remove duplicate code from dbrx.py (#17550) 2025-05-01 11:51:58 -07:00
Juan Villamizar
811a6c0972
[ROCM] Add gfx950 to the custom attention archs (#16034) 
Signed-off-by: jpvillam <Juan.Villamizar@amd.com>
Signed-off-by: seungrokjung <seungrok.jung@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: seungrokjung <seungrok.jung@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-01 11:18:28 -07:00
Cyrus Leung
9b1769dd9a
[Bugfix] Fix lint error (#17547) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-01 11:12:19 -07:00
Chen Xia
61c299f81f
[Misc]add configurable cuda graph size (#17201) 
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-01 11:04:50 -07:00
Hongxia Yang
4acfa3354a
[ROCm] update installation guide to include build aiter from source instructions (#17542) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-05-01 11:01:28 -07:00
Isotr0py
88c8304104
[Model] Refactor Ovis2 to support original tokenizer (#17537) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-01 11:00:53 -07:00
Harry Mellor
6768ff4a22
Move the last arguments in arg_utils.py to be in their final groups (#17531) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-01 10:31:44 -07:00
Cyrus Leung
f2e7af9b86
[CI/Build] Remove awscli dependency (#17532) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-01 09:20:54 -07:00
Reid
7423cf0a9b
[Misc] refactor example - cpu_offload_lmcache (#17460) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-01 15:05:24 +00:00
Sage Moore
460a2b1100
[torch.compile] Add torch inductor pass for fusing silu_and_mul with subsequent scaled_fp8_quant operations (#10867) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-05-01 07:59:28 -07:00
Hongxia Yang
28566d73b3
[ROCm] remove unsupported archs from rocm triton flash-attention supported list (#17536) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
 2025-05-01 07:54:25 -07:00
Chauncey
98060b001d
[Feature][Frontend]: Deprecate --enable-reasoning (#17452) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-05-01 06:46:16 -07:00
TJian
f5a3c655b2
[FEAT] [ROCm]: Add Qwen/Qwen3-235B-A22B-FP8 TP4 triton fused moe config (#17535) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-01 06:37:17 -07:00
Reid
7169f87ad0
[doc] add streamlit integration (#17522) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-01 13:34:02 +00:00
Huy Do
b74d888c63
Fix more broken speculative decode tests (#17450) 
Signed-off-by: Huy Do <huydhn@gmail.com>
 2025-05-01 06:05:58 -07:00
TJian
2007d4d54f
[FEAT] [ROCm]: Add Qwen/Qwen3-30B-A3B-FP8 fused moe config for MI300X (#17530) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-01 06:03:13 -07:00
Cyrus Leung
48e925fab5
[Misc] Clean up test docstrings and names (#17521) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-01 05:19:32 -07:00
Cyrus Leung
1903c0b8a3
[Frontend] Show progress bar for adding requests (#17525) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-01 05:15:32 -07:00
Teruaki Ishizaki
86a1f67a3b
[Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to select a model (#17285) 
Signed-off-by: Teruaki Ishizaki <teruaki.ishizaki@ntt.com>
 2025-05-01 11:54:51 +00:00
Harry Mellor
a257d9bccc
Improve configs - ObservabilityConfig (#17453) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-01 03:52:05 -07:00
Chauncey
015069b017
[Misc] Optimize the Qwen3_ReasoningParser extract_reasoning_content (#17515) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-05-01 03:29:01 -07:00
Russell Bryant
fbefc8a78d
[Core] Enable IPv6 with vllm.utils.make_zmq_socket() (#16506) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-05-01 09:38:18 +00:00
Keyun Tong
26bc4bbcd8
Avoid overwriting vllm_compile_cache.py (#17418) 
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
 2025-05-01 07:30:57 +00:00
Lucas Wilkinson
3c3d767201
[BugFix] Fix mla cpu - missing 3 required positional arguments (#17494) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-01 14:36:52 +08:00
Noah Yoshida
13cf6b6236
[BugFix] fix speculative decoding memory leak when speculation is disabled (#15506) 
Signed-off-by: Noah Yoshida <noahcy117@gmail.com>
 2025-04-30 23:28:17 -07:00
Hongxia Yang
90d0a54c4d
[ROCm] Effort to reduce the number of environment variables in command line (#17229) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
 2025-04-30 23:27:06 -07:00
Russell Bryant
7a0a146c54
[Build] Require setuptools >= 77.0.3 for PEP 639 (#17389) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-30 23:25:36 -07:00
Alexei-V-Ivanov-AMD
7ab643e425
FIxing the AMD test failures caused by PR#16457 (#17511) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2025-04-30 23:23:07 -07:00
Cyrus Leung
afb4429b4f
[CI/Build] Reorganize models tests (#17459) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-30 23:03:08 -07:00
Michael Goin
aa4502e7f3
[CI][Bugfix] Fix failing V1 Test due to missing 'cache_salt' arg (#17500) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-30 21:03:30 -07:00
Michael Goin
17b4d85f63
[CI][TPU] Skip structured outputs+spec decode tests on TPU (#17510) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-30 20:36:20 -07:00
NaLan ZeYu
1144a8efe7
[Bugfix] Temporarily disable gptq_bitblas on ROCm (#17411) 
Signed-off-by: Yan Cangang <nalanzeyu@gmail.com>
 2025-04-30 19:51:45 -07:00
Gregory Shtrasberg
08fb5587b4
[Bugfix][ROCm] Fix import error on ROCm (#17495) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-04-30 19:51:42 -07:00
Siyuan Liu
dbc18e7816
[CI][TPU] Skip Multimodal test (#17488) 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
 2025-04-30 19:51:39 -07:00
Alex Brooks
02bd654846
[Misc] Rename Audios -> Audio in Qwen2audio Processing (#17507) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-04-30 19:51:36 -07:00
Rahul Tuli
200bbf92e8
Bump Compressed Tensors version to 0.9.4 (#17478) 
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-04-30 15:24:45 -07:00
Chen Zhang
81ecf425f0
[v1][Spec Decode] Make sliding window compatible with eagle prefix caching (#17398) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-04-30 18:25:53 +00:00
David Xia
42d9a2c4c7
doc: fix bug report Github template formatting (#17486) 
Signed-off-by: David Xia <david@davidxia.com>
 2025-04-30 10:03:20 -07:00
Reid
2ac74d098e
[doc] add install tips (#17373) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-30 17:02:41 +00:00
Gregory Shtrasberg
584f5fb4c6
[Bugfix][ROCm] Restrict ray version due to a breaking release (#17480) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-04-30 09:59:06 -07:00
zh Wang
d586ddc691
[BugFix] Fix authorization of openai_transcription_client.py (#17321) 
Signed-off-by: zh Wang <rekind133@outlook.com>
 2025-04-30 09:51:05 -07:00
Michael Goin
0b7e701dd4
[Docs] Update optimization.md doc (#17482) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-30 09:34:02 -07:00
Russell Bryant
947f2f5375
[V1] Allow turning off pickle fallback in vllm.v1.serial_utils (#17427) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-04-30 16:10:54 +00:00
Pete Savage
739e03b344
[Bugfix] Fixed mistral tokenizer path when pointing to file (#17457) 
Signed-off-by: Pete Savage <psavage@redhat.com>
 2025-04-30 08:08:37 -07:00
Aaron Pham
da4e7687b5
[Fix] Support passing args to logger (#17425) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-04-30 08:06:58 -07:00
Russell Bryant
39317cf42b
[Docs] Add command for running mypy tests from CI (#17475) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-30 08:06:09 -07:00
Chauncey
2990cee95b
[Feature] The Qwen3 reasoning parser supports guided decoding (#17466) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-04-30 07:48:21 -07:00
Alec
0be6d05b5e
[V1][Metrics] add support for kv event publishing (#16750) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-30 07:44:45 -07:00