xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-16 12:55:02 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,134 Commits 155 Branches 89 Tags
Commit Graph

6134 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Richard Barnes
d6da8a8ff2
[Bugfix] Fix numel() downcast in fused_layernorm_dynamic_per_token_quant.cu (#17316) 2025-04-28 19:23:18 -07:00
Lucia Fang
b4ac4fa04d
[model] make llama4 compatible with pure dense layers (#17315) 
Signed-off-by: Lucia Fang <fanglu@fb.com>
 2025-04-29 10:22:22 +08:00
Ekagra Ranjan
e136000595
[V1][Spec Decode] Make Eagle model arch config driven (#17323) 2025-04-29 10:22:02 +08:00
Michał Moskal
86d9fc29cb
implement Structural Tag with Guidance backend (#17333) 
Signed-off-by: Michal Moskal <michal@moskal.me>
 2025-04-29 02:21:32 +00:00
Cyrus Leung
506475de5f
[Optim] Compute multimodal hash only once per item (#17314) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-29 09:40:35 +08:00
Ekagra Ranjan
cfe4532093
[Benchmark] Add single turn MTBench to Serving Bench (#17202) 2025-04-28 16:46:15 -07:00
Michael Goin
8fc88d63f1
[Model] Add tuned triton fused_moe configs for Qwen3Moe (#17328) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-28 15:20:24 -07:00
Alex Wu
6e74fd4945
Support loading transformers models with named parameters (#16868) 
Signed-off-by: Alex <alexwu@character.ai>
 2025-04-28 23:15:58 +01:00
Simon Mo
dcbac4cb4b
[Model] Qwen3 Dense FP8 Compat Fixes (#17318) 
Signed-off-by: simon-mo <xmo@berkeley.edu>
 2025-04-28 14:12:01 -07:00
Charlie Fu
ed2462030f
[Bugfix] Fix moe weight losing all extra attrs after process_weights_after_loading. (#16854) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-04-28 21:05:07 +00:00
Lucas Wilkinson
cc5befbced
[BugFix] Fix cascade attention - RuntimeError: scheduler_metadata must have shape (metadata_size) (#17283) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-04-28 13:55:50 -07:00
Aaron Pham
2c89cd96a8
[Chore] cleanup license indicators in light of SPDX (#17259) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-04-28 19:43:52 +00:00
Russell Bryant
a0304dc504
[Security] Don't bind tcp zmq socket to all interfaces (#17197) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-28 10:08:20 -07:00
Harry Mellor
c7941cca18
Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:55:31 +00:00
Harry Mellor
b6dd32aa07
Make name of compressed-tensors quant method consistent across vLLM (#17255) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:28:13 +00:00
Harry Mellor
f94886946e
Improve conversion from dataclass configs to argparse arguments (#17303) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:22:12 +00:00
Russell Bryant
72dfe4c74f
[Docs] Add a security guide (#17230) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-04-28 15:12:17 +00:00
Cyrus Leung
8b464d9660
[Misc] Clean up Qwen2.5-Omni code (#17301) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 06:20:45 -07:00
Nicolò Lucchesi
889ebb2638
[Misc] Minor typo/grammar in platforms/interface.py (#17307) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-04-28 05:45:42 -07:00
Reid
3ad986c28b
[doc] update wrong model id (#17287) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-28 04:20:51 -07:00
Cyrus Leung
344e193b7d
[Bugfix] Add missing get_language_model to new MLLMs (#17300) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 04:09:57 -07:00
Harry Mellor
fb1c933ade
Add missing class docstring for PromptAdapterConfig (#17302) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 04:06:59 -07:00
idouba
72c5b97231
Update tpu_worker.py 's typo (#17288) 2025-04-28 04:01:15 -07:00
Alex Brooks
fa93cd9f60
[Model] Add Granite Speech Support (#16246) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-04-28 10:05:00 +00:00
Cyrus Leung
aec9674dbe
[Core] Remove legacy input mapper/processor from V0 (#15686) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 15:38:48 +08:00
Wanrui Dai
7fcc4223dc
[Minor][Models] Pass partial_rotary_factor parameter to rope (#17266) 
Signed-off-by: evian <eviantai@u.nus.edu>
Co-authored-by: evian <eviantai@u.nus.edu>
 2025-04-28 04:28:59 +00:00
Nick Hill
8262a3e23b
[Misc] Validate stop_token_ids contents (#17268) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-28 03:54:05 +00:00
Reid
f211331c48
[Doc] small fix (#17277) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-28 03:53:35 +00:00
Kuntai Du
9053d0b134
[Doc] Fix wrong github link in LMCache examples (#17274) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-04-28 03:09:11 +00:00
Michael Goin
cb3f2d8d10
[Bugfix] Fix Mistral3 spatial merge error (#17270) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-27 19:40:05 -07:00
TherLF
c12df53b60
[Bugfix] Fix cutlass dispatch for fp8/int8 to properly invoke M<=16 c… (#16751) 
Signed-off-by: Ther-LF <2639852836@qq.com>
 2025-04-27 19:38:42 -07:00
Lennart K. M. Schulz
d1aeea7553
[Bugfix] Fix missing ARG in Dockerfile for arm64 platforms (#17261) 
Signed-off-by: lkm-schulz <44176356+lkm-schulz@users.noreply.github.com>
 2025-04-27 19:38:14 -07:00
Lucas Wilkinson
d8bccde686
[BugFix] Fix vllm_flash_attn install issues (#17267) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-04-27 17:27:56 -07:00
Lily Liu
20e489eaa1
[V1][Spec Decode] Make eagle compatible with prefix caching. (#17137) 
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-04-27 09:29:43 -07:00
Cyrus Leung
4213475ec7
[Metrics] Fix minor inconsistencies in bucket progression (#17262) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-27 16:19:39 +00:00
Reid
d92879baf6
[doc] Add feature status legend (#17257) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-27 08:17:02 -07:00
cascade
690fe019f0
[Feature] support sequence parallelism using compilation pass (#16155) 
Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-04-27 06:29:35 -07:00
Kaixi Hou
ed7a29d9f8
[NVIDIA] Support Cutlass MLA for Blackwell GPUs (#16032) 
Signed-off-by: kaixih <kaixih@nvidia.com>
 2025-04-27 06:29:21 -07:00
Alex Brooks
756848e79e
[Bugfix] Fix Lora Name Parsing (#17196) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-27 20:33:09 +08:00
Flex Wang
18445edd0f
[Misc] Change buckets of histogram_iteration_tokens to [1, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8096] to represent number of tokens (#17033) 
Signed-off-by: sfc-gh-zhwang <flex.wang@snowflake.com>
 2025-04-27 12:30:53 +00:00
Jade Zheng
30215ca61f
[MISC] Use string annotation types for class definitions (#17244) 
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
 2025-04-27 08:39:57 +00:00
Chen Zhang
838cedade7
[Bugfix] Get a specific type of layer from forward context (#17222) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-04-27 00:58:05 -07:00
Jee Jee Li
4283a28c2f
[Bugfix] Fix QWen2 VL multimodal mapping (#17240) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-27 05:53:23 +00:00
Cyrus Leung
93a126fbc7
[Misc] Make cached tokenizer pickle-compatible (#17048) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-27 13:05:00 +08:00
rasmith
8e4b351a0c
[Kernel][Triton][FP8] Adding fp8 and variable length sequence support to Triton FAv2 kernel (#12591) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-04-27 00:35:08 +00:00
Happy
9869453c42
Update test_flash_attn.py (#17102) 
Signed-off-by: ShuaibinLi <lishuaibin@live.cn>
 2025-04-26 22:17:35 +00:00
Reid
3642c59aa8
[CI/Build] remove -t for run-lm-eval-gsm-hf-baseline.sh (#16271) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-26 18:25:05 +00:00
Woosuk Kwon
43eea2953b
[Minor] Fix lint error in main branch (#17233) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-26 11:10:14 -07:00
Kero Liang
de7eb10ce4
[Bugfix] Fix Qwen2.5-Omni M-RoPE position ids generation (#16878) 
Signed-off-by: imkero <kerorek@outlook.com>
 2025-04-26 10:41:35 -07:00
Ning Xie
fd11a325b8
[MISC] rename interval to max_recent_requests (#14285) 2025-04-26 16:59:18 +00:00