xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-06-03 15:51:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,894 Commits 155 Branches 89 Tags
Commit Graph

122 Commits

Author SHA1 Message Date
Isotr0py
71b25b0d48
[V0 deprecation] Clean up V0 fallback in compilation config (#25675) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-25 17:29:51 +00:00
Cyrus Leung
0ea80c87d9
[Model] Define merge_by_field_config MM interface (#25676) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 17:13:07 +00:00
wang.yuqi
7f570f1caa
[V0 deprecation] Remove unreachable model_config.supported_tasks (#25642) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-25 11:26:31 +00:00
Tyler Michael Smith
1260180c67
Revert "[Performance] Move apply_w8a8_block_fp8_linear to an op class… (#25607) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-25 08:05:21 +00:00
XuruiYang
845adb3ec6
[Model] Add LongCat-Flash (#23991) 
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
 2025-09-24 21:53:40 -07:00
Harry Mellor
8938774c79
Move DeviceConfig, ObservabilityConfig, SpeechToTextConfig to their own files (#25564) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-24 13:59:05 +00:00
Woosuk Kwon
2e19a848d4
[V0 Deprecation] Remove max_seq_len_to_capture (#25543) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-24 01:51:39 -07:00
ahao-anyscale
c8bde93367
[BUG] Allows for RunAI Streamer and Torch.compile cache to be used together (#24922) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2025-09-23 18:13:32 -06:00
jiahanc
d5944d5146
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2025-09-23 15:44:35 -04:00
Michael Goin
24fab45d96
[Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 15:29:26 -04:00
ElizaWszola
63400259d0
[Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-09-23 12:03:10 -07:00
Lucas Wilkinson
cc1dc7ed6d
[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-23 16:02:10 +00:00
Luka Govedič
d5e0fca264
[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-22 12:30:05 -07:00
Lucia Fang
922979bfcc
[DP] support torchrun external launcher with Data Parallelism (#24899) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-22 12:06:05 -07:00
Yizhou
b6f01bd9a7
refactor: abstract graph mode support into platform interface (#25161) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2025-09-22 10:22:29 +00:00
Eldar Kurtić
21467f9a1c
Enable Eagle3 speculative decoding for GPT-OSS model (#25246) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
 2025-09-22 08:50:39 +00:00
Woosuk Kwon
bc6e542d9f
Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Rahul Tuli
c438b2951c
feat: Enable engine-level arguments with speculators models (#25250) 
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-09-21 11:04:45 -06:00
Woosuk Kwon
0ff8ebb2d7
[V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 08:52:32 -07:00
Cyrus Leung
c60e6137f0
[Optimization] Avoid repeated model architecture conversion for pooling models (#25261) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 13:30:22 +08:00
Boyuan Feng
8945b001db
[torch.compile] CUDAGraph Inductor partition integration (#24281) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-20 01:02:15 +00:00
Zhiyu
431535b522
Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-19 22:40:33 +00:00
Cyrus Leung
6c117cff7d
[Frontend] Pass API server count to each process (#23717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 01:15:19 +08:00
Harry Mellor
aed16879a9
Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Harry Mellor
058525b997
Move PoolerConfig from config/__init__.py to config/pooler.py (#25181) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 11:02:55 +00:00
Wentao Ye
d2a30a2d93
[Bug] Fix torch Compilation Cache Hit Error (#25093) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-18 12:38:37 -07:00
qizixi
c4cb0af98a
[spec decode] Fix MTP inference path for MiMo-7B model (#25136) 
Signed-off-by: zixi-qi <qizixi@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-18 09:12:19 -07:00
Harry Mellor
3ed1ec4af2
Fix validate-config pre-commit check (#25157) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 12:06:28 +00:00
Harry Mellor
8ed039d527
Move StructuredOutputsConfig from config/__init__.py to config/structured_outputs.py (#25153) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 11:24:27 +00:00
Aaron Pham
29283e8976
[Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
rongfu.leng
350c94deb3
[Bugfix] when use s3 model cannot use default load_format (#24435) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-09-18 07:47:43 +00:00
Benjamin Chislett
b7433ca1a4
[Spec Decode] Efficient padded speculation (#24539) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-09-18 01:07:24 -04:00
ahao-anyscale
f20c3b0951
[BUG] Exclude .pth files when pulling remote files (#25092) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2025-09-17 20:42:09 +00:00
Michael Goin
67532a1a68
[UX] Remove "quantization is not fully optimized yet" log (#25012) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-16 20:57:51 -07:00
Sage Moore
567939953b
[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
Harry Mellor
0faf3cc3e8
Move SpeculativeConfig from config/__init__.py to config/speculative.py (#24904) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-16 12:51:35 +01:00
Chen Bruce
7ea5c73ad7
[Feat][EPLB] A novel static EPLB placement strategy for MoE models. (#23745) 
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
Signed-off-by: Chen Bruce <bruceszchen@tencent.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Chen Bruce <cszwwdz@vip.qq.com>
Co-authored-by: lemon412 <lemon412@foxmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-16 10:55:16 +00:00
Woosuk Kwon
759ef49b15
Remove V0 Encoder-Decoder Support (#24907) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-15 21:17:14 -07:00
Harry Mellor
c4afdb69cc
Move MultiModalConfig from config/__init__.py to config/multimodal.py (#24659) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 17:43:16 +00:00
dongluw
a5b84f1cbf
[Core] Shared memory based object store for Multimodal data caching and IPC (#20452) 
Signed-off-by: donglu <donglu@cohere.com>
 2025-09-12 07:54:17 -07:00
wang.yuqi
d21a36f5f9
[CI] Add ci_envs for convenient local testing (#24630) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-12 08:52:25 +00:00
Zazzle516
7a30fa8708
[Doc] Clarify cudagraph capture size logic and default behavior in scheduler (#18698) 
Signed-off-by: Zazzle516 <2405677060@qq.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 23:18:09 +00:00
Mengqing Cao
4f6593b058
[HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-09-11 21:47:58 +08:00
Boyuan Feng
94e6b2d55f
Allow users to specify kv cache memory size (#21489) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 13:41:07 +00:00
Harry Mellor
5f5271f1ee
Move LoRAConfig from config/__init__.py to config/lora.py (#24644) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 11:01:38 +00:00
Harry Mellor
d6249d0699
Fix typing for safetensors_load_strategy (#24641) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 10:41:39 +00:00
wang.yuqi
25bb9e8c65
[CI Failure] fix models/language/pooling/test_auto_prefix_cache_support.py (#24636) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-11 03:31:23 -07:00
Tao He
e93f4cc9e3
Add the support for the qwen3 next model (a hybrid attention model). (#24526) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 15:32:09 +08:00
Didier Durand
e2b1f863aa
[Doc]: fixing doc typos (#24635) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-10 23:19:28 -07:00
shengshiqi-google
41329a0ff9
[Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469) 
Signed-off-by: Shiqi Sheng <shengshiqi@google.com>
Signed-off-by: shengshiqi-google <160179165+shengshiqi-google@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-10 23:10:01 -07:00