xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 20:15:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,300 Commits 155 Branches 89 Tags
Commit Graph

3179 Commits

Author SHA1 Message Date
Julien Denize
c6187f55f7
Refactor MistralTokenizer (#26358) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-10-09 22:48:58 +00:00
elvischenv
44f633dba1
[Flashinfer][gpt-oss] Support FP8-qkv Flashinfer TRTLLM Sinks Attention (#25674) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-10-09 16:13:39 -04:00
Jiangyun Zhu
5728da11ea
Revert #26113 "[Frontend] CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops" (#26472) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-10-09 05:43:55 -07:00
Wenzheng Bi
ec10fd0abc
[Bugfix] Move current_platform import to avoid python import cache. (#16601) 
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
 2025-10-09 10:46:19 +00:00
Cyrus Leung
4bdf7ac593
[Bugfix] Fix SHM cache initialization (#26427) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 02:48:04 -07:00
Cyrus Leung
dc7976dd9f
[Misc] Upgrade more code to Python 3.10 (#26463) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 10:43:53 +01:00
Jerry Zhang
a83ff278d6
[torchao] Add support for ModuleFqnToConfig using regex (#26001) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
 2025-10-09 08:32:32 +00:00
Rahul Tuli
cf4cd6c24f
Add: Support for multiple hidden layers in Eagle3 (#26164) 
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
 2025-10-09 07:30:50 +00:00
elvischenv
5e49c3e777
Bump Flashinfer to v0.4.0 (#26326) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-10-08 23:58:44 -07:00
Cyrus Leung
0f29dca988
[CI/Build] Fix model nightly tests (#26466) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-08 23:44:16 -07:00
Zhiyuan Li
d24cf322e1
[Hybrid]: Decouple Kernel Block Size from KV Page Size (#24486) 
Signed-off-by: lizhiyuan <uniartisan2017@gmail.com>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-08 23:43:39 -07:00
Qier Li
d17f0fbf30
[Core][KVConnector] Propagate all tokens on resumed preemptions (#24926) 
Signed-off-by: Qier Li <kevin44036@gmail.com>
Co-authored-by: Qier Li <qier@fb.com>
 2025-10-09 14:43:31 +08:00
bnellnm
da364615fc
[Kernels] Modular kernel refactor (#24812) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-10-08 17:51:52 -04:00
Elaine Zhao
f08919b7d1
[Bugfix] Respect min_tokens in scheduler stop check (#26317) 
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
 2025-10-08 14:08:24 -07:00
Matthew Bonanni
76879cc160
[Attention] Implement universal BACKEND_MAP (#25900) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-08 12:00:25 -07:00
Wentao Ye
4ba8875749
[Bug] Fix Test in Batch Invariant (#26128) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-08 10:13:47 -07:00
Wentao Ye
9fb3ae4e6f
[Bug] Fix DeepGEMM Attention Test (#26423) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-08 12:23:41 -04:00
Harry Mellor
2f99f2f506
Tidy vllm/config/__init__.py to only add classes and functions (#26405) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 07:10:00 -07:00
wang.yuqi
e39dc46f8f
[CI] Pooling models mteb test disable enforce_eager (#26408) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-08 12:15:36 +00:00
liangel-02
b32260ab85
[torchao] safetensors integration (#25969) 
Signed-off-by: Angel Li <liangel@meta.com>
 2025-10-07 20:12:35 -06:00
Lucas Wilkinson
f80e7866c0
[Misc] Clean up cruft from previous FlashMLA sparse implementation (#26125) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-10-08 10:09:34 +08:00
Thomas Parnell
31a4b3e6c4
Revert #24446 and #26168 (#26332) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-07 16:38:19 -06:00
Sergei Skvortsov
6ebaf43ee4
[V1] Logit processors for rejection sampler (#19482) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Signed-off-by: Sergei Skvortsov <sergeyskv@nebius.com>
Signed-off-by: Sergei Skvortsov <yvorott@gmail.com>
Co-authored-by: Sergei Skvortsov <sergeyskv@nebius.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-10-07 13:02:49 -07:00
Morrison Turnansky
0c824fc46f
[Frontend] CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops (#26113) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2025-10-07 12:53:43 -07:00
Michael Goin
30a3e5af69
[CI] Add Qwen3 MoE NVFP4 to Blackwell lm-eval (#26316) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-07 10:36:15 -07:00
fxmarty-amd
a38c1bfe09
[ci] Rename test_mxfp4_moe.py to test_ocp_mx_moe.py (#26364) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2025-10-07 09:52:24 -07:00
Paul Pak
320feae6f5
[Model] Lfm2Moe (#26344) 
Signed-off-by: Paul Pak <paulpak58@gmail.com>
 2025-10-07 16:03:05 +00:00
Cyrus Leung
1e4ecca1d0
[V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:42:31 +00:00
Cyrus Leung
c0a7b89d8e
[Misc] Move LRUCache into its own file (#26342) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:08:40 +00:00
antrec
6f59beaf0b
[Model] Add support for ModernBertForTokenClassification (#26340) 
Signed-off-by: Antoine Recanati Le Goat <antoine.recanati@sancare.fr>
Signed-off-by: antrec <antoine.recanati@gmail.com>
Co-authored-by: Antoine Recanati Le Goat <antoine.recanati@sancare.fr>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-07 14:29:19 +00:00
fxmarty-amd
41f1cf38f2
[Feature][OCP MX] Support mxfp6 and mixed mxfp6-mxfp4 (#21166) 2025-10-07 09:35:26 -04:00
Daniel Cámpora
e1098ced95
Add topk logits torch op for DS3.2. (#25945) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Daniel Cámpora <961215+dcampora@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-10-07 10:07:32 +00:00
Grant Holmes (Ren)
d100d78eb3
Optimize KV cache distribution for asymmetric pipeline parallelism (#25164) 
Signed-off-by: gholmes829 <g.holmes429@gmail.com>
 2025-10-07 09:20:30 +00:00
Andrew Xia
185d8ed44f
[responsesAPI][bugfix] serialize harmony messages (#26185) 
Signed-off-by: Andrew Xia <axia@meta.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-07 07:07:53 +00:00
Michael Goin
c6873c4e6d
[UX] Support nested dicts in hf_overrides (#25727) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-07 11:19:16 +08:00
Sage Moore
2111b4643c
[Core] Simplify the Dp padding/should ubatch coordination logic (#25768) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-10-07 01:57:49 +00:00
Gregory Shtrasberg
f231e5bc21
[ROCm] Split AITER unified attention into its own backend (#25507) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-10-06 22:49:23 +00:00
Raushan Turganbay
7cd95dc8a3
[Bugfix] Fix gemma3 with transformers backend (#23178) 
Signed-off-by: raushan <raushan@huggingface.co>
Signed-off-by: Raushan Turganbay <raushan@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 18:42:32 +00:00
Crefeda Rodrigues
c02058c222
Add bias handling to CPUFusedMOE kernel (#26289) 
Signed-off-by: Crefeda Rodrigues <crefeda.rodrigues@arm.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Crefeda Rodrigues <65665931+cfRod@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Sharif Inamdar <Sharif.Inamdar@arm.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-10-06 18:39:10 +00:00
Michael Goin
20db99cc69
[CI Bugfix] Make sure TRTLLM attention is available in test_blackwell_moe (#26188) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-06 13:50:11 -04:00
Yannick Schnider
6431be808f
[Tests] conftest: Extending VllmRunner and HfRunner to accept token_ids as input (#26295) 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-06 17:19:34 +00:00
Matthew Bonanni
4727a8afa7
[Attention] Remove unused reorder_batch method (#24463) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-06 13:13:39 -04:00
Raushan Turganbay
ab5e7d93f4
[Bugfix] Fix mrope in Transformers Backend (#26087) 
Signed-off-by: raushan <raushan@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 11:40:50 +00:00
Cyrus Leung
391612e78b
[Frontend] Consolidate tokenizer init code (#26276) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-06 09:34:52 +00:00
Harry Mellor
6c04638214
Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
wuhang
91ac7f764d
[CI][gpt-oss] Enable python tool tests in CI (#24315) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2025-10-06 04:20:06 +00:00
Thomas Parnell
d3c84297c3
[CI] Add comment about the single cudagraph capture size that is used (#26252) 2025-10-06 02:35:37 +00:00
Michael Goin
60bc25e74c
[CI] Add Blackwell LM Eval Small Models test to nightly (#26052) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-05 14:59:50 -06:00
Harry Mellor
b893d661b1
Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Jiangyun Zhu
9c3c21c519
[CI] fix mamba kernel test (#26250) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-10-05 18:26:59 +00:00