xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 05:04:58 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,450 Commits 155 Branches 89 Tags
Commit Graph

3575 Commits

Author SHA1 Message Date
Nicolò Lucchesi
6f1e7f7226
[DisaggEverything] Tokens in<>out /generate endpoint (#24261) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-14 09:58:01 -07:00
Harry Mellor
5f3cd7f7f2
[Docs] Update the name of Transformers backend -> Transformers modeling backend (#28725) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-14 16:34:14 +00:00
dongbo910220
c934caee88
[Fix] improve aspect ratio in dummy image generation and add common VLM tests for PaddleOCR-VL (#28711) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-11-14 16:07:20 +00:00
Cyrus Leung
511a6b611d
[Config] Clean up SchedulerConfig initialization (#28665) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-14 22:41:02 +08:00
Nicolò Lucchesi
96b23b8e3b
[Bugfix][Nixl] Fix kernel physical<>logical block_size issue (#28677) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-11-14 22:40:05 +08:00
Lucas Wilkinson
db56a59970
[BugFix] Fix FA3 IMA with FULL_AND_PIECEWISE and cascade attention (default) (#28702) 2025-11-14 12:19:22 +00:00
Yong Hoon Shin
9324e10275
Fix KV sharing fast prefill with cudagraph enabled (#28537) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-14 11:53:42 +00:00
Jingchun Gao
4516d44b7f
[DCP] Support Decode Context Parallel (DCP) for GQA with Flashinfer (#25438) 
Signed-off-by: gaojc <1055866782@qq.com>
Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com>
Signed-off-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: gaojingchun (A) <g00955623@china.huawei.com>
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
 2025-11-14 11:24:10 +00:00
Srreyansh Sethi
360bd8762f
[Frontend] Added chat-style multimodal support to /classify. (#27516) 
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: vnadathur <glvikramn@gmail.com>
Co-authored-by: wang.yuqi <noooop@126.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-11-14 11:03:55 +00:00
Xing Liu
8cfbe89b93
[Misc] fix comment in test_envs (#28529) 
Signed-off-by: Xing Liu <xingliu14@gmail.com>
 2025-11-14 09:32:46 +00:00
Boyuan Feng
fd75d3e8c0
[Minor] avoid register new custom and just import silly_attn (#28578) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-11-14 09:32:31 +00:00
Michael Goin
c9a3a02149
Add output token counting to gsm8k eval (#28594) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-14 09:32:03 +00:00
rasmith
93103575ce
[BugFix][CI/Build][ROCM] Fix import error and apply assert in appropriate case in test_struct_output_generate (#28311) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-11-13 22:41:29 -08:00
Hank_
4d5943bda6
[quantization][config] enable override existing quant_config (#28510) 
Signed-off-by: Hank <hcc.mayday@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-11-14 01:24:10 +00:00
Mark McLoughlin
6e25b1cddf
[KV Connector] Test async mode in scheduler tests (#28550) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-11-13 18:30:59 -05:00
Yanan Cao
262d263f6c
[Bugfix] Eliminate tuple inputs to submodules in graph partitioning (#28533) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2025-11-13 15:09:05 -05:00
elvischenv
5d6ce2b960
[Perf] Support stream interval for reducing host overhead (#27869) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-13 13:21:25 -05:00
Yannick Schnider
119c4927b3
[Bugfix] Fix validate model input for decoder models (#27099) 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-13 10:18:47 -08:00
Varun Sundar Rabindranath
fe1cd7704d
[Performance][B200] silu_mul_quant: pack scales in int32 (#28358) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-11-13 10:16:55 -08:00
Roger Wang
d3387750f1
[Misc] Turn off encoder torch compile by default (#28634) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-13 08:38:08 -08:00
tjandy98
4504e8029b
[Bugfix] Prevent crash on empty grammar string (#28210) 
Signed-off-by: tjandy98 <3953059+tjandy98@users.noreply.github.com>
 2025-11-13 06:42:29 +00:00
Andrew Xia
1a0b157a2e
[Frontend][responsesAPI][1/n] convert responses API tool input to chat completions tool format (#28231) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-11-13 04:47:22 +00:00
Andrew Xia
7c38ed0f1c
[Frontend] split append tool output (#28333) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-11-13 04:03:23 +00:00
Jialin Ouyang
a1d3866dda
[n-gen] DO NOT repeatedly return finished child requests (#28591) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-11-13 03:36:07 +00:00
wangxiyuan
2dacd57394
[platform] Move get_cu_count to utils (#27005) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-13 08:48:47 +08:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟
4ca5cd5740
[Core][AMD] Migrate fully transparent sleep mode to ROCm platform (#12695) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
 2025-11-12 15:24:12 -08:00
Andy Lo
58ce8d12b7
[BugFix] Priority scheduling and spec tokens preemption (#28558) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2025-11-12 20:29:21 +00:00
Harry Mellor
a39dd7bb06
[CI] Skip "Multi-Modal Models Test (Extended) 3" test that's broken in current Transformers (#28559) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-12 19:38:13 +00:00
alberto
bac904565f
Implement ARC KV cache eviction policy for CPU offloader (#27039) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: alberto <aperdomo@redhat.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2025-11-12 09:51:39 -08:00
Harry Mellor
a742134cc5
Remove deprecated fields from CompilationConfig (#27593) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-12 16:10:28 +00:00
wangxiyuan
10138c92a5
[V0 deprecation] Deprecate use_v1 parameter (#28112) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-12 14:03:52 +00:00
TJian
edb59a9470
[ROCm] [Bugfix] Fix fused_qknorm_rope_kernel rocm compatibility (#28500) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-11-12 05:01:14 -08:00
Chenguang Zheng
4ccffe561f
[Core] Encoder separation for Encode-Prefill-Decode Disaggregation (#25233) 
Signed-off-by: n00909098 <nguyen.kha.long@huawei.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: herotai214 <herotai214@gmail.com>
Signed-off-by: Khuong Le <khuong.le.manh@huawei.com>
Signed-off-by: Khuong Le <lemanhkhuong2611@gmail.com>
Co-authored-by: n00909098 <nguyen.kha.long@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: herotai214 <herotai214@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Khuong Le <khuong.le.manh@huawei.com>
Co-authored-by: Khuong Le <lemanhkhuong2611@gmail.com>
 2025-11-11 18:58:33 -08:00
Andreas Karatzas
9f0247cfa4
VLLM_USE_TRITON_FLASH_ATTN V0 variable deprecation (#27611) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2025-11-11 18:34:36 -08:00
Li, Jiang
7f829be7d3
[CPU] Refactor CPU attention backend (#27954) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-12 09:43:06 +08:00
wangxiyuan
e1710393c4
[[V0 deprecation]]Remove VLLM_USE_V1 env (#28204) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-11 18:22:16 -07:00
Yanan Cao
48c879369f
[Frontend] Change CompilationMode to a proper Enum (#28165) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2025-11-11 19:46:18 -05:00
Adrian Abeyta
d23539549a
Use FLASHINFER MLA backend when testing fp8_kv_scale_compile (#28491) 
Signed-off-by: adabeyta <aabeyta@redhat.com>
 2025-11-12 00:34:58 +00:00
Jialin Ouyang
4228be7959
[Perf] Use np.ndarray instead of list[list[int]] to reduce GC overhead (#28245) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-11-11 10:28:47 -08:00
Zhewen Li
e553424919
[CI/Build] Refactor Attention backend for test_prefix_prefill from xformers to SDPA (#28424) 
Signed-off-by: zhewenli <zhewenli@meta.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-12 01:09:47 +08:00
xuebwang-amd
5a1271d83a
[Quantization] fix attention quantization of gpt_oss model (#27334) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2025-11-11 12:06:00 -05:00
xuebwang-amd
05576df85c
[ROCm][Quantization] extend AMD Quark to support mixed-precision quantized model (#24239) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Co-authored-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-11 12:05:22 -05:00
zhrrr
68c09efc37
[Kernel][Perf] fuse QK Norm and RoPE into one cuda kernel for Qwen Model (#27165) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2025-11-11 12:00:31 -05:00
Nicolò Lucchesi
a7ef3eb0cd
[NIXL] Generalize block-first backend layouts (FlashInfer-like) (#28282) 2025-11-11 16:57:43 +00:00
jvlunteren
533b018f72
[BugFix] Fix Failing Ruff Check (#28469) 
Signed-off-by: Jan van Lunteren <jvl@zurich.ibm.com>
 2025-11-11 06:41:43 -08:00
bnellnm
a1448b4b69
[Kernels] Split up fused_moe/layer.py, isolate more modular kernel code (#28064) 2025-11-11 07:29:02 -07:00
Matthew Bonanni
b30dfa03c5
[Attention] Refactor CUDA attention backend selection logic (#24794) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-11-11 07:40:44 -05:00
Robert Shaw
e605e8e323
[Bugfix] Fix Stream Sync for Shared Expert Overlap (#28430) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-11-11 05:59:08 +00:00
Zuyi Zhao
bca74e32b7
[Frontend] Add sagemaker_standards dynamic lora adapter and stateful session management decorators to vLLM OpenAI API server (#27892) 
Signed-off-by: Zuyi Zhao <zhaozuy@amazon.com>
Signed-off-by: Shen Teng <sheteng@amazon.com>
Co-authored-by: Shen Teng <sheteng@amazon.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-11-11 04:57:01 +00:00
Zhuohan Li
8d706cca90
[Misc] FlattenLogprobs -> FlatLogprobs (#28335) 2025-11-11 03:41:23 +00:00