xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-15 11:37:02 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,827 Commits 155 Branches 89 Tags
Commit Graph

188 Commits

Author SHA1 Message Date
Li, Jiang
e3a0e43d7f
[bugfix] Fix auto thread-binding when world_size > 1 in CPU backend and refactor code (#21032) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-19 05:13:55 -07:00
Kaixi Hou
6d0734c562
[NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) 
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-19 02:33:01 -07:00
Shu Wang
c7d8724e78
[Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) 
Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-17 21:32:45 -07:00
Woosuk Kwon
4de7146351
[V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Elfie Guo
30800b01c2
[Nvidia] Integrate SM100 cudnn prefill API to MLA prefill (#20411) 
Signed-off-by: Elfie Guo <elfieg@nvidia.com>
Co-authored-by: Elfie Guo <eflieg@nvidia.com>
 2025-07-15 17:56:45 -07:00
Chen LI
10be209493
[Bug Fix] get_distributed_init_method should get the ip from get_ip i… (#20889) 
Signed-off-by: Chen Li <lcpingping@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-07-15 21:23:52 +00:00
Boyuan Feng
91b3d190ae
[cold start] replace VLLM_COMPILE_DEPYF with debug_dump_dir (#20940) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-07-15 13:02:17 +08:00
Boyuan Feng
c1c8ca57ff
[cold start time] add envs.VLLM_COMPILE_DEPYF to guard decompile (#20790) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-07-11 23:06:13 -07:00
Pavani Majety
7bd4c37ae7
[Core] Add Flashinfer TRTLLM Backend for Flashinfer decode path (SM100). (#19825) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: shuw <shuw@nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 09:23:23 +00:00
Nathan Hoos
d6902ce79f
[V0][V1][Core] Add outlines integration for V1, and update V0 integration. (#15975) 
Signed-off-by: Nathan Hoos <thwackyy.y@gmail.com>
 2025-07-10 15:30:26 -04:00
fxmarty-amd
332d4cb17b
[Feature][Quantization] MXFP4 support for MOE models (#17888) 
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>
 2025-07-09 13:19:02 -07:00
Nicolò Lucchesi
71d1d75b7a
[PD][Nixl] Remote consumer READ timeout for clearing request blocks (#20139) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-08 08:56:40 +01:00
Wentao Ye
9dae7d46bf
[Refactor] Remove Unused Env VLLM_ENABLE_MOE_ALIGN_BLOCK_SIZE_TRITON (#20334) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-01 19:03:43 -07:00
Li, Jiang
6cc1e7d96d
[CPU] Update custom ops for the CPU backend (#20255) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-01 07:25:03 +00:00
li haoyang
0740e29b66
[Feature] add quick all reduce (#19744) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-06-26 20:54:24 -07:00
Chenyaaang
2d7620c3eb
[TPU] Add TPU specific var VLLM_TPU_MOST_MODEL_LEN (#19919) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-06-25 15:51:02 -07:00
Dipika Sikka
02c97d9a92
[Quantization] Add compressed-tensors emulations support for NVFP4 (#19879) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
 2025-06-25 14:28:19 -04:00
bnellnm
015fab8c2f
[Kernels][Bugfix] Use torch op for all kernels in FusedMoE forward. Add additional testing for cudagraphs. (#19717) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-24 23:22:58 -07:00
jinqinn
f39ab2d4bd
[Misc] Configurable timeout for execute_model RPC calls via env var (#19544) 
Signed-off-by: jinqinn <goodqinjin@163.com>
 2025-06-22 20:36:26 -07:00
Ye (Charlotte) Qi
33d51f599e
[BugFix] Add an env to disable moe chunking to work around compile incompatibility (#19642) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-06-22 15:17:49 -07:00
Vlad Tiberiu Mihailescu
2e3e3c86dc
Export NaNs in logits to scheduler_stats if output is corrupted (#18777) 
Signed-off-by: Vlad Mihailescu <vtmihailescu@gmail.com>
 2025-06-20 22:47:16 +08:00
Zzz9990
8b6e1d639c
[Hardware][AMD] integrate aiter chunked prefill into vllm (#18596) 
Signed-off-by: fsx950223 <fsx950223@outlook.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: fsx950223 <fsx950223@outlook.com>
Co-authored-by: charlifu <charlifu@amd.com>
 2025-06-18 08:46:51 -07:00
Nicolò Lucchesi
4c8f64faa7
[V1][Kernel] Flashinfer HND KV cache layout (#19280) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-17 09:09:22 -04:00
Varun Sundar Rabindranath
9d880f594d
[Misc] Turn MOE_DP_CHUNK_SIZE into an env var (#19506) 2025-06-12 18:01:16 +00:00
Luka Govedič
f98548b9da
[torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
 2025-06-12 08:31:04 -07:00
Louie Tsai
9368cc90b2
Automatically bind CPU OMP Threads of a rank to CPU ids of a NUMA node. (#17930) 
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
 2025-06-10 06:22:05 +00:00
liusiqian-tal
31f58be96a
[Frontend] Make TIMEOUT_KEEP_ALIVE configurable through env var (#18472) 
Signed-off-by: liusiqian <liusiqian@tal.com>
 2025-06-09 21:41:21 +00:00
Varun Sundar Rabindranath
5cf2daea9a
[Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-09 10:50:39 -04:00
jennyyyyzhen
cda10fa3e2
[Multi Modal] Add an env var for message queue max chunk bytes (#19242) 
Signed-off-by: yZhen <yZhen@fb.com>
Co-authored-by: yZhen <yZhen@fb.com>
 2025-06-08 21:39:12 +08:00
Povilas Kanapickas
85e2b7bb13
[MISC][Bugfix] Use less CPU when message queue has been empty for some time (#16226) 
Signed-off-by: Povilas Kanapickas <povilas@radix.lt>
 2025-06-05 16:53:08 +00:00
Varun Sundar Rabindranath
fa98d77773
[Kernel] DeepEP dispatch-combine kernel integration (#18434) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-03 12:30:02 -07:00
Simon Mo
02f0c7b220
[Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Siyuan Liu
9112b443a0
[Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-06-03 00:06:20 +00:00
Will Eaton
1dab4d5718
Tool parser regex timeout handling (#18960) 
Signed-off-by: Will Eaton <weaton@redhat.com>
 2025-05-30 21:02:54 +00:00
Rabi Mishra
5f1d0c8118
[Bugfix][Failing Test] Fix test_vllm_port.py (#18618) 
Signed-off-by: rabi <ramishra@redhat.com>
 2025-05-30 17:13:47 +08:00
Richard Zou
a521ef06e5
Use standalone_compile by default in torch >= 2.8.0 (#18846) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-05-30 06:41:58 +08:00
Gregory Shtrasberg
da4b69d0b4
[Attention][V1] Toggle for v1 attention backend (#18275) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-29 10:48:24 -04:00
Cyrus Leung
1cb194a018
[Doc] Reorganize user guide (#18661) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-24 07:25:33 -07:00
Pavani Majety
f2036734fb
[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control blockscale tensor allocation (#18160) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-05-23 15:52:20 -07:00
youkaichao
6a7988c55b
Refactor pplx init logic to make it modular (prepare for deepep) (#18200) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-05-23 23:43:43 +08:00
Harry Mellor
a1fe24d961
Migrate docs from Sphinx to MkDocs (#18145) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 02:09:53 -07:00
Will Eaton
6b31c84aff
Throw better error for when running into k8s service discovery issue (#18209) 
Signed-off-by: Will Eaton <weaton@redhat.com>
 2025-05-15 21:07:28 -07:00
Chenheli Hua
4f07a64075
Support custom implementations of VideoLoader backends. (#18091) 2025-05-15 13:26:49 +08:00
youkaichao
6266c57bae
[core][distributed] add ep group and all2all interface (#18077) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-05-14 10:46:49 +08:00
Jonathan Berkhahn
98ea35601c
[Lora][Frontend]Add default local directory LoRA resolver plugin. (#16855) 
Signed-off-by: jberkhahn <jaberkha@us.ibm.com>
 2025-05-12 10:39:10 -07:00
Robert Shaw
d19110204c
[P/D] NIXL Integration (#17751) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>
 2025-05-12 09:46:16 -07:00
Richard Zou
ea2236bf95
Add option to use torch._inductor.standalone_compile (#17057) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-05-09 12:59:04 -07:00
Russell Bryant
6930a41116
[V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-05-08 13:34:02 +08:00
Bowen Bao
db593aa67f
[Quantization] Quark MXFP4 format loading (#16943) 2025-05-07 15:05:05 -04:00
Areeb Syed
ed2e464653
Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) 
Signed-off-by: sydarb <areebsyed237@gmail.com>
 2025-04-23 19:55:00 -07:00