xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-20 05:36:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,200 Commits 155 Branches 89 Tags
Commit Graph

373 Commits

Author SHA1 Message Date
youkaichao
e66faf4809
[torch.compile] store inductor compiled Python file (#12182) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-19 16:27:26 +08:00
Chen Zhang
d1adb9b403
[BugFix] add more is not None check in VllmConfig.__post_init__ (#12138) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-01-17 05:33:22 +00:00
Chen Zhang
d06e824006
[Bugfix] Set enforce_eager automatically for mllama (#12127) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-01-16 15:30:08 -05:00
youkaichao
bf53e0c70b
Support torchrun and SPMD-style offline inference (#12071) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-16 19:58:53 +08:00
Isotr0py
dd7c9ad870
[Bugfix] Remove hardcoded head_size=256 for Deepseek v2 and v3 (#12067) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-16 10:11:54 +00:00
Roger Wang
70755e819e
[V1][Core] Autotune encoder cache budget (#11895) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-15 11:29:00 -08:00
Joe Runde
edce722eaa
[Bugfix] use right truncation for non-generative tasks (#12050) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-01-16 00:31:01 +08:00
kewang-xlnx
de0526f668
[Misc][Quark] Upstream Quark format to VLLM (#10765) 
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-15 11:05:15 -05:00
youkaichao
ad34c0df0f
[core] platform agnostic executor via collective_rpc (#11256) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-15 13:45:21 +08:00
Cyrus Leung
12664ddda5
[Doc] [1/N] Initial guide for merged multi-modal processor (#11925) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-10 14:30:25 +00:00
Chen Zhang
cf5f000d21
[torch.compile] Hide KV cache behind torch.compile boundary (#11677) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-01-10 13:14:42 +08:00
Cyrus Leung
d848800e88
[Misc] Move print_*_once from utils to logger (#11298) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
 2025-01-09 12:48:12 +08:00
youkaichao
f12141170a
[torch.compile] consider relevant code in compilation cache (#11614) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-08 10:46:43 +00:00
Wallas Henrique
cfd3219f58
[Hardware][Apple] Native support for macOS Apple Silicon (#11696) 
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-08 16:35:49 +08:00
Cyrus Leung
ef68eb28d8
[Bug] Fix pickling of ModelConfig when RunAI Model Streamer is used (#11825) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 13:40:09 +08:00
Jee Jee Li
f645eb6954
[Bugfix] Add checks for LoRA and CPU offload (#11810) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-01-08 13:08:48 +08:00
Cyrus Leung
ee77fdb5de
[Doc][2/N] Reorganize Models and Usage sections (#11755) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 21:40:31 +08:00
Cody Yu
408e560015
[Bugfix] Remove block size constraint (#11723) 2025-01-06 12:49:55 +08:00
Aurick Qiao
e1a5c2f0a1
[Model] Whisper model implementation (#11280) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-01-03 16:39:19 +08:00
youkaichao
b12e87f942
[platforms] enable platform plugins (#11602) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-30 20:24:45 +08:00
youkaichao
3682e33f9f
[v1] fix compilation cache (#11598) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-30 04:24:12 +00:00
Kuntai Du
faef77c0d6
[Misc] KV cache transfer connector registry (#11481) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2024-12-29 16:08:09 +00:00
youkaichao
328841d002
[bugfix] interleaving sliding window for cohere2 model (#11583) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-28 16:55:42 +00:00
Simon Mo
f49777ba62
Deepseek v3 (#11502) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
 2024-12-26 16:09:44 -08:00
Michael Goin
2072924d14
[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>
 2024-12-26 15:33:30 -08:00
Cyrus Leung
eec906d811
[Misc] Add placeholder module (#11501) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-26 13:12:51 +00:00
Rafael Vasquez
32aa2059ad
[Docs] Convert rST to MyST (Markdown) (#11145) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2024-12-23 22:35:38 +00:00
omer-dayan
995f56236b
[Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) 
Signed-off-by: OmerD <omer@run.ai>
 2024-12-20 16:46:24 +00:00
Akash kaothalkar
48edab8041
[Bugfix][Hardware][POWERPC] Fix auto dtype failure in case of POWER10 (#11331) 
Signed-off-by: Akash Kaothalkar <0052v2@linux.vnet.ibm.com>
 2024-12-20 01:32:07 +00:00
Yanyi Liu
5aef49806d
[Feature] Add load generation config from model (#11164) 
Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2024-12-19 10:50:38 +00:00
Alexander Matveev
fdea8ec167
[V1] VLM - enable processor cache by default (#11305) 
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
 2024-12-18 18:54:46 -05:00
Konrad Zawora
866fa4550d
[Bugfix] Restore support for larger block sizes (#11259) 
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
 2024-12-17 16:39:07 -08:00
Roger Wang
59c9b6ebeb
[V1][VLM] Proper memory profiling for image language models (#11210) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
 2024-12-16 22:10:57 -08:00
youkaichao
88a412ed3d
[torch.compile] fast inductor (#11108) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-16 16:15:22 -08:00
shangmingc
d263bd9df7
[Core] Support disaggregated prefill with Mooncake Transfer Engine (#10884) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2024-12-15 21:28:18 +00:00
Brad Hilton
9c3dadd1c9
[Frontend] Add logits_processors as an extra completion argument (#11150) 
Signed-off-by: Brad Hilton <brad.hilton.nw@gmail.com>
 2024-12-14 16:46:42 +00:00
youkaichao
be39e3cd18
[core] clean up cudagraph batchsize padding logic (#10996) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-13 06:57:50 +00:00
Alexander Matveev
4e11683368
[V1] VLM preprocessor hashing (#11020) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-12-12 00:55:30 +00:00
youkaichao
91642db952
[torch.compile] use depyf to dump torch.compile internals (#10972) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-11 10:43:05 -08:00
Cyrus Leung
cad5c0a6ed
[Doc] Update docs to refer to pooling models (#11093) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 13:36:27 +00:00
Cyrus Leung
8f10d5e393
[Misc] Split up pooling tasks (#10820) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 01:28:00 -08:00
Mor Zusman
ffa48c9146
[Model] PP support for Mamba-like models (#10992) 
Signed-off-by: mzusman <mor.zusmann@gmail.com>
 2024-12-10 21:53:37 -05:00
Aurick Qiao
d5c5154fcf
[Misc] LoRA + Chunked Prefill (#9057) 2024-12-11 10:09:20 +08:00
youkaichao
1a2f8fb828
[v1] fix use compile sizes (#11000) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-09 13:47:24 -08:00
wangxiyuan
aea2fc38c3
[Platform] Move async output check to platform (#10768) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2024-12-09 17:24:46 +00:00
youkaichao
46004e83a2
[misc] clean up and unify logging (#10999) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-08 17:28:27 -08:00
youkaichao
43b05fa314
[torch.compile][misc] fix comments (#10993) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-08 11:18:18 -08:00
youkaichao
fd57d2b534
[torch.compile] allow candidate compile sizes (#10984) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-08 11:05:21 +00:00
youkaichao
1b62745b1d
[core][executor] simplify instance id (#10976) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-07 09:33:45 -08:00
Cyrus Leung
bf0e382e16
[Model] Composite weight loading for multimodal Qwen2 (#10944) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 07:22:52 -07:00