xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-09 13:17:17 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,770 Commits 155 Branches 89 Tags
Commit Graph

2507 Commits

Author SHA1 Message Date
Mengqing Cao
7ea3cd7c3e
[Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2024-11-25 05:14:56 +00:00
Maximilien de Bayser
214efc2c3c
Support Cross encoder models (#10400) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-24 18:56:20 -08:00
youkaichao
c055747867
[model][utils] add extract_layer_index utility function (#10599) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-23 22:22:54 -08:00
Jee Jee Li
1700c543a5
[Bugfix] Fix LoRA weight sharding (#10450) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-23 17:23:17 -08:00
Jee Jee Li
17d8fc1806
[bugfix] Fix example/tensorize_vllm_model tests (#10595) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-23 17:22:33 -08:00
Isotr0py
04668ebe7a
[Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-23 18:12:20 +00:00
JiHuazhong
86a44fb896
[Platforms] Refactor openvino code (#10573) 
Signed-off-by: statelesshz <hzji210@gmail.com>
 2024-11-22 22:23:12 -08:00
Isotr0py
4cfe5d2bca
[Bugfix] multi_modal_kwargs broadcast for CPU tensor parallel (#10541) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-22 21:25:46 -08:00
Cyrus Leung
c8acd80548
[2/N] handling placeholders in merged multi-modal processor (#10485) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-22 21:25:09 -08:00
Ricky Xu
4634a89d18
Prefix Cache Aware Scheduling [1/n] (#10128) 
Signed-off-by: rickyx <rickyx@anyscale.com>
 2024-11-22 21:15:55 -08:00
kliuae
7c25fe45a6
[AMD] Add support for GGUF quantization on ROCm (#10254) 2024-11-22 21:14:49 -08:00
Michael Goin
02a43f82a9
Update default max_num_batch_tokens for chunked prefill to 2048 (#10544) 2024-11-22 21:14:19 -08:00
Chen Wu
cfea9c04ef
[Model] Fix Baichuan BNB online quantization (#10572) 
Signed-off-by: Chen Wu <cntryroa@gmail.com>
 2024-11-22 21:13:59 -08:00
Varun Vinayak Shenoy
7d8ffb344f
[Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) 
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
 2024-11-22 21:13:29 -08:00
youkaichao
4aba6e3d1a
[core] gemma2 full context length support (#10584) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 20:13:54 -08:00
Tyler Michael Smith
978b39744b
[Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) 2024-11-22 22:14:03 -05:00
Russell Bryant
ebda51968b
[Core] Fix broken log configuration (#10458) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-23 10:23:51 +08:00
youkaichao
d559979c54
[bugfix] fix cpu tests (#10585) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 17:34:03 -08:00
Zhonghua Deng
d345f409b7
[V1] EngineCore supports profiling (#10564) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
 2024-11-22 17:16:15 -08:00
Russell Bryant
28598f3939
[Core] remove temporary local variables in LLMEngine.__init__ (#10577) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-22 16:22:53 -08:00
zixuanzhang226
948c859571
support bitsandbytes quantization with qwen model (#10549) 
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
 2024-11-22 16:16:14 -08:00
Ricky Xu
97814fbf0f
[v1] Refactor KVCacheManager for more hash input than token ids (#10507) 
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2024-11-22 23:27:25 +00:00
youkaichao
eebad39f26
[torch.compile] support all attention backends (#10558) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 14:04:42 -08:00
Noam Gat
11fcf0e066
Remove token-adding chat embedding params (#10551) 
Signed-off-by: Noam Gat <noamgat@gmail.com>
 2024-11-21 23:59:47 -08:00
Isotr0py
b6374e09b0
[Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-22 15:01:56 +08:00
youkaichao
a111d0151f
[platforms] absorb worker cls difference into platforms folder (#10555) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2024-11-21 21:00:32 -08:00
Woosuk Kwon
446c7806b2
[Minor] Fix line-too-long (#10563) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-11-21 19:40:40 -08:00
youkaichao
33e0a2540a
[9/N] torch.compile LLM usage (#10552) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-21 19:13:31 -08:00
youkaichao
cf656f5a02
[misc] improve error message (#10553) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-21 13:13:17 -08:00
Woosuk Kwon
f9310cbd0c
[V1] Fix Compilation config & Enable CUDA graph by default (#10528) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-11-21 12:53:39 -08:00
youkaichao
7560ae5caf
[8/N] enable cli flag without a space (#10529) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-21 12:30:42 -08:00
Cyrus Leung
e7a8341c7c
[Bugfix] Allow token ID-only inputs in Qwen2-Audio (#10536) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-21 18:09:43 +00:00
Roger Wang
c51e397fe8
[Misc] Suppress duplicated logging regarding multimodal input pipeline (#10530) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-21 09:21:31 -08:00
Jee Jee Li
2385b60d83
[Kernel] Register punica ops directly (#10522) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-21 09:18:11 -08:00
Chauncey
da7e702c6f
[Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2024-11-21 16:24:32 +00:00
Xiaoyu Zhang
4d676f0852
[Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) 2024-11-21 14:40:02 +00:00
Isotr0py
d5ec121f95
[Model] Expose dynamic_image_size as mm_processor_kwargs for InternVL2 models (#10518) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-21 14:20:08 +00:00
Alex Brooks
1cfde82ffd
[Model] Add Support for Multimodal Granite Models (#10291) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-21 10:46:20 +00:00
Zhong Qishuai
f0e0238016
[Doc] fix a small typo in docstring of llama_tool_parser (#10513) 2024-11-21 09:05:23 +00:00
youkaichao
aaddce5d26
[platforms] improve error message for unspecified platforms (#10520) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 23:07:56 -08:00
Cyrus Leung
3430857b64
[Misc] Increase default video fetch timeout (#10495) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-20 23:06:42 -08:00
Luka Govedič
8b0fe06c89
[torch.compile] Inductor code caching fix (#10273) 
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
 2024-11-20 21:44:57 -08:00
Mengqing Cao
9d827170a3
[Platforms] Add device_type in Platform (#10508) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2024-11-21 04:44:20 +00:00
Pavani Majety
6c1208d083
[Core] Add Sliding Window Support with Flashinfer (#10462) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2024-11-20 19:56:47 -08:00
youkaichao
388ee3de66
[torch.compile] limit inductor threads and lazy import quant (#10482) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 18:36:33 -08:00
Woosuk Kwon
2f77b6cfec
[TPU] Implement prefix caching for TPUs (#10307) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-11-20 13:54:15 -08:00
Guillaume Calmettes
c68f7ede6a
[Bugfix]: allow extra fields in requests to openai compatible server (#10463) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2024-11-20 16:42:21 -05:00
youkaichao
0cd3d9717e
[7/N] torch.compile, reduce compilation time (#10460) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 11:20:38 -08:00
youkaichao
772a66732d
[platforms] restore xpu check for parallel config (#10479) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 17:13:28 +00:00
Li, Jiang
63f1fde277
[Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2024-11-20 10:57:39 +00:00