xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-22 07:25:02 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,557 Commits 155 Branches 89 Tags
Commit Graph

1962 Commits

Author SHA1 Message Date
youkaichao
25d806e953
[misc] add torch.compile compatibility check (#10618) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-24 23:40:08 -08:00
youkaichao
571841b7fc
[torch.compile] support encoder based models (#10613) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-25 05:24:33 +00:00
Maximilien de Bayser
214efc2c3c
Support Cross encoder models (#10400) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-24 18:56:20 -08:00
Jee Jee Li
1700c543a5
[Bugfix] Fix LoRA weight sharding (#10450) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-23 17:23:17 -08:00
Cyrus Leung
c8acd80548
[2/N] handling placeholders in merged multi-modal processor (#10485) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-22 21:25:09 -08:00
Ricky Xu
4634a89d18
Prefix Cache Aware Scheduling [1/n] (#10128) 
Signed-off-by: rickyx <rickyx@anyscale.com>
 2024-11-22 21:15:55 -08:00
Varun Vinayak Shenoy
7d8ffb344f
[Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) 
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
 2024-11-22 21:13:29 -08:00
youkaichao
4aba6e3d1a
[core] gemma2 full context length support (#10584) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 20:13:54 -08:00
Tyler Michael Smith
978b39744b
[Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) 2024-11-22 22:14:03 -05:00
Travis Johnson
9195dbdbca
[Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-11-23 10:17:38 +08:00
Ricky Xu
97814fbf0f
[v1] Refactor KVCacheManager for more hash input than token ids (#10507) 
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2024-11-22 23:27:25 +00:00
youkaichao
eebad39f26
[torch.compile] support all attention backends (#10558) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 14:04:42 -08:00
youkaichao
db100c5cde
[bugfix] fix full graph tests (#10581) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 10:02:14 -08:00
youkaichao
33e0a2540a
[9/N] torch.compile LLM usage (#10552) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-21 19:13:31 -08:00
youkaichao
7560ae5caf
[8/N] enable cli flag without a space (#10529) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-21 12:30:42 -08:00
Jee Jee Li
2385b60d83
[Kernel] Register punica ops directly (#10522) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-21 09:18:11 -08:00
Chauncey
da7e702c6f
[Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2024-11-21 16:24:32 +00:00
Isotr0py
d5ec121f95
[Model] Expose dynamic_image_size as mm_processor_kwargs for InternVL2 models (#10518) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-21 14:20:08 +00:00
Luka Govedič
8b0fe06c89
[torch.compile] Inductor code caching fix (#10273) 
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
 2024-11-20 21:44:57 -08:00
Pavani Majety
6c1208d083
[Core] Add Sliding Window Support with Flashinfer (#10462) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2024-11-20 19:56:47 -08:00
youkaichao
388ee3de66
[torch.compile] limit inductor threads and lazy import quant (#10482) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 18:36:33 -08:00
Guillaume Calmettes
c68f7ede6a
[Bugfix]: allow extra fields in requests to openai compatible server (#10463) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2024-11-20 16:42:21 -05:00
youkaichao
0cd3d9717e
[7/N] torch.compile, reduce compilation time (#10460) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 11:20:38 -08:00
Li, Jiang
63f1fde277
[Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2024-11-20 10:57:39 +00:00
Lucas Wilkinson
d200972e7f
[Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2024-11-19 19:40:33 -08:00
ElizaWszola
b00b33d77e
[Model][Quantization] HQQ support through Marlin kernel expansion (#9766) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
 2024-11-19 13:31:12 -08:00
youkaichao
803f37eaaa
[6/N] torch.compile rollout to users (#10437) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-19 10:09:03 -08:00
Mengqing Cao
8c1fb50705
[Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2024-11-19 11:22:26 +08:00
Lucas Wilkinson
96d999fbe8
[Kernel] Initial Machete W4A8 support + Refactors (#9855) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2024-11-18 12:59:29 -07:00
Yan Ma
6b2d25efc7
[Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2024-11-18 11:18:05 -07:00
lkchen
c7dec926f6
[VLM] Report multi_modal_placeholders in output (#10407) 
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
 2024-11-18 16:06:16 +08:00
youkaichao
4fd9375028
[2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-16 18:02:14 -08:00
电脑星人
361c29e174
[Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 02:10:00 +08:00
Cyrus Leung
32e46e000f
[Frontend] Automatic detection of chat content format from AST (#9919) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-16 13:35:40 +08:00
ElizaWszola
79ee45b428
[Misc] Bump up test_fused_moe tolerance (#10364) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
 2024-11-15 16:31:18 +00:00
Cyrus Leung
b311efd0bd
[Misc] Fix import error in tensorizer tests and cleanup some code (#10349) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 09:34:17 +00:00
Cyrus Leung
2ac6d0e75b
[Misc] Consolidate pooler config overrides (#10351) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 06:59:00 +00:00
Cyrus Leung
b40cf6402e
[Model] Support Qwen2 embeddings and use tags to select model tests (#10184) 2024-11-14 20:23:09 -08:00
Luka Govedič
bf2ddc6610
[bugfix] Fix static asymmetric quantization case (#10334) 
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-11-15 09:35:11 +08:00
Cyrus Leung
972112d82f
[Bugfix] Fix unable to load some models (#10312) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-14 16:55:54 -08:00
Patrick von Platen
11cd1ae6ad
[Tool parsing] Improve / correct mistral tool parsing (#10333) 2024-11-15 00:42:49 +00:00
Maximilien de Bayser
4a18fd14ba
Support Roberta embedding models (#9387) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-14 21:23:29 +00:00
youkaichao
29f3ef26a3
[ci][distributed] disable hanging tests (#10317) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-14 00:23:39 -08:00
Mike Depinet
f67ce05d0b
[Frontend] Pythonic tool parser (#9859) 
Signed-off-by: Mike Depinet <mike@fixie.ai>
 2024-11-14 04:14:34 +00:00
Isotr0py
15bb8330aa
[Bugfix] Fix tensor parallel for qwen2 classification model (#10297) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-14 10:54:59 +08:00
HoangCongDuc
ac49b59d8b
[Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) 
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
 2024-11-13 09:56:39 -07:00
Cyrus Leung
0b8bb86bf1
[1/N] Initial prototype for multi-modal processor (#10044) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-13 12:39:03 +00:00
Austin Veselka
1b886aa104
[Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) 
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
 2024-11-13 08:28:13 +00:00
电脑星人
3945c82346
[Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-13 07:07:22 +00:00
youkaichao
0d4ea3fb5c
[core][distributed] use tcp store directly (#10275) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-12 17:36:08 -08:00