xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-19 21:05:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,847 Commits 155 Branches 89 Tags
Commit Graph

3847 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaxin Shan
0d8451c3a4
[Distributed] Allow the placement group more time to wait for resources to be ready (#11138) 
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
 2024-12-13 20:17:37 +00:00
Jani Monoses
0a56bcc03d
[Bugfix][Hardware][CPU] Enable Gemma2 with SDPA on CPU backend (#11169) 2024-12-13 18:00:40 +00:00
Cyrus Leung
0920ab9131
[Doc] Reorganize online pooling APIs (#11172) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-14 00:22:22 +08:00
Alexander Matveev
238c0d93b4
[Misc] Add tokenizer_mode param to benchmark_serving.py (#11174) 
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
 2024-12-13 16:19:10 +00:00
zhangjf
5b0ed8391d
[Bugfix] using len(tokenizer) instead of tokenizer.vocab_size in AllowedTokenIdsLogitsProcessor (#11156) 2024-12-13 15:56:19 +00:00
Sungjae Lee
c31d4a57a6
[Core] support LoRA and prompt adapter in content-based hashing for Block Manager v2 prefix caching (#8240) 2024-12-13 07:51:25 -08:00
Chenguang Li
d1fa714cb1
[Refactor]A simple device-related refactor (#11163) 
Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>
 2024-12-13 13:39:00 +00:00
Roger Wang
969da7d70b
[V1][VLM] Fix edge case bug for InternVL2 (#11165) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-12-13 11:09:30 +00:00
Cyrus Leung
eeec9e3390
[Frontend] Separate pooling APIs in offline inference (#11129) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-13 10:40:07 +00:00
Li, Jiang
f93bf2b189
[Bugfix][CI][CPU] add missing datasets package to requirements-cpu.txt (#11159) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2024-12-13 08:50:35 +00:00
Jani Monoses
7cd7409142
PaliGemma 2 support (#11142) 2024-12-13 07:40:07 +00:00
youkaichao
be39e3cd18
[core] clean up cudagraph batchsize padding logic (#10996) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-13 06:57:50 +00:00
Cody Yu
34f1a806d5
[Bugfix][V1] Fix 'NoneType' object has no attribute 'hash_value' (#11157) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2024-12-13 06:30:06 +00:00
Gregory Shtrasberg
00c1bde5d8
[ROCm][AMD] Disable auto enabling chunked prefill on ROCm (#11146) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2024-12-13 05:31:26 +00:00
Dipika Sikka
3989a79824
[Bugfix] Update starcoder2 to remap k/v scale names for kv_cache quantization (#11148) 2024-12-13 05:07:20 +00:00
Pooya Davoodi
1efce68605
[Bugfix] Use runner_type instead of task in GritLM (#11144) 
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
 2024-12-13 04:09:53 +00:00
Luka Govedič
30870b4f66
[torch.compile] Dynamic fp8 + rms_norm fusion (#10906) 
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-12-13 03:19:23 +00:00
Cody Yu
78ed8f57d8
[Misc][V1] Fix type in v1 prefix caching (#11151) 2024-12-13 00:57:40 +00:00
shangmingc
db6c264a1e
[Bugfix] Fix value unpack error of simple connector for KVCache transfer. (#11058) 
Signed-off-by: ShangmingCai <csmthu@gmail.com>
 2024-12-12 21:19:17 +00:00
Jeremy Arnold
9f3974a319
Fix logging of the vLLM Config (#11143) 2024-12-12 12:05:57 -08:00
Cody Yu
2c97eca1ff
[Misc] Validate grammar and fail early (#11119) 2024-12-12 18:34:26 +00:00
Jeff Cook
5d712571af
[Bugfix] Quick fix to make Pixtral-HF load correctly again after 39e227c7ae. (#11024) 2024-12-12 18:09:20 +00:00
Ramon Ziai
d4d5291cc2
fix(docs): typo in helm install instructions (#11141) 
Signed-off-by: Ramon Ziai <ramon.ziai@bettermarks.com>
 2024-12-12 17:36:32 +00:00
Roger Wang
4816d20aa4
[V1] Fix torch profiling for offline inference (#11125) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-12-12 15:51:53 +00:00
Jiaxin Shan
85362f028c
[Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094) 
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-12 09:25:16 +00:00
youkaichao
62de37a38e
[core][distributed] initialization from StatelessProcessGroup (#10986) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-12 09:04:19 +00:00
Sanju C Sudhakaran
8195824206
[Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565) 
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
 2024-12-12 08:09:28 +00:00
Woosuk Kwon
f092153fbe
[V1] Use more persistent buffers to optimize input preparation overheads (#11111) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-12-11 23:14:20 -08:00
Pooya Davoodi
1da8f0e1dd
[Model] Add support for embedding model GritLM (#10816) 
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
 2024-12-12 06:39:16 +00:00
Russell Bryant
ccede2b264
[Core] cleanup zmq ipc sockets on exit (#11115) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-12-11 19:12:24 -08:00
Yuan Tang
24a36d6d5f
Update link to LlamaStack remote vLLM guide in serving_with_llamastack.rst (#11112) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2024-12-12 02:39:21 +00:00
Simon Mo
8fb26dac61
[Docs] Add media kit (#11121) 2024-12-11 17:33:11 -08:00
Clayton
7439a8b5fc
[Bugfix] Multiple fixes to tool streaming with hermes and mistral (#10979) 
Signed-off-by: cedonley <clayton@donley.io>
 2024-12-12 01:10:12 +00:00
Alexander Matveev
4e11683368
[V1] VLM preprocessor hashing (#11020) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-12-12 00:55:30 +00:00
Tyler Michael Smith
452a723bf2
[V1][Core] Remove should_shutdown to simplify core process termination (#11113) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-11 23:34:54 +00:00
Cyrus Leung
d1e21a979b
[CI/Build] Split up VLM tests (#11083) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-12 06:18:16 +08:00
Rui Qiao
72ff3a9686
[core] Bump ray to use _overlap_gpu_communication in compiled graph tests (#10410) 
Signed-off-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Rui Qiao <ubuntu@ip-172-31-15-128.us-west-2.compute.internal>
 2024-12-11 11:36:35 -08:00
youkaichao
66aaa7722d
[torch.compile] remove graph logging in ci (#11110) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-11 10:59:50 -08:00
Woosuk Kwon
d643c2aba1
[V1] Use input_ids as input for text-only models (#11032) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-12-11 10:49:23 -08:00
youkaichao
91642db952
[torch.compile] use depyf to dump torch.compile internals (#10972) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-11 10:43:05 -08:00
bingps
fd22220687
[Doc] Installed version of llmcompressor for int8/fp8 quantization (#11103) 
Signed-off-by: Guangda Liu <bingps@users.noreply.github.com>
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
 2024-12-11 15:43:24 +00:00
hissu-hyvarinen
b2f775456e
[CI/Build] Enable prefix caching test for AMD (#11098) 
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com>
 2024-12-11 15:23:37 +00:00
Cyrus Leung
cad5c0a6ed
[Doc] Update docs to refer to pooling models (#11093) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 13:36:27 +00:00
Cyrus Leung
8f10d5e393
[Misc] Split up pooling tasks (#10820) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 01:28:00 -08:00
Rafael Vasquez
40766ca1b8
[Bugfix]: Clamp -inf logprob values in prompt_logprobs (#11073) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2024-12-11 01:27:39 -08:00
B-201
2e32f5d28d
[Bugfix] Fix Idefics3 fails during multi-image inference (#11080) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
 2024-12-11 01:27:07 -08:00
Russell Bryant
61b1d2f6ae
[Core] v1: Use atexit to handle engine core client shutdown (#11076) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-12-11 01:26:36 -08:00
Kevin H. Luu
9974fca047
[ci/build] Fix entrypoints test and pin outlines version (#11088) 2024-12-11 01:01:53 -08:00
Kevin H. Luu
3fb4b4f163
[ci/build] Fix AMD CI dependencies (#11087) 2024-12-11 00:39:53 -08:00
Cyrus Leung
2e33fe4191
[CI/Build] Check transformers v4.47 (#10991) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 05:02:02 +00:00