xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-23 08:57:53 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Misc] Update benchmark_prefix_caching.py fixed example usage (#11920)

Browse Source 
Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn>
Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
This commit is contained in:
minmin 2025-01-11 04:39:22 +08:00 committed by GitHub
parent 46fa98ccad
commit 8a579408f3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
benchmarks/benchmark_prefix_caching.py
View File

@ -10,7 +10,8 @@ Fixed example usage:
--model meta-llama/Llama-2-7b-chat-hf \ --model meta-llama/Llama-2-7b-chat-hf \
--enable-prefix-caching \ --enable-prefix-caching \
--num-prompts 1 \ --num-prompts 1 \
--repeat-count 100 --repeat-count 100 \
--input-length-range 128:256
ShareGPT example usage: ShareGPT example usage:
# This command samples 20 prompts with input lengths # This command samples 20 prompts with input lengths