xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 22:05:44 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update test case parameter to have the throughput above 8.0 (#19994)

Browse Source 
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
This commit is contained in:
QiliangCui 2025-06-23 17:18:10 -07:00 committed by GitHub
parent 33d5e29be9
commit a738dbb2a1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.buildkite/scripts/tpu/config_v6e_1.env
View File

@ -4,8 +4,8 @@ CONTAINER_NAME=vllm-tpu
# vllm config
MODEL=meta-llama/Llama-3.1-8B-Instruct
MAX_NUM_SEQS=512
MAX_NUM_BATCHED_TOKENS=512
MAX_NUM_SEQS=256
MAX_NUM_BATCHED_TOKENS=1024
TENSOR_PARALLEL_SIZE=1
MAX_MODEL_LEN=2048
DOWNLOAD_DIR=/mnt/disks/persist