xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-24 16:27:20 +08:00
Code Issues Packages Projects Releases Wiki Activity

Doc fix in benchmark_long_document_qa_throughput.py (#11933)

Browse Source 
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
This commit is contained in:
Kuntai Du 2025-01-10 23:51:43 +08:00 committed by GitHub
parent f33e033e27
commit 5959564f94
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
benchmarks/benchmark_long_document_qa_throughput.py
View File

@ -2,8 +2,7 @@
Offline benchmark to test the long document QA throughput. Offline benchmark to test the long document QA throughput.
Example usage: Example usage:
# This command run the vllm with 50GB CPU memory for offloading # This workload samples 8 different prompts with a default input
# The workload samples 8 different prompts with a default input
# length of 20000 tokens, then replicates each prompt 2 times # length of 20000 tokens, then replicates each prompt 2 times
# in random order. # in random order.
python benchmark_long_document_qa_throughput.py \ python benchmark_long_document_qa_throughput.py \