xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 01:05:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Feature] Default ignore_eos True for random dataset (#28227)

Browse Source 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
This commit is contained in:
Wentao Ye 2025-11-07 07:35:33 -05:00 committed by GitHub
parent e0d6b4a867
commit 4b1ff13221
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
vllm/benchmarks/serve.py
View File

@ -1359,6 +1359,14 @@ async def main_async(args: argparse.Namespace) -> dict[str, Any]:
"'--dataset-path' if required." "'--dataset-path' if required."
) )
# when using random datasets, default to ignoring EOS
# so generation runs to the requested length
if (
args.dataset_name in ("random", "random-mm")
and args.backend in OPENAI_COMPATIBLE_BACKENDS
):
args.ignore_eos = True
# Load the dataset. # Load the dataset.
input_requests = get_samples(args, tokenizer) input_requests = get_samples(args, tokenizer)
goodput_config_dict = check_goodput_args(args) goodput_config_dict = check_goodput_args(args)