xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-13 11:25:40 +08:00
Code Issues Packages Projects Releases Wiki Activity

Sort the outputs before return (#402)

Browse Source
This commit is contained in:
Woosuk Kwon 2023-07-08 14:48:18 -07:00 committed by GitHub
parent 2179e4f4c5
commit b6fbb9a565
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/entrypoints/llm.py
View File

@ -155,4 +155,8 @@ class LLM:
pbar.update(1) pbar.update(1)
if use_tqdm: if use_tqdm:
pbar.close() pbar.close()
# Sort the outputs by request ID.
# This is necessary because some requests may be finished earlier than
# its previous requests.
outputs = sorted(outputs, key=lambda x: int(x.request_id))
return outputs return outputs