xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-23 03:57:04 +08:00
Code Issues Packages Projects Releases Wiki Activity

minor

Browse Source 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
This commit is contained in:
Woosuk Kwon 2025-09-18 16:31:01 -07:00
parent 82da219ff9
commit efda08481b
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/v1/worker/gpu/sampler.py
View File

@ -254,8 +254,8 @@ def compute_logprobs(
) )
# NOTE(woosuk): Here, to save GPU memory, we do not materialize the full # NOTE(woosuk): Here, to save GPU memory, we do not materialize the full
# logprobs tensor. Instead, we only compute the logprobs of the topk + 1 # logprobs tensor. Instead, we only compute and return the logprobs of
# tokens. # the topk + 1 tokens.
BLOCK_SIZE = 1024 BLOCK_SIZE = 1024
_topk_logprobs_kernel[(batch_size, )]( _topk_logprobs_kernel[(batch_size, )](
logprobs, logprobs,