xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-16 11:35:50 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Misc] update the comments (#15780)

Browse Source 
Signed-off-by: chengyang liu <lcy4869@gmail.com>
Co-authored-by: chengyang liu <lcy4869@gmail.com>
This commit is contained in:
Chengyang LIU 2025-03-30 19:39:56 -07:00 committed by GitHub
parent 9b459eca88
commit 18ed3132d2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/v1/worker/gpu_model_runner.py
View File

@ -673,7 +673,7 @@ class GPUModelRunner(LoRAModelRunnerMixin):
# use two kernels for cascade attention. Let's imagine:
# Request 3's input query: [D]
# Request 3's kv cache: [A, B, C, D]
# Request 3's num_computed_tokens: 4 (i.e., [A, B, C, D])
# Request 3's num_computed_tokens: 3 (i.e., [A, B, C])
# If we use [A, B, C, D] as the common prefix for Request 1-3,
# then Request 3 will be processed only by the first kernel,
# and the second kernel will get an empty input. While this is not