xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-06-01 02:27:06 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Misc]add some explanations for BlockHashType (#11847)

Browse Source
This commit is contained in:
WangErXiao 2025-01-09 02:21:30 +08:00 committed by GitHub
parent 5984499e47
commit 47de8821d3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/v1/core/kv_cache_utils.py
View File

@ -11,8 +11,10 @@ logger = init_logger(__name__)
class BlockHashType(NamedTuple): class BlockHashType(NamedTuple):
"""Hash value of a block (int), the token IDs in the block, and extra keys. """Hash value of a block (int), the token IDs in the block, and extra keys.
The reason we keep a tuple of token IDs and extra keys is to make sure We keep a tuple of token IDs and extra keys to reduce the likelihood of
no hash collision happens when the hash value is the same. hash collisions when the hash value is the same. But please note that
hash collisions can still theoretically occur, albeit with an extremely
low probability.
""" """
# Hash value of the block in an integer. # Hash value of the block in an integer.
hash_value: int hash_value: int