xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-30 17:07:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

[LMCache][Example] Align the PYTHONHASHSEED for prefillers and decoders for KV chunks hashing (#21161)

Browse Source 
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
This commit is contained in:
zejunchen-zejun 2025-08-12 17:05:14 +08:00 committed by GitHub
parent 46ae7f6666
commit 50f2aae1b4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
examples/others/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh
View File

@ -15,6 +15,14 @@ else
MODEL=$2 MODEL=$2
fi fi
# The prefillers and decoders in LMCache use the same hash seed for all chunk keys.
# This seed must be aligned so that decoders can identify and retrieve KV cache
# entries stored by prefillers.
#
# WARNING: Using a fixed hash seed is insecure and makes the application vulnerable to
# denial-of-service attacks. In a production environment, this should be set to a
# secure random value. This is set to a fixed value for demonstration purposes only.
export PYTHONHASHSEED=${VLLM_PYTHON_HASH_SEED:-123}
if [[ $1 == "prefiller" ]]; then if [[ $1 == "prefiller" ]]; then
# Prefiller listens on port 8100 # Prefiller listens on port 8100