xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-01-23 08:14:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: update metrics design doc to use new vllm:kv_cache_usage_perc (#30041)

Browse Source 
Signed-off-by: Tim <tim.wang03@sap.com>
This commit is contained in:
TimWang 2025-12-05 07:37:14 +08:00 committed by GitHub
parent 1f0d184590
commit 690cc3ef20
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/design/metrics.md
View File

@ -62,7 +62,7 @@ The subset of metrics exposed in the Grafana dashboard gives us an indication of
- `vllm:time_per_output_token_seconds` - Inter-token latency (Time Per Output Token, TPOT) in seconds.
- `vllm:time_to_first_token_seconds` - Time to First Token (TTFT) latency in seconds.
- `vllm:num_requests_running` (also, `_swapped` and `_waiting`) - Number of requests in the RUNNING, WAITING, and SWAPPED states.
- `vllm:gpu_cache_usage_perc` - Percentage of used cache blocks by vLLM.
- `vllm:kv_cache_usage_perc` - Percentage of used cache blocks by vLLM.
- `vllm:request_prompt_tokens` - Request prompt length.
- `vllm:request_generation_tokens` - Request generation length.
- `vllm:request_success` - Number of finished requests by their finish reason: either an EOS token was generated or the max sequence length was reached.