diff --git a/docs/source/models/engine_args.rst b/docs/source/models/engine_args.rst index a70c22e9af11..d89b79514950 100644 --- a/docs/source/models/engine_args.rst +++ b/docs/source/models/engine_args.rst @@ -89,9 +89,11 @@ Below, you can find an explanation of every engine argument for vLLM: CPU swap space size (GiB) per GPU. -.. option:: --gpu-memory-utilization +.. option:: --gpu-memory-utilization - The percentage of GPU memory to be used for the model executor. + The fraction of GPU memory to be used for the model executor, which can range from 0 to 1. + For example, a value of 0.5 would imply 50% GPU memory utilization. + If unspecified, will use the default value of 0.9. .. option:: --max-num-batched-tokens