From 89ec06c33b9b3e64a6562a82049de44cbf7f9e09 Mon Sep 17 00:00:00 2001 From: Cade Daniel Date: Tue, 11 Jun 2024 10:31:56 -0700 Subject: [PATCH] [Docs] [Spec decode] Fix docs error in code example (#5427) --- docs/source/models/spec_decode.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/models/spec_decode.rst b/docs/source/models/spec_decode.rst index 57ff4517e9b91..9fb62397b9aaf 100644 --- a/docs/source/models/spec_decode.rst +++ b/docs/source/models/spec_decode.rst @@ -17,6 +17,7 @@ Speculating with a draft model The following code configures vLLM to use speculative decoding with a draft model, speculating 5 tokens at a time. .. code-block:: python + from vllm import LLM, SamplingParams prompts = [ @@ -45,6 +46,7 @@ The following code configures vLLM to use speculative decoding where proposals a matching n-grams in the prompt. For more information read `this thread. `_ .. code-block:: python + from vllm import LLM, SamplingParams prompts = [