From 60a98b2de5ffab549ec768275f08c2dfb236f99b Mon Sep 17 00:00:00 2001 From: Harry Mellor <19981378+hmellor@users.noreply.github.com> Date: Mon, 10 Mar 2025 13:13:10 +0100 Subject: [PATCH] [Docs] Mention `model_impl` arg when explaining Transformers fallback (#14552) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> --- docs/source/models/supported_models.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/source/models/supported_models.md b/docs/source/models/supported_models.md index ff28fde5b7ff9..1fde1761672e9 100644 --- a/docs/source/models/supported_models.md +++ b/docs/source/models/supported_models.md @@ -59,6 +59,10 @@ llm.apply_model(lambda model: print(type(model))) If it is `TransformersModel` then it means it's based on Transformers! +:::{tip} +You can force the use of `TransformersModel` by setting `model_impl="transformers"` for or `--model-impl transformers` for the . +::: + :::{note} vLLM may not fully optimise the Transformers implementation so you may see degraded performance if comparing a native model to a Transformers model in vLLM. :::