From 9c1d5b456d916c68f41f471bfe8ea13fc69b80fa Mon Sep 17 00:00:00 2001 From: Nathan Weinberg <31703736+nathan-weinberg@users.noreply.github.com> Date: Fri, 18 Apr 2025 02:10:49 -0400 Subject: [PATCH] [Doc] add podman setup instructions for official image (#16796) Signed-off-by: Nathan Weinberg --- docs/source/deployment/docker.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/source/deployment/docker.md b/docs/source/deployment/docker.md index 6b794db656c05..ca56710bc2ef2 100644 --- a/docs/source/deployment/docker.md +++ b/docs/source/deployment/docker.md @@ -19,6 +19,18 @@ $ docker run --runtime nvidia --gpus all \ --model mistralai/Mistral-7B-v0.1 ``` +This image can also be used with other container engines such as [Podman](https://podman.io/). + +```console +$ podman run --gpus all \ + -v ~/.cache/huggingface:/root/.cache/huggingface \ + --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \ + -p 8000:8000 \ + --ipc=host \ + vllm/vllm-openai:latest \ + --model mistralai/Mistral-7B-v0.1 +``` + You can add any other you need after the image tag (`vllm/vllm-openai:latest`). :::{note}