From 49d849b3ab7aa6ae493ccde1d85d226833f73fbb Mon Sep 17 00:00:00 2001 From: Yuan Tang Date: Fri, 1 Mar 2024 14:04:14 -0500 Subject: [PATCH] docs: Add tutorial on deploying vLLM model with KServe (#2586) Signed-off-by: Yuan Tang --- docs/source/index.rst | 1 + docs/source/serving/deploying_with_kserve.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 docs/source/serving/deploying_with_kserve.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index 32929257661a..bdc541cb2d58 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -70,6 +70,7 @@ Documentation serving/distributed_serving serving/run_on_sky + serving/deploying_with_kserve serving/deploying_with_triton serving/deploying_with_docker serving/serving_with_langchain diff --git a/docs/source/serving/deploying_with_kserve.rst b/docs/source/serving/deploying_with_kserve.rst new file mode 100644 index 000000000000..7f22766e09ae --- /dev/null +++ b/docs/source/serving/deploying_with_kserve.rst @@ -0,0 +1,8 @@ +.. _deploying_with_kserve: + +Deploying with KServe +============================ + +vLLM can be deployed with `KServe `_ on Kubernetes for highly scalable distributed model serving. + +Please see `this guide `_ for more details on using vLLM with KServe.