xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-04-13 03:27:03 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Doc] add KubeAI to serving integrations (#10837)

Browse Source 
Signed-off-by: Sam Stoelinga <sammiestoel@gmail.com>
This commit is contained in:
Sam Stoelinga 2024-12-06 09:03:56 -08:00 committed by GitHub
parent 8b59631855
commit 7406274041
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
docs/source/serving/deploying_with_kubeai.rst Normal file
View File

@ -0,0 +1,17 @@
.. _deploying_with_kubeai:
Deploying with KubeAI
=====================
`KubeAI <https://github.com/substratusai/kubeai>`_ is a Kubernetes operator that enables you to deploy and manage AI models on Kubernetes. It provides a simple and scalable way to deploy vLLM in production. Functionality such as scale-from-zero, load based autoscaling, model caching, and much more is provided out of the box with zero external dependencies.
Please see the Installation Guides for environment specific instructions:
* `Any Kubernetes Cluster <https://www.kubeai.org/installation/any/>`_
* `EKS <https://www.kubeai.org/installation/eks/>`_
* `GKE <https://www.kubeai.org/installation/gke/>`_
Once you have KubeAI installed, you can
`configure text generation models <https://www.kubeai.org/how-to/configure-text-generation-models/>`_
using vLLM.

1
docs/source/serving/integrations.rst
View File

@ -6,6 +6,7 @@ Integrations
run_on_sky
deploying_with_kserve
deploying_with_kubeai
deploying_with_triton
deploying_with_bentoml
deploying_with_cerebrium