xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 07:34:57 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Docs] Add Modal to deployment frameworks (#11907)

Browse Source
This commit is contained in:
Charles Frye 2025-01-09 15:26:37 -08:00 committed by GitHub
parent 9a228348d2
commit 36f5303578
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 9 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/deployment/frameworks/bentoml.md
View File

@ -2,6 +2,6 @@
# BentoML
[BentoML](https://github.com/bentoml/BentoML) allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. You can serve the model locally or containerize it as an OCI-complicant image and deploy it on Kubernetes.
[BentoML](https://github.com/bentoml/BentoML) allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. You can serve the model locally or containerize it as an OCI-compliant image and deploy it on Kubernetes.
For details, see the tutorial [vLLM inference in the BentoML documentation](https://docs.bentoml.com/en/latest/use-cases/large-language-models/vllm.html).

1
docs/source/deployment/frameworks/index.md
View File

@ -8,6 +8,7 @@ cerebrium
dstack
helm
lws
modal
skypilot
triton
```

7
docs/source/deployment/frameworks/modal.md Normal file
View File

@ -0,0 +1,7 @@
(deployment-modal)=
# Modal
vLLM can be run on cloud GPUs with [Modal](https://modal.com), a serverless computing platform designed for fast auto-scaling.
For details on how to deploy vLLM on Modal, see [this tutorial in the Modal documentation](https://modal.com/docs/examples/vllm_inference).