xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 10:46:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add documentation to Triton server tutorial (#983)

Browse Source
This commit is contained in:
Tanmay Verma 2023-09-20 10:32:40 -07:00 committed by GitHub
parent bc0644574c
commit 6f2dd6c37e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 7 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/source/index.rst
View File

@ -64,6 +64,7 @@ Documentation
serving/distributed_serving
serving/run_on_sky
serving/deploying_with_triton
.. toctree::
:maxdepth: 1

6
docs/source/serving/deploying_with_triton.rst Normal file
View File

@ -0,0 +1,6 @@
.. _deploying_with_triton:
Deploying with NVIDIA Triton
============================
The `Triton Inference Server <https://github.com/triton-inference-server>`_ hosts a tutorial demonstrating how to quickly deploy a simple `facebook/opt-125m <https://huggingface.co/facebook/opt-125m>`_ model using vLLM. Please see `Deploying a vLLM model in Triton <https://github.com/triton-inference-server/tutorials/blob/main/Quick_Deploy/vLLM/README.md#deploying-a-vllm-model-in-triton>`_ for more details.