xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 16:35:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Doc]: Update tensorizer docs to include vllm[tensorizer] (#7889)

Browse Source 
Co-authored-by: Kaunil Dhruv <dhruv.kaunil@gmail.com>
This commit is contained in:
Seth Kimmel 2024-10-22 15:43:25 -07:00 committed by GitHub
parent b17046e298
commit 208cb34c81
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/source/serving/tensorizer.rst
View File

@ -9,4 +9,7 @@ shorter Pod startup times and CPU memory usage. Tensor encryption is also suppor
For more information on CoreWeave's Tensorizer, please refer to
`CoreWeave's Tensorizer documentation <https://github.com/coreweave/tensorizer>`_. For more information on serializing a vLLM model, as well a general usage guide to using Tensorizer with vLLM, see
the `vLLM example script <https://docs.vllm.ai/en/stable/getting_started/examples/tensorize_vllm_model.html>`_.
the `vLLM example script <https://docs.vllm.ai/en/stable/getting_started/examples/tensorize_vllm_model.html>`_.
.. note::
Note that to use this feature you will need to install `tensorizer` by running `pip install vllm[tensorizer]`.