xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-10 00:47:15 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Doc] Installed version of llmcompressor for int8/fp8 quantization (#11103)

Browse Source 
Signed-off-by: Guangda Liu <bingps@users.noreply.github.com>
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
This commit is contained in:
bingps 2024-12-11 23:43:24 +08:00 committed by GitHub
parent b2f775456e
commit fd22220687
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/quantization/fp8.rst
View File

@ -45,7 +45,7 @@ To produce performant FP8 quantized models with vLLM, you'll need to install the
.. code-block:: console .. code-block:: console
$ pip install llmcompressor==0.1.0 $ pip install llmcompressor
Quantization Process Quantization Process
-------------------- --------------------

4
docs/source/quantization/int8.rst
View File

@ -19,7 +19,7 @@ To use INT8 quantization with vLLM, you'll need to install the `llm-compressor <
.. code-block:: console .. code-block:: console
$ pip install llmcompressor==0.1.0 $ pip install llmcompressor
Quantization Process Quantization Process
-------------------- --------------------
@ -142,4 +142,4 @@ Best Practices
Troubleshooting and Support Troubleshooting and Support
--------------------------- ---------------------------
If you encounter any issues or have feature requests, please open an issue on the ``vllm-project/llm-compressor`` GitHub repository. If you encounter any issues or have feature requests, please open an issue on the ``vllm-project/llm-compressor`` GitHub repository.