xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 03:15:20 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: fix langchain (#2736)

Browse Source
This commit is contained in:
Massimiliano Pronesti 2024-02-04 03:17:55 +01:00 committed by GitHub
parent 4abf6336ec
commit 5ed704ec8c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/source/serving/serving_with_langchain.rst
View File

@ -9,13 +9,13 @@ To install langchain, run
.. code-block:: console .. code-block:: console
$ pip install langchain -q $ pip install langchain langchain_community -q
To run inference on a single or multiple GPUs, use ``VLLM`` class from ``langchain``. To run inference on a single or multiple GPUs, use ``VLLM`` class from ``langchain``.
.. code-block:: python .. code-block:: python
from langchain.llms import VLLM from langchain_community.llms import VLLM
llm = VLLM(model="mosaicml/mpt-7b", llm = VLLM(model="mosaicml/mpt-7b",
trust_remote_code=True, # mandatory for hf models trust_remote_code=True, # mandatory for hf models
@ -28,4 +28,4 @@ To run inference on a single or multiple GPUs, use ``VLLM`` class from ``langcha
print(llm("What is the capital of France ?")) print(llm("What is the capital of France ?"))
Please refer to this `Tutorial <https://github.com/langchain-ai/langchain/blob/master/docs/docs/integrations/llms/vllm.ipynb>`_ for more details. Please refer to this `Tutorial <https://python.langchain.com/docs/integrations/llms/vllm>`_ for more details.