xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 23:15:31 +08:00
Code Issues Packages Projects Releases Wiki Activity

[doc] add missing import (#18133)

Browse Source 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
This commit is contained in:
Reid 2025-05-14 18:57:34 +08:00 committed by GitHub
parent d62a076e84
commit 9ccc6ded42
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/serving/offline_inference.md
View File

@ -74,6 +74,8 @@ Tensor parallelism (`tensor_parallel_size` option) can be used to split the mode
The following code splits the model across 2 GPUs. The following code splits the model across 2 GPUs.
```python ```python
from vllm import LLM
llm = LLM(model="ibm-granite/granite-3.1-8b-instruct", llm = LLM(model="ibm-granite/granite-3.1-8b-instruct",
tensor_parallel_size=2) tensor_parallel_size=2)
``` ```