xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-24 02:44:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

Improve validation of TP in Transformers backend (#15540)

Browse Source 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Harry Mellor 2025-03-26 14:26:48 +00:00 committed by GitHub
parent 1aa162e030
commit c091c0a588
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
vllm/model_executor/models/transformers.py
View File

@ -229,7 +229,10 @@ class TransformersModel(nn.Module):
Apply the model's tensor parallelization plan. Apply the model's tensor parallelization plan.
Currently only supports linear layers. Currently only supports linear layers.
""" """
if self.tp_size > 1 and self.config.base_model_tp_plan is None: if not self.model.supports_tp_plan:
if self.tp_size <= 1:
return
raise ValueError( raise ValueError(
f"{type(self.model)} does not support tensor parallel yet!") f"{type(self.model)} does not support tensor parallel yet!")