xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 10:30:37 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916)

Browse Source 
Signed-off-by: Jin Huang <jinhun@amazon.com>
Co-authored-by: Jin Huang <jinhun@amazon.com>
This commit is contained in:
Jin Huang 2025-05-13 03:10:07 -04:00 committed by GitHub
parent f0d610a8ae
commit 8dd0671bac
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/v1/worker/gpu_model_runner.py
View File

@ -1107,7 +1107,7 @@ class GPUModelRunner(LoRAModelRunnerMixin):
else: else:
mm_embeds = [] mm_embeds = []
if self.is_multimodal_model: if self.is_multimodal_model and get_pp_group().is_first_rank:
# NOTE(woosuk): To unify token ids and soft tokens (vision # NOTE(woosuk): To unify token ids and soft tokens (vision
# embeddings), we always use embeddings (rather than token ids) # embeddings), we always use embeddings (rather than token ids)
# as input to the multimodal model, even when the input is text. # as input to the multimodal model, even when the input is text.