xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-18 10:16:58 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix gemma3 results all zero (#17364)

Browse Source 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
This commit is contained in:
Qiming Zhang 2025-04-29 09:40:25 -07:00 committed by GitHub
parent a39203f99e
commit d3cf61b89b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
vllm/model_executor/layers/layernorm.py
View File

@ -241,7 +241,10 @@ class GemmaRMSNorm(CustomOp):
"""PyTorch-native implementation equivalent to forward().""" """PyTorch-native implementation equivalent to forward()."""
orig_dtype = x.dtype orig_dtype = x.dtype
if residual is not None: if residual is not None:
x = x + residual if orig_dtype == torch.float16:
x = x + residual.float()
else:
x = x + residual
residual = x residual = x
x = x.float() x = x.float()