xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-13 16:00:19 +08:00
Code Issues Packages Projects Releases Wiki Activity

[bugfix][quantization] Fix fp8 per_tensor scale shape (#30257)

Browse Source 
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
This commit is contained in:
haoyangli-amd 2025-12-09 19:28:50 +08:00 committed by GitHub
parent c72ea10723
commit 03416eada6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/_custom_ops.py
View File

@ -1726,7 +1726,7 @@ def scaled_fp8_quant(
output, input, scale, scale_ub output, input, scale, scale_ub
) )
else: else:
scale = torch.empty((1, 1), device=input.device, dtype=torch.float32) scale = torch.empty(1, device=input.device, dtype=torch.float32)
torch.ops._C.dynamic_scaled_fp8_quant(output, input, scale) torch.ops._C.dynamic_scaled_fp8_quant(output, input, scale)
else: else:
assert scale.numel() == 1, f"{scale.shape}" assert scale.numel() == 1, f"{scale.shape}"