xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-16 06:35:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

[BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514)

Browse Source 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
This commit is contained in:
Varun Sundar Rabindranath 2025-06-12 00:28:12 -04:00 committed by GitHub
parent 2f1c19b245
commit e5d35d62f5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
vllm/model_executor/layers/quantization/utils/fp8_utils.py
View File

@ -143,6 +143,7 @@ def apply_w8a8_block_fp8_linear(
column_major_scales=True, column_major_scales=True,
) )
import vllm.model_executor.layers.quantization.deepgemm # noqa: F401
output = torch.ops.vllm.w8a8_block_fp8_matmul_deepgemm( output = torch.ops.vllm.w8a8_block_fp8_matmul_deepgemm(
q_input, q_input,
weight, weight,