xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-01-23 18:54:33 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Hardware][CPU][PowerPC]Disable torch.compile() in toptopk sampling (#26987)

Browse Source 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
This commit is contained in:
Akash kaothalkar 2025-10-16 11:06:59 +05:30 committed by GitHub
parent 76f0d05bc6
commit f7d318de2b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/v1/sample/ops/topk_topp_sampler.py
View File

@ -65,7 +65,11 @@ class TopKTopPSampler(nn.Module):
)
self.forward = self.forward_native
elif current_platform.is_cpu():
if current_platform.get_cpu_architecture() == CpuArchEnum.RISCV:
arch = current_platform.get_cpu_architecture()
# Fall back to native implementation for POWERPC and RISCV.
# On PowerPC argmax produces incorrect output with torch.compile.
# PR: https://github.com/vllm-project/vllm/pull/26987
if arch in (CpuArchEnum.RISCV, CpuArchEnum.POWERPC):
self.forward = self.forward_native
else:
self.forward = self.forward_cpu