xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-06-07 01:42:17 +08:00
Code Issues Packages Projects Releases Wiki Activity

[ROCm][Bugfix] Add MLACommonMetadata to allowed attention types for speculative decoding (#30430)

Browse Source 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
Andreas Karatzas 2025-12-11 13:25:01 -06:00 committed by GitHub
parent 0e71eaa644
commit 72aaac5b66
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/v1/spec_decode/eagle.py
View File

@ -178,6 +178,12 @@ class EagleProposer:
) )
rocm_types.append(AiterFlashAttentionMetadata) rocm_types.append(AiterFlashAttentionMetadata)
# TRITON_MLA backend support for MLA models (e.g., DeepSeek)
from vllm.v1.attention.backends.mla.common import MLACommonMetadata
rocm_types.append(MLACommonMetadata)
self.allowed_attn_types = tuple(rocm_types) self.allowed_attn_types = tuple(rocm_types)
# Parse the speculative token tree. # Parse the speculative token tree.