xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-09 11:37:10 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Kernel] Revert the API change of Attention.forward (#12038)

Browse Source 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
This commit is contained in:
Chen Zhang 2025-01-14 20:59:32 +08:00 committed by GitHub
parent bb354e6b2d
commit 1f18adb245
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/attention/layer.py
View File

@ -134,8 +134,8 @@ class Attention(nn.Module):
query: torch.Tensor, query: torch.Tensor,
key: torch.Tensor, key: torch.Tensor,
value: torch.Tensor, value: torch.Tensor,
_kv_cache: torch.Tensor, kv_cache: torch.Tensor,
_attn_metadata: AttentionMetadata, attn_metadata: AttentionMetadata,
) -> torch.Tensor: ) -> torch.Tensor:
if self.use_output: if self.use_output:
output = torch.empty_like(query) output = torch.empty_like(query)