xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-24 11:11:17 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add reference

Browse Source
This commit is contained in:
Woosuk Kwon 2024-04-01 02:02:13 +00:00
parent 52a1e908e4
commit 86f073edd6
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/model_executor/models/tpu/gemma.py
View File

@ -1,5 +1,8 @@
"""Inference-only Gemma model compatible with HF weights. """Inference-only Gemma model compatible with HF weights.
Adapted from
https://github.com/google/gemma_pytorch/blob/main/gemma/model_xla.py
NOTE(woosuk): This is a temporary workaround to run the Gemma model using NOTE(woosuk): This is a temporary workaround to run the Gemma model using
PyTorch XLA. This should be merged into the main Gemma model implementation PyTorch XLA. This should be merged into the main Gemma model implementation
once the custom ops are refactored and the model becomes torch.compile-able. once the custom ops are refactored and the model becomes torch.compile-able.