shawnington 8299ebdaae
Update attention.py to respect FORCE_UPCAST_ATTENTION_DTYPE
Fixed attention precision not being cast in attention_pytorch and others functions. This led to the functions not being able to respect the `--dont-upcast-attention` flag.  

change calls to `.float()` to `.to(dtype=torch.float32)` in several locations, as it profiles much faster. 

removed unneeded check for `attn_precision == torch.float32`, as the change from `.float()` to `.to(dtype=cast_to_type)` does not cast or copy if `tensor.dtype == cast_to_type`
2024-06-03 08:21:35 -07:00
..
2024-05-18 10:11:44 -04:00
2023-01-16 22:37:14 -05:00
2024-01-14 17:28:31 -05:00
2023-01-16 22:37:14 -05:00
2023-12-24 07:13:18 -05:00