Rattus 5a7e929849 Account for dequantization and type-casts in offload costs
When measuring the cost of offload, identify weights that need a type
change or dequantization and add the size of the conversion result
to the offload cost.

This is mutually exclusive with lowvram patches which already has
a large conservative estimate and wont overlap the dequant cost so\
dont double count.
2025-12-09 08:55:38 +10:00
..
2024-06-27 18:43:11 -04:00
2025-11-28 19:40:19 -05:00
2025-09-02 15:36:22 -04:00
2025-01-24 06:15:54 -05:00
2025-07-06 07:07:39 -04:00
2025-10-25 23:07:29 -04:00
2025-12-05 22:20:22 -05:00
2025-12-05 23:01:19 -05:00