comfyanonymous
6f18daaa58
Disable omnigen2 fp16 on older pytorch versions. ( #8672 )
2025-06-26 03:39:09 -04:00
comfyanonymous
e0c3ebe52b
Apple most likely is never fixing the fp16 attention bug. ( #8485 )
2025-06-10 13:06:24 -04:00
comfyanonymous
9771d0edb8
Enable AMD fp8 and pytorch attention on some GPUs. ( #8474 )
...
Information is from the pytorch source code.
2025-06-09 12:50:39 -04:00
comfyanonymous
c51b6cba7d
Enable fp8 ops by default on gfx1201 ( #8464 )
2025-06-08 14:15:34 -04:00
comfyanonymous
bf5d00fba0
Cleaner torch version comparisons. ( #8453 )
2025-06-07 10:01:15 -04:00
comfyanonymous
7bcf2fdbc5
Put ROCm version in tuple to make it easier to enable stuff based on it. ( #8348 )
2025-05-30 15:41:02 -04:00
comfyanonymous
f8a2632e3a
Disable initial GPU load when novram is used. ( #8294 )
2025-05-26 16:39:27 -04:00
comfyanonymous
f2862bd3ab
Enable pytorch attention by default on AMD gfx1151 ( #8282 )
2025-05-26 04:29:25 -04:00
comfyanonymous
30ecd213fb
Add argument to explicitly enable fp8 compute support. ( #8257 )
...
This can be used to test if your current GPU/pytorch version supports fp8 matrix mult in combination with --fast or the fp8_e4m3fn_fast dtype.
2025-05-23 17:43:50 -04:00
comfyanonymous
0280886398
Per device stream counters for async offload. ( #7873 )
2025-04-29 20:28:52 -04:00
comfyanonymous
44ddce3a45
Fix stream priority to support older pytorch. ( #7856 )
2025-04-28 13:07:21 -04:00
comfyanonymous
520b3c22e3
Use stream for casting if enabled. ( #7833 )
2025-04-27 05:38:11 -04:00
comfyanonymous
0782adba34
Add experimental --async-offload lowvram weight offloading. ( #7820 )
...
This should speed up the lowvram mode a bit. It currently is only enabled when --async-offload is used but it will be enabled by default in the future if there are no problems.
2025-04-26 16:11:21 -04:00
comfyanonymous
27d0247321
Add option for using fp8_e8m0fnu for model weights. ( #7733 )
...
Seems to break every model I have tried but worth testing?
2025-04-22 06:17:38 -04:00
BiologicalExplosion
933ba105ff
MLU memory optimization ( #7470 )
...
Co-authored-by: huzhan <huzhan@cambricon.com>
2025-04-02 19:24:04 -04:00
BVH
d7ab5c10ce
Add option to store TE in bf16 ( #7461 )
2025-04-01 13:48:53 -04:00
comfyanonymous
0097c4d380
Support more float8 types.
2025-03-25 05:23:49 -04:00
FeepingCreature
d466fbcae5
Add --use-flash-attention flag. ( #7223 )
...
* Add --use-flash-attention flag.
This is useful on AMD systems, as FA builds are still 10% faster than Pytorch cross-attention.
2025-03-14 03:22:41 -04:00
comfyanonymous
8a845e4e3e
Fix.
2025-03-13 15:03:18 -04:00
comfyanonymous
7385b43e8e
Print mac version.
2025-03-13 10:05:40 -04:00
comfyanonymous
b1810e9ff6
Fix stable cascade VAE on some lowvram machines.
2025-03-08 20:24:04 -05:00
Chenlei Hu
ba586589cd
Use enum list for --fast options ( #7024 )
2025-03-01 02:37:35 -05:00
comfyanonymous
b421f6553d
--fast now takes a number as argument to indicate how fast you want it.
...
The idea is that you can indicate how much quality vs speed you want.
At the moment:
--fast 2 enables fp16 accumulation if your pytorch supports it.
--fast 5 enables fp8 matrix mult on fp8 models and the optimization above.
--fast without a number enables all optimizations.
2025-02-28 02:48:20 -05:00
comfyanonymous
8c85daeccc
Use fp16 for intermediate for fp8 weights with --fast if supported.
2025-02-28 02:17:50 -05:00
comfyanonymous
eedeb71616
Use fp16 if checkpoint weights are fp16 and the model supports it.
2025-02-27 16:39:57 -05:00
BiologicalExplosion
d019ddaf95
Support Cambricon MLU ( #6964 )
...
Co-authored-by: huzhan <huzhan@cambricon.com>
2025-02-26 20:45:13 -05:00
comfyanonymous
114be728b1
Speedup on some models by not upcasting bfloat16 to float32 on mac.
2025-02-24 05:41:32 -05:00
comfyanonymous
8a7f84bbd2
Prioritize fp16 compute when using allow_fp16_accumulation
2025-02-23 04:45:54 -05:00
comfyanonymous
67d7eeb002
Assume the mac black image bug won't be fixed before v16.
2025-02-21 20:24:07 -05:00
comfyanonymous
58a37f57b0
Latest mac still has the black image bug.
2025-02-21 20:14:30 -05:00
comfyanonymous
6ecbdae55b
Let all model memory be offloaded on nvidia.
2025-02-21 06:32:21 -05:00
comfyanonymous
de308facd5
Apparently directml supports fp16.
2025-02-20 09:30:24 -05:00
comfyanonymous
aa50906886
Fix typo.
...
Let me know if this slows things down on 2000 series and below.
2025-02-18 07:28:33 -05:00
comfyanonymous
4e37ad25fd
Improve AMD arch detection.
2025-02-17 04:53:40 -05:00
comfyanonymous
4f1809faf8
bf16 manual cast works on old AMD.
2025-02-17 04:42:40 -05:00
comfyanonymous
e9c105523f
Refactor torch version checks to be more future proof.
2025-02-17 04:36:45 -05:00
comfyanonymous
b02aa6ea50
Disable bf16 on AMD GPUs that don't support it.
2025-02-16 05:46:10 -05:00
comfyanonymous
539bc3e2e0
Disable pytorch attention in VAE for AMD.
2025-02-14 05:42:14 -05:00
comfyanonymous
98c0df4724
Auto enable mem efficient attention on gfx1100 on pytorch nightly 2.7
...
I'm not not sure which arches are supported yet. If you see improvements in
memory usage while using --use-pytorch-cross-attention on your AMD GPU let
me know and I will add it to the list.
2025-02-14 04:18:14 -05:00
comfyanonymous
1f657be9a7
Better memory estimation for ROCm that support mem efficient attention.
...
There is no way to check if the card actually supports it so it assumes
that it does if you use --use-pytorch-cross-attention with yours.
2025-02-13 08:32:36 -05:00
comfyanonymous
66d579cd19
Fix ruff.
2025-02-12 06:49:16 -05:00
zhoufan2956
0f7eb12d62
mix_ascend_bf16_infer_err ( #6794 )
2025-02-12 06:48:11 -05:00
HishamC
f49d8d3d1e
Fix for running via DirectML ( #6542 )
...
* Fix for running via DirectML
Fix DirectML empty image generation issue with Flux1. add CPU fallback for unsupported path. Verified the model works on AMD GPUs
* fix formating
* update casual mask calculation
2025-02-11 17:11:32 -05:00
comfyanonymous
a7e5e662ab
Make --force-fp16 actually force the diffusion model to be fp16.
2025-02-11 08:33:09 -05:00
catboxanon
ca2a621e0b
Allow FP16 accumulation with --fast ( #6453 )
...
Currently only applies to PyTorch nightly releases. (>=20250208)
2025-02-08 17:00:56 -05:00
comfyanonymous
5da1230892
Lower minimum ratio of loaded weights on Nvidia.
2025-01-27 05:26:51 -05:00
comfyanonymous
00e27c6202
Remove redundant code.
2025-01-25 19:04:53 -05:00
comfyanonymous
3e9956ca05
Remove old unused function.
2025-01-04 07:20:54 -05:00
comfyanonymous
7ce8ef2b20
Clear cache as often on AMD as Nvidia.
...
I think the issue this was working around has been solved.
If you notice that this change slows things down or causes stutters on
your AMD GPU with ROCm on Linux please report it.
2025-01-02 08:44:16 -05:00
comfyanonymous
9ed17efbb7
Use python 3.9 in launch test instead of 3.8
...
Fix ruff check.
2024-12-26 20:05:54 -05:00
