comfyanonymous
0280886398
Per device stream counters for async offload. ( #7873 )
2025-04-29 20:28:52 -04:00
comfyanonymous
44ddce3a45
Fix stream priority to support older pytorch. ( #7856 )
2025-04-28 13:07:21 -04:00
comfyanonymous
520b3c22e3
Use stream for casting if enabled. ( #7833 )
2025-04-27 05:38:11 -04:00
comfyanonymous
0782adba34
Add experimental --async-offload lowvram weight offloading. ( #7820 )
...
This should speed up the lowvram mode a bit. It currently is only enabled when --async-offload is used but it will be enabled by default in the future if there are no problems.
2025-04-26 16:11:21 -04:00
comfyanonymous
27d0247321
Add option for using fp8_e8m0fnu for model weights. ( #7733 )
...
Seems to break every model I have tried but worth testing?
2025-04-22 06:17:38 -04:00
BiologicalExplosion
933ba105ff
MLU memory optimization ( #7470 )
...
Co-authored-by: huzhan <huzhan@cambricon.com>
2025-04-02 19:24:04 -04:00
BVH
d7ab5c10ce
Add option to store TE in bf16 ( #7461 )
2025-04-01 13:48:53 -04:00
comfyanonymous
0097c4d380
Support more float8 types.
2025-03-25 05:23:49 -04:00
FeepingCreature
d466fbcae5
Add --use-flash-attention flag. ( #7223 )
...
* Add --use-flash-attention flag.
This is useful on AMD systems, as FA builds are still 10% faster than Pytorch cross-attention.
2025-03-14 03:22:41 -04:00
comfyanonymous
8a845e4e3e
Fix.
2025-03-13 15:03:18 -04:00
comfyanonymous
7385b43e8e
Print mac version.
2025-03-13 10:05:40 -04:00
comfyanonymous
b1810e9ff6
Fix stable cascade VAE on some lowvram machines.
2025-03-08 20:24:04 -05:00
Chenlei Hu
ba586589cd
Use enum list for --fast options ( #7024 )
2025-03-01 02:37:35 -05:00
comfyanonymous
b421f6553d
--fast now takes a number as argument to indicate how fast you want it.
...
The idea is that you can indicate how much quality vs speed you want.
At the moment:
--fast 2 enables fp16 accumulation if your pytorch supports it.
--fast 5 enables fp8 matrix mult on fp8 models and the optimization above.
--fast without a number enables all optimizations.
2025-02-28 02:48:20 -05:00
comfyanonymous
8c85daeccc
Use fp16 for intermediate for fp8 weights with --fast if supported.
2025-02-28 02:17:50 -05:00
comfyanonymous
eedeb71616
Use fp16 if checkpoint weights are fp16 and the model supports it.
2025-02-27 16:39:57 -05:00
BiologicalExplosion
d019ddaf95
Support Cambricon MLU ( #6964 )
...
Co-authored-by: huzhan <huzhan@cambricon.com>
2025-02-26 20:45:13 -05:00
comfyanonymous
114be728b1
Speedup on some models by not upcasting bfloat16 to float32 on mac.
2025-02-24 05:41:32 -05:00
comfyanonymous
8a7f84bbd2
Prioritize fp16 compute when using allow_fp16_accumulation
2025-02-23 04:45:54 -05:00
comfyanonymous
67d7eeb002
Assume the mac black image bug won't be fixed before v16.
2025-02-21 20:24:07 -05:00
comfyanonymous
58a37f57b0
Latest mac still has the black image bug.
2025-02-21 20:14:30 -05:00
comfyanonymous
6ecbdae55b
Let all model memory be offloaded on nvidia.
2025-02-21 06:32:21 -05:00
comfyanonymous
de308facd5
Apparently directml supports fp16.
2025-02-20 09:30:24 -05:00
comfyanonymous
aa50906886
Fix typo.
...
Let me know if this slows things down on 2000 series and below.
2025-02-18 07:28:33 -05:00
comfyanonymous
4e37ad25fd
Improve AMD arch detection.
2025-02-17 04:53:40 -05:00
comfyanonymous
4f1809faf8
bf16 manual cast works on old AMD.
2025-02-17 04:42:40 -05:00
comfyanonymous
e9c105523f
Refactor torch version checks to be more future proof.
2025-02-17 04:36:45 -05:00
comfyanonymous
b02aa6ea50
Disable bf16 on AMD GPUs that don't support it.
2025-02-16 05:46:10 -05:00
comfyanonymous
539bc3e2e0
Disable pytorch attention in VAE for AMD.
2025-02-14 05:42:14 -05:00
comfyanonymous
98c0df4724
Auto enable mem efficient attention on gfx1100 on pytorch nightly 2.7
...
I'm not not sure which arches are supported yet. If you see improvements in
memory usage while using --use-pytorch-cross-attention on your AMD GPU let
me know and I will add it to the list.
2025-02-14 04:18:14 -05:00
comfyanonymous
1f657be9a7
Better memory estimation for ROCm that support mem efficient attention.
...
There is no way to check if the card actually supports it so it assumes
that it does if you use --use-pytorch-cross-attention with yours.
2025-02-13 08:32:36 -05:00
comfyanonymous
66d579cd19
Fix ruff.
2025-02-12 06:49:16 -05:00
zhoufan2956
0f7eb12d62
mix_ascend_bf16_infer_err ( #6794 )
2025-02-12 06:48:11 -05:00
HishamC
f49d8d3d1e
Fix for running via DirectML ( #6542 )
...
* Fix for running via DirectML
Fix DirectML empty image generation issue with Flux1. add CPU fallback for unsupported path. Verified the model works on AMD GPUs
* fix formating
* update casual mask calculation
2025-02-11 17:11:32 -05:00
comfyanonymous
a7e5e662ab
Make --force-fp16 actually force the diffusion model to be fp16.
2025-02-11 08:33:09 -05:00
catboxanon
ca2a621e0b
Allow FP16 accumulation with --fast ( #6453 )
...
Currently only applies to PyTorch nightly releases. (>=20250208)
2025-02-08 17:00:56 -05:00
comfyanonymous
5da1230892
Lower minimum ratio of loaded weights on Nvidia.
2025-01-27 05:26:51 -05:00
comfyanonymous
00e27c6202
Remove redundant code.
2025-01-25 19:04:53 -05:00
comfyanonymous
3e9956ca05
Remove old unused function.
2025-01-04 07:20:54 -05:00
comfyanonymous
7ce8ef2b20
Clear cache as often on AMD as Nvidia.
...
I think the issue this was working around has been solved.
If you notice that this change slows things down or causes stutters on
your AMD GPU with ROCm on Linux please report it.
2025-01-02 08:44:16 -05:00
comfyanonymous
9ed17efbb7
Use python 3.9 in launch test instead of 3.8
...
Fix ruff check.
2024-12-26 20:05:54 -05:00
Huazhong Ji
c4c9608fb7
Support ascend npu ( #5436 )
...
* support ascend npu
Co-authored-by: YukMingLaw <lymmm2@163.com>
Co-authored-by: starmountain1997 <guozr1997@hotmail.com>
Co-authored-by: Ginray <ginray0215@gmail.com>
2024-12-26 19:36:50 -05:00
comfyanonymous
2eaf854b41
Cleanup some mac related code.
2024-12-25 05:32:51 -05:00
comfyanonymous
f765a81411
Disable bfloat16 on older mac.
2024-12-25 05:18:50 -05:00
comfyanonymous
5910728742
Clean up the VAE dtypes code.
2024-12-25 04:50:34 -05:00
comfyanonymous
6b963ebdc5
Add a try except block so if torch version is weird it won't crash.
2024-12-23 03:22:48 -05:00
Simon Lui
668cc89307
Add oneAPI device selector for xpu and some other changes. ( #6112 )
...
* Add oneAPI device selector and some other minor changes.
* Fix device selector variable name.
* Flip minor version check sign.
* Undo changes to README.md.
2024-12-23 03:18:32 -05:00
comfyanonymous
e7747a1e35
Make --novram completely offload weights.
...
This flag is mainly used for testing the weight offloading, it shouldn't
actually be used in practice.
Remove useless import.
2024-12-23 01:51:08 -05:00
comfyanonymous
12f0beabf0
Relax minimum ratio of weights loaded in memory on nvidia.
...
This should make it possible to do higher res images/longer videos by
further offloading weights to CPU memory.
Please report an issue if this slows down things on your system.
2024-12-22 03:06:37 -05:00
Chenlei Hu
917717a0a4
Replace print with logging ( #6138 )
...
* Replace print with logging
* nit
* nit
* nit
* nit
* nit
* nit
2024-12-20 16:24:55 -05:00
