comfyanonymous
8dc45adbe4
Add: --use-sage-attention to enable SageAttention.
...
You need to have the library installed first.
2024-12-18 01:56:10 -05:00
Chenlei Hu
efca88cfb3
Lint all unused variables ( #5989 )
...
* Enable F841
* Autofix
* Remove all unused variable assignment
2024-12-12 17:59:16 -05:00
comfyanonymous
159848ad06
Set initial load devices for te and model to mps device on mac.
2024-12-12 06:00:31 -05:00
comfyanonymous
5086936750
Fix case where a memory leak could cause crash.
...
Now the only symptom of code messing up and keeping references to a model
object when it should not will be endless prints in the log instead of the
next workflow crashing ComfyUI.
2024-12-02 19:49:49 -05:00
comfyanonymous
1176023ebb
Improved memory management. ( #5450 )
...
* Less fragile memory management.
* Fix issue.
* Remove useless function.
* Prevent and detect some types of memory leaks.
* Run garbage collector when switching workflow if needed.
* Fix issue.
2024-12-02 14:39:34 -05:00
comfyanonymous
df553487ed
Add option to inference the diffusion model in fp32 and fp64.
2024-11-25 05:00:23 -05:00
comfyanonymous
c1473e99c8
Bump up mac version for attention upcast bug workaround.
2024-10-31 15:15:31 -04:00
comfyanonymous
adfa061b98
fp8 casting is fast on GPUs that support fp8 compute.
2024-10-20 00:54:47 -04:00
comfyanonymous
90105ed662
Use the lowvram cast_to function for everything.
2024-10-17 17:25:56 -04:00
Jonathan Avila
f792d1ce70
Increase maximum macOS version to 15.0.1 when forcing upcast attention ( #5191 )
2024-10-09 22:21:41 -04:00
comfyanonymous
6b0d97384c
Add a weight_dtype fp8_e4m3fn_fast to the Diffusion Model Loader node.
...
This is used to load weights in fp8 and use fp8 matrix multiplication.
2024-10-09 19:43:17 -04:00
comfyanonymous
5c7e72f3b1
Load controlnet in fp8 if weights are in fp8.
2024-09-21 04:50:12 -04:00
Simon Lui
c4affc7e35
Fix xpu Pytorch nightly build from calling optimize which doesn't exist. ( #4978 )
2024-09-19 05:11:42 -04:00
comfyanonymous
ebba1f2563
Prioritize freeing partially offloaded models first.
2024-09-04 19:47:32 -04:00
comfyanonymous
ac3c801c9d
Speed up inference on nvidia 10 series on Linux.
2024-09-01 17:29:31 -04:00
comfyanonymous
8e7d3547e4
Make minimum_inference_memory() depend on --reserve-vram
2024-09-01 01:18:34 -04:00
comfyanonymous
19cdff6796
Cleanup.
2024-08-30 12:53:59 -04:00
comfyanonymous
a80296ecb6
Fix case where model was not properly unloaded in merging workflows.
2024-08-27 19:03:51 -04:00
comfyanonymous
65a577e0db
Cleanup.
2024-08-23 04:06:27 -04:00
comfyanonymous
103d705247
Fix.
2024-08-23 04:04:55 -04:00
Simon Lui
24af9ec35a
Rework IPEX check for future inclusion of XPU into Pytorch upstream and do a bit more optimization of ipex.optimize(). ( #4562 )
2024-08-23 03:59:57 -04:00
comfyanonymous
23836e6596
Code cleanups.
2024-08-22 17:05:12 -04:00
comfyanonymous
7bad18b66d
Fix issue with models staying loaded in memory.
2024-08-22 15:58:20 -04:00
David
ff022fccc2
Generalize MacOS version check for force-upcast-attention ( #4548 )
...
This code automatically forces upcasting attention for MacOS versions 14.5 and 14.6. My computer returns the string "14.6.1" for `platform.mac_ver()[0]`, so this generalizes the comparison to catch more versions.
I am running MacOS Sonoma 14.6.1 (latest version) and was seeing black image generation on previously functional workflows after recent software updates. This PR solved the issue for me.
See comfyanonymous/ComfyUI#3521
2024-08-22 13:24:21 -04:00
comfyanonymous
190a03c892
fp16 is actually faster than fp32 on a GTX 1080.
2024-08-21 23:23:50 -04:00
comfyanonymous
883e5a9aa5
Fix slow performance on 10 series Nvidia GPUs.
2024-08-21 16:39:02 -04:00
comfyanonymous
d8af5d17b5
Remove useless line, adjust windows default reserved vram.
2024-08-21 00:47:19 -04:00
comfyanonymous
7c0c89e70c
Add --fast argument to enable experimental optimizations.
...
Optimizations that might break things/lower quality will be put behind
this flag first and might be enabled by default in the future.
Currently the only optimization is float8_e4m3fn matrix multiplication on
4000/ADA series Nvidia cards or later. If you have one of these cards you
will see a speed boost when using fp8_e4m3fn flux for example.
2024-08-20 11:55:51 -04:00
comfyanonymous
c3bec99a9b
Fix potential issue with multi devices.
2024-08-20 10:46:36 -04:00
comfyanonymous
5115d926eb
Add a --reserve-vram argument if you don't want comfy to use all of it.
...
--reserve-vram 1.0 for example will make ComfyUI try to keep 1GB vram free.
This can also be useful if workflows are failing because of OOM errors but
in that case please report it if --reserve-vram improves your situation.
2024-08-19 17:16:18 -04:00
comfyanonymous
48335ad8ca
Remove duplication.
2024-08-19 15:26:50 -04:00
comfyanonymous
2119417b9d
Fix bug when model cannot be partially unloaded.
2024-08-13 03:57:55 -04:00
comfyanonymous
a552dfba70
Fix some issues with TE being in lowvram mode.
2024-08-12 23:42:21 -04:00
comfyanonymous
5faeccb7f7
Memory tweaks.
2024-08-12 15:07:11 -04:00
comfyanonymous
d95ec784a4
Fix some potential issues with the previous commits.
2024-08-12 00:23:29 -04:00
comfyanonymous
089485a7c6
Load TE model straight to vram if certain conditions are met.
2024-08-11 23:52:43 -04:00
comfyanonymous
cbb1bde521
Fix some issues with inference slowing down.
2024-08-10 16:21:25 -04:00
comfyanonymous
1280396253
Fix regression.
2024-08-09 03:36:40 -04:00
comfyanonymous
114af84d12
Try to improve inference speed on some machines.
2024-08-08 17:29:27 -04:00
comfyanonymous
4f196c19a3
Fix.
2024-08-08 15:16:51 -04:00
comfyanonymous
ede5622146
Partial model shift support.
2024-08-08 14:45:06 -04:00
comfyanonymous
5a25225024
Make supported_dtypes a priority list.
2024-08-07 15:00:06 -04:00
comfyanonymous
001ff4d6a6
Fix OOMs happening in some cases.
...
A cloned model patcher sometimes reported a model was loaded on a device
when it wasn't.
2024-08-06 13:36:04 -04:00
comfyanonymous
5228fbf0ad
Unload models and load them back in lowvram mode no free vram.
2024-08-06 03:22:39 -04:00
comfyanonymous
f7183e2c73
Improve performance on some lowend GPUs.
2024-08-05 16:24:04 -04:00
comfyanonymous
10ad73d520
Fix crash.
2024-08-03 16:55:38 -04:00
comfyanonymous
e7287d2fc2
Tweak lowvram memory formula.
2024-08-03 16:44:50 -04:00
comfyanonymous
d4ca060482
Lower lowvram memory to 1/3 of free memory.
2024-08-03 15:14:07 -04:00
comfyanonymous
f27f0f8a48
Fix some issues.
2024-08-03 15:06:40 -04:00
comfyanonymous
e83d97d649
Cap lowvram to half of free memory.
2024-08-03 14:50:20 -04:00
