comfyanonymous
511e70ab62
Fix attention mask batch size in some attention functions.
2024-02-17 15:22:21 -05:00
comfyanonymous
4a9216bca3
Properly fix attention masks in CLIP with batches.
2024-02-17 12:13:13 -05:00
comfyanonymous
1cec78c95e
Don't use numpy for calculating sigmas.
2024-02-07 18:52:51 -05:00
comfyanonymous
50fb52bc5b
Remove some unused imports.
2024-01-25 23:42:37 -05:00
comfyanonymous
30b062a477
Make unclip more deterministic.
...
Pass a seed argument note that this might make old unclip images different.
2024-01-14 17:28:31 -05:00
comfyanonymous
62a7bf2919
Use basic attention implementation for small inputs on old pytorch.
2024-01-09 13:46:52 -05:00
comfyanonymous
8cf0a2dfe7
Update optimized_attention_for_device function for new functions that
...
support masked attention.
2024-01-07 13:52:08 -05:00
comfyanonymous
f9f0822512
Add attention mask support to sub quad attention.
2024-01-07 04:13:58 -05:00
comfyanonymous
59259b0de3
Support attention mask in split attention.
2024-01-06 13:16:48 -05:00
comfyanonymous
1240b45932
Implement attention mask on xformers.
2024-01-06 04:33:03 -05:00
comfyanonymous
a36114ce30
Implement noise augmentation for SD 4X upscale model.
2024-01-03 14:27:11 -05:00
comfyanonymous
d41cf1e412
Remove useless code.
2024-01-02 01:50:29 -05:00
comfyanonymous
09dfd709f2
Fix lowvram mode not working with unCLIP and Revision code.
2023-12-26 05:02:02 -05:00
comfyanonymous
af134f6c64
Fix SVD lowvram mode.
2023-12-24 07:13:18 -05:00
comfyanonymous
5d2e2630ef
A few missing comfy ops in the VAE.
2023-12-22 04:05:42 -05:00
comfyanonymous
99f61f753a
Remove useless code.
2023-12-15 01:28:16 -05:00
comfyanonymous
98cc7883b7
Refactor comfy.ops
...
comfy.ops -> comfy.ops.disable_weight_init
This should make it more clear what they actually do.
Some unused code has also been removed.
2023-12-11 23:27:13 -05:00
comfyanonymous
782cf4f295
Cleaner CLIP text encoder implementation.
...
Use a simple CLIP model implementation instead of the one from
transformers.
This will allow some interesting things that would too hackish to implement
using the transformers implementation.
2023-12-06 23:50:03 -05:00
comfyanonymous
39bb52dc2c
Missed this one.
2023-12-05 12:48:41 -05:00
comfyanonymous
080f5f4e84
UNET weights can now be stored in fp8.
...
--fp8_e4m3fn-unet and --fp8_e5m2-unet are the two different formats
supported by pytorch.
2023-12-04 11:10:00 -05:00
comfyanonymous
03e65605de
All the unet ops with weights are now handled by comfy.ops
2023-12-04 03:12:18 -05:00
comfyanonymous
60f4b792c8
Fix regression from last commit.
2023-11-26 03:43:02 -05:00
comfyanonymous
e4900be7f3
Clean up the extra_options dict for the transformer patches.
...
Now everything in transformer_options gets put in extra_options.
2023-11-26 03:13:56 -05:00
comfyanonymous
eb17325740
Make buggy xformers fall back on pytorch attention.
2023-11-24 03:55:35 -05:00
comfyanonymous
5e4d60a231
Support SVD img2vid model.
2023-11-23 19:41:33 -05:00
comfyanonymous
cd49281bf8
Remove useless code.
2023-11-21 17:27:28 -05:00
comfyanonymous
7676c8f41a
Make deep shrink behave like it should.
2023-11-16 15:26:28 -05:00
comfyanonymous
87d2cb7b0b
Print warning when controlnet can't be applied instead of crashing.
2023-11-16 12:57:12 -05:00
comfyanonymous
f6333be536
Add a way to add patches to the input block.
2023-11-14 00:08:12 -05:00
comfyanonymous
0d52957c1d
Fix typo.
2023-11-07 23:41:55 -05:00
comfyanonymous
cb22d11fc8
Code refactor.
2023-11-07 19:33:40 -05:00
comfyanonymous
397acd549a
Fix unet ops not entirely on GPU.
2023-11-07 04:30:37 -05:00
comfyanonymous
83a79be597
Fix some memory issues in sub quad attention.
2023-10-30 15:30:49 -04:00
comfyanonymous
8eae4c0adb
Fix some OOM issues with split attention.
2023-10-30 13:14:11 -04:00
comfyanonymous
d4bc91d58f
Support SSD1B model and make it easier to support asymmetric unets.
2023-10-27 14:45:15 -04:00
comfyanonymous
f083f6b663
Fix some OOM issues with split and sub quad attention.
2023-10-25 20:17:28 -04:00
comfyanonymous
c73b5fab20
attention_basic now works with hypertile.
2023-10-22 03:59:53 -04:00
comfyanonymous
57381b0892
Make sub_quad and split work with hypertile.
2023-10-22 03:51:29 -04:00
comfyanonymous
5a9a1a50af
Make xformers work with hypertile.
2023-10-21 13:23:03 -04:00
comfyanonymous
459787f78f
Make VAE code closer to sgm.
2023-10-17 15:18:51 -04:00
comfyanonymous
28b98a96d3
Refactor the attention stuff in the VAE.
2023-10-17 03:19:29 -04:00
comfyanonymous
bffd427388
Add a separate optimized_attention_masked function.
2023-10-16 02:31:24 -04:00
comfyanonymous
728139a5b9
Refactor code so model can be a dtype other than fp32 or fp16.
2023-10-13 14:41:17 -04:00
comfyanonymous
494ddf7717
pytorch_attention_enabled can now return True when xformers is enabled.
2023-10-11 21:30:57 -04:00
comfyanonymous
95df4b6174
Allow attn_mask in attention_pytorch.
2023-10-11 20:38:48 -04:00
comfyanonymous
c60864b5e4
Refactor the attention functions.
...
There's no reason for the whole CrossAttention object to be repeated when
only the operation in the middle changes.
2023-10-11 20:38:48 -04:00
comfyanonymous
23ed2b1654
Model patches can now know which batch is positive and negative.
2023-09-27 12:04:07 -04:00
comfyanonymous
a990441ba0
Add a way to set output block patches to modify the h and hsp.
2023-09-22 20:26:47 -04:00
comfyanonymous
ef0c0892f6
Add a force argument to soft_empty_cache to force a cache empty.
2023-09-04 00:58:18 -04:00
Simon Lui
f63ffd1bb1
Revert changes in comfy/ldm/modules/diffusionmodules/util.py, which is unused.
2023-09-02 20:07:52 -07:00
