comfyanonymous
8dc45adbe4
Add: --use-sage-attention to enable SageAttention.
...
You need to have the library installed first.
2024-12-18 01:56:10 -05:00
comfyanonymous
f426e83a83
Don't expand mask when not necessary.
...
Expanding seems to slow down inference.
2024-12-16 18:22:50 -05:00
Raphael Walker
ec2ea78501
Add support for attention masking in Flux ( #5942 )
...
* fix attention OOM in xformers
* allow passing attention mask in flux attention
* allow an attn_mask in flux
* attn masks can be done using replace patches instead of a separate dict
* fix return types
* fix return order
* enumerate
* patch the right keys
* arg names
* fix a silly bug
* fix xformers masks
* replace match with if, elif, else
* mask with image_ref_size
* remove unused import
* remove unused import 2
* fix pytorch/xformers attention
This corrects a weird inconsistency with skip_reshape.
It also allows masks of various shapes to be passed, which will be
automtically expanded (in a memory-efficient way) to a size that is
compatible with xformers or pytorch sdpa respectively.
* fix mask shapes
2024-12-16 18:21:17 -05:00
Chenlei Hu
efca88cfb3
Lint all unused variables ( #5989 )
...
* Enable F841
* Autofix
* Remove all unused variable assignment
2024-12-12 17:59:16 -05:00
comfyanonymous
0c21ed154e
Fix mask issue in some attention functions.
2024-11-22 02:10:09 -05:00
comfyanonymous
bb7614876f
Fix mask issue with attention_xformers.
2024-11-20 17:07:46 -05:00
comfyanonymous
0e63952db1
Mochi VAE encoder.
2024-11-01 17:33:09 -04:00
comfyanonymous
f88fa3a5e2
Fix issue.
2024-08-14 02:51:47 -04:00
comfyanonymous
1eb91dbfbb
Initial support for the stable audio open model.
2024-06-15 12:14:56 -04:00
comfyanonymous
93094050bb
Remove some unused imports.
2024-05-27 19:08:27 -04:00
comfyanonymous
bdbaa4491c
Work around black image bug on Mac 14.5 by forcing attention upcasting.
2024-05-21 16:56:33 -04:00
comfyanonymous
e0c00cac39
Disable xformers when tracing model.
2024-05-21 13:55:49 -04:00
comfyanonymous
a8c173cc1b
Fix potential issue.
2024-05-20 08:19:54 -04:00
comfyanonymous
c19335f030
Cleanup.
2024-05-18 10:11:44 -04:00
comfyanonymous
fddf60ebb7
Remove unnecessary code.
2024-05-18 09:36:44 -04:00
comfyanonymous
e2d71cc7b6
Add debug options to force on and off attention upcasting.
2024-05-16 04:09:41 -04:00
comfyanonymous
e02598e417
Fix SAG.
2024-05-14 18:02:27 -04:00
comfyanonymous
d4db823cfe
Only enable attention upcasting on models that actually need it.
2024-05-14 17:00:50 -04:00
comfyanonymous
89e38bcd63
Refactor attention upcasting code part 1.
2024-05-14 12:47:31 -04:00
comfyanonymous
95e80c5df1
Workaround xformers bug.
2024-04-30 21:23:40 -04:00
comfyanonymous
9956e2b6e7
Switch some more prints to logging.
2024-03-11 16:34:58 -04:00
comfyanonymous
681300661a
Fix attention masks properly for multiple batches.
2024-02-17 16:15:18 -05:00
comfyanonymous
511e70ab62
Fix attention mask batch size in some attention functions.
2024-02-17 15:22:21 -05:00
comfyanonymous
4a9216bca3
Properly fix attention masks in CLIP with batches.
2024-02-17 12:13:13 -05:00
comfyanonymous
50fb52bc5b
Remove some unused imports.
2024-01-25 23:42:37 -05:00
comfyanonymous
62a7bf2919
Use basic attention implementation for small inputs on old pytorch.
2024-01-09 13:46:52 -05:00
comfyanonymous
8cf0a2dfe7
Update optimized_attention_for_device function for new functions that
...
support masked attention.
2024-01-07 13:52:08 -05:00
comfyanonymous
f9f0822512
Add attention mask support to sub quad attention.
2024-01-07 04:13:58 -05:00
comfyanonymous
59259b0de3
Support attention mask in split attention.
2024-01-06 13:16:48 -05:00
comfyanonymous
1240b45932
Implement attention mask on xformers.
2024-01-06 04:33:03 -05:00
comfyanonymous
99f61f753a
Remove useless code.
2023-12-15 01:28:16 -05:00
comfyanonymous
98cc7883b7
Refactor comfy.ops
...
comfy.ops -> comfy.ops.disable_weight_init
This should make it more clear what they actually do.
Some unused code has also been removed.
2023-12-11 23:27:13 -05:00
comfyanonymous
782cf4f295
Cleaner CLIP text encoder implementation.
...
Use a simple CLIP model implementation instead of the one from
transformers.
This will allow some interesting things that would too hackish to implement
using the transformers implementation.
2023-12-06 23:50:03 -05:00
comfyanonymous
39bb52dc2c
Missed this one.
2023-12-05 12:48:41 -05:00
comfyanonymous
03e65605de
All the unet ops with weights are now handled by comfy.ops
2023-12-04 03:12:18 -05:00
comfyanonymous
60f4b792c8
Fix regression from last commit.
2023-11-26 03:43:02 -05:00
comfyanonymous
e4900be7f3
Clean up the extra_options dict for the transformer patches.
...
Now everything in transformer_options gets put in extra_options.
2023-11-26 03:13:56 -05:00
comfyanonymous
eb17325740
Make buggy xformers fall back on pytorch attention.
2023-11-24 03:55:35 -05:00
comfyanonymous
5e4d60a231
Support SVD img2vid model.
2023-11-23 19:41:33 -05:00
comfyanonymous
83a79be597
Fix some memory issues in sub quad attention.
2023-10-30 15:30:49 -04:00
comfyanonymous
8eae4c0adb
Fix some OOM issues with split attention.
2023-10-30 13:14:11 -04:00
comfyanonymous
f083f6b663
Fix some OOM issues with split and sub quad attention.
2023-10-25 20:17:28 -04:00
comfyanonymous
c73b5fab20
attention_basic now works with hypertile.
2023-10-22 03:59:53 -04:00
comfyanonymous
57381b0892
Make sub_quad and split work with hypertile.
2023-10-22 03:51:29 -04:00
comfyanonymous
5a9a1a50af
Make xformers work with hypertile.
2023-10-21 13:23:03 -04:00
comfyanonymous
bffd427388
Add a separate optimized_attention_masked function.
2023-10-16 02:31:24 -04:00
comfyanonymous
95df4b6174
Allow attn_mask in attention_pytorch.
2023-10-11 20:38:48 -04:00
comfyanonymous
c60864b5e4
Refactor the attention functions.
...
There's no reason for the whole CrossAttention object to be repeated when
only the operation in the middle changes.
2023-10-11 20:38:48 -04:00
comfyanonymous
23ed2b1654
Model patches can now know which batch is positive and negative.
2023-09-27 12:04:07 -04:00
comfyanonymous
ef0c0892f6
Add a force argument to soft_empty_cache to force a cache empty.
2023-09-04 00:58:18 -04:00
