Jedrzej Kosinski
76b9ed1f2d
Merge branch 'master' into patch_hooks_improved_memory
2024-11-21 17:46:25 -06:00
comfyanonymous
41444b5236
Add some new weight patching functionality.
...
Add a way to reshape lora weights.
Allow weight patches to all weight not just .weight and .bias
Add a way for a lora to set a weight to a specific value.
2024-11-21 07:19:17 -05:00
Jedrzej Kosinski
d38c535771
Turn off hook patch caching when only 1 hook present in sampling, replace some current_hook = None with calls to self.patch_hooks(None) instead to avoid a potential edge case
2024-11-20 18:57:18 -06:00
Jedrzej Kosinski
35016983fb
Small cleanup of comments
2024-11-19 17:49:57 -06:00
Jedrzej Kosinski
9b2b1303b7
Removed a .to call on results of calculate_weight in patch_hook_weight_to_device that was screwing up the intermediate results for fp8 prior to being passed into stochastic_rounding call
2024-11-18 20:42:51 -06:00
Jedrzej Kosinski
de6013ccc4
Increased MemoryCounter minimum memory to leave free by *2 until a better way to get inference memory estimate of currently loaded models exists
2024-11-18 08:08:00 -06:00
Jedrzej Kosinski
0850ae5c04
Remove unnecessary torch.no_grad calls for hook patches
2024-11-18 07:38:11 -06:00
Jedrzej Kosinski
e844695292
Fixed issue with setting weights from hooks instead of copying them, added additional memory_counter check when caching hook patches
2024-11-18 07:25:53 -06:00
Jedrzej Kosinski
f465004a58
Fix cached_hook_patches not respecting target_device/memory_counter results
2024-11-17 12:52:40 -06:00
Jedrzej Kosinski
e177149ae4
Made opt_hooks append by default instead of replace, renamed comfy.hooks set functions to be more accurate
2024-11-16 16:40:45 -06:00
Jedrzej Kosinski
bcc6a22178
Made WrapperHook functional, added another wrapper/callback getter, added ON_DETACH callback to ModelPatcher
2024-11-14 08:06:02 -06:00
Jedrzej Kosinski
5909b06b2e
Removed Register/Direct hook nodes since they were present only for testing, removed diff-related weight hook calculation as improved_memory removes unload_model_clones and using sample time registered hooks is less hacky
2024-11-12 09:05:19 -06:00
Jedrzej Kosinski
1766d903ad
Fixed existing weight hook_patches (pre-registered) not working properly for CLIP
2024-11-12 08:12:12 -06:00
Jedrzej Kosinski
66b3386ee7
Updated different hook types to reflect actual progress of implementation, initial scaffolding for working WrapperHook functionality
2024-11-11 10:13:02 -06:00
Jedrzej Kosinski
b12cc83c5b
Modified callbacks and wrappers so that unregistered types can be used, allowing custom_nodes to have their own unique callbacks/wrappers if desired
2024-11-11 09:05:07 -06:00
Jedrzej Kosinski
638c4086a3
Fixed inconsistency of results when schedule_clip is set to False, small renaming/typo fixing, added initial support for ControlNet extra_hooks to work in tandem with normal cond hooks, initial work on calc_cond_batch merging all subdicts in returned transformer_options
2024-11-11 08:41:08 -06:00
Jedrzej Kosinski
0fbefb8428
Refactored code to store wrappers and callbacks in transformer_options, added apply_model and diffusion_model.forward wrappers
2024-11-03 06:22:48 -06:00
Jedrzej Kosinski
51e8d5554c
Moved WrappersMP/CallbacksMP/WrapperExecutor to patcher_extension.py
2024-11-02 22:21:16 -05:00
Jedrzej Kosinski
45f16c2dad
Merge branch 'improved_memory' into patch_hooks_improved_memory
2024-11-01 03:58:17 -05:00
Jedrzej Kosinski
16735c98e0
Make encode_from_tokens_scheduled call cleaner, rollback change in model_patcher.py for hook_patches_backup dict
2024-11-01 02:01:45 -05:00
comfyanonymous
d8bd2a9baa
Less fragile memory management.
2024-11-01 02:41:51 -04:00
Jedrzej Kosinski
d5169df808
Added initial support within CLIP Text Encode (Prompt) node for scheduling weight hook CLIP strength via clip_start_percent/clip_end_percent on conds, added schedule_clip toggle to Set CLIP Hooks node, small cleanup/fixes
2024-10-30 04:56:09 -05:00
Jedrzej Kosinski
2047bf211f
Changed CreateHookModelAsLoraTest to be the new CreateHookModelAsLora, rename old ones as 'direct' and will be removed prior to merge
2024-10-25 19:58:38 -05:00
Jedrzej Kosinski
daeb2624a9
Fixed default conds not respecting hook keyframes, made keyframes not reset cache when strength is unchanged, fixed Cond Set Default Combine throwing error, fixed model-as-lora throwing error during calculate_weight after a recent ComfyUI update, small refactoring/scaffolding changes for hooks
2024-10-25 18:32:22 -05:00
Jedrzej Kosinski
4bbdf2bfe5
Merge branch 'master' into patch_hooks
2024-10-23 21:10:46 -05:00
comfyanonymous
f9f9faface
Fixed model merging issue with scaled fp8.
2024-10-20 06:24:31 -04:00
comfyanonymous
a68bbafddb
Support diffusion models with scaled fp8 weights.
2024-10-19 23:47:42 -04:00
Jedrzej Kosinski
1f8d9c040b
Fixed models not being unloaded properly due to current_patcher reference; the current ComfyUI model cleanup code requires that nothing else has a reference to the ModelPatcher instances
2024-10-11 06:50:55 -05:00
Jedrzej Kosinski
06fbdb03ef
Merge branch 'master' into patch_hooks
2024-10-05 06:43:43 -05:00
comfyanonymous
3bb4dec720
Fix issue with loras, lowvram and --fast fp8.
2024-09-28 14:42:32 -04:00
kosinkadink1@gmail.com
0f7d379d24
Refactored WrapperExecutor code to remove need for WrapperClassExecutor (now gone), added sampler.sample wrapper (pending review, will likely keep but will see what hacks this could currently let me get rid of in ACN/ADE)
2024-09-27 12:14:36 +09:00
kosinkadink1@gmail.com
09cbd69161
Added create_model_options_clone func, modified type annotations to use __future__ so that I can use the better type annotations
2024-09-25 20:29:49 +09:00
kosinkadink1@gmail.com
d3229cbba7
Implement basic MemoryCounter system for determing with cached weights due to hooks should be offloaded in hooks_backup
2024-09-24 17:28:18 +09:00
kosinkadink1@gmail.com
c422553b0b
Added get_attachment func on ModelPatcher
2024-09-24 16:20:53 +09:00
kosinkadink1@gmail.com
da6c0455cc
Added forward_timestep_embed_patch type, added helper functions on ModelPatcher for emb_patch and forward_timestep_embed_patch, added helper functions for removing callbacks/wrappers/additional_models by key, added custom_should_register prop to hooks
2024-09-24 12:40:54 +09:00
kosinkadink1@gmail.com
7c86407619
Refactored callbacks+wrappers to allow storing lists by id
2024-09-22 16:36:40 +09:00
kosinkadink1@gmail.com
5052a78be2
Added WrapperExecutor for non-classbound functions, added calc_cond_batch wrappers
2024-09-22 11:52:35 +09:00
kosinkadink1@gmail.com
298397d198
Updated clone_has_same_weights function to account for new ModelPatcher properties, improved AutoPatcherEjector usage in partially_load
2024-09-21 21:50:51 +09:00
kosinkadink1@gmail.com
f28d892c16
Fix skip_until_exit logic bug breaking injection after first run of model
2024-09-21 16:34:40 +09:00
kosinkadink1@gmail.com
5f450d3351
Started scaffolding for other hook types, refactored get_hooks_from_cond to organize hooks by type
2024-09-21 10:37:18 +09:00
kosinkadink1@gmail.com
59d72b4050
Added wrappers to ModelPatcher to facilitate standardized function wrapping
2024-09-20 20:05:29 +09:00
kosinkadink1@gmail.com
55014293b1
Added injections support to ModelPatcher + necessary bookkeeping, added additional_models support in ModelPatcher, conds, and hooks
2024-09-19 21:43:58 +09:00
kosinkadink1@gmail.com
e80dc96627
Fix incorrect ref to create_hook_patches_clone after moving function
2024-09-19 11:57:19 +09:00
kosinkadink1@gmail.com
787ef34842
Continued work on simpler Create Hook Model As LoRA node, started to implement ModelPatcher callbacks, attachments, and additional_models
2024-09-19 11:47:25 +09:00
kosinkadink1@gmail.com
6b14fc8795
Merge branch 'master' into patch_hooks
2024-09-17 15:31:03 +09:00
comfyanonymous
0849c80e2a
get_key_patches now works without unloading the model.
2024-09-17 01:57:59 -04:00
kosinkadink1@gmail.com
4b472ba44c
Added support for adding weight hooks that aren't registered on the ModelPatcher at sampling time
2024-09-17 06:22:41 +09:00
kosinkadink1@gmail.com
f5c899f42a
Fixed MaxSpeed and default conds implementations
2024-09-15 21:00:45 +09:00
kosinkadink1@gmail.com
a5034df6db
Made CLIP work with hook patches
2024-09-15 15:47:09 +09:00
kosinkadink1@gmail.com
9ded65a616
Added initial set of hook-related nodes, added code to register hooks for loras/model-as-loras, small renaming/refactoring
2024-09-15 08:33:17 +09:00
