1501 Commits

Author SHA1 Message Date
comfyanonymous
cbc01ac278 Switch mochi and wan modes to use pytorch RMSNorm. (#7925)
* Switch genmo model to native RMSNorm.

* Switch WAN to native RMSNorm.
2025-05-03 19:07:55 -04:00
Kohaku-Blueleaf
204e96f2d3 Fix the bugs in OFT/BOFT moule (#7909)
* Correct calculate_weight and load for OFT

* Correct calculate_weight and loading for BOFT
2025-05-02 13:12:37 -04:00
comfyanonymous
58c07d0def Switch ltxv to use the pytorch RMSNorm. (#7897) 2025-05-01 06:33:42 -04:00
comfyanonymous
bedb911371 Cleanup chroma PR. 2025-04-30 20:57:30 -04:00
Silver
0a476dd474 Support for Chroma - Flux1 Schnell distilled with CFG (#7355)
* Upload files for Chroma Implementation

* Remove trailing whitespace

* trim more trailing whitespace..oops

* remove unused imports

* Add supported_inference_dtypes

* Set min_length to 0 and remove attention_mask=True

* Set min_length to 1

* get_mdulations added from blepping and minor changes

* Add lora conversion if statement in lora.py

* Update supported_models.py

* update model_base.py

* add uptream commits

* set modelType.FLOW, will cause beta scheduler to work properly

* Adjust memory usage factor and remove unnecessary code

* fix mistake

* reduce code duplication

* remove unused imports

* refactor for upstream sync

* sync chroma-support with upstream via syncbranch patch

* Update sd.py

* Add Chroma as option for the OptimalStepsScheduler node
2025-04-30 20:57:00 -04:00
comfyanonymous
554a681864 Better vace memory estimation. (#7875) 2025-04-29 20:42:00 -04:00
comfyanonymous
0280886398 Per device stream counters for async offload. (#7873) 2025-04-29 20:28:52 -04:00
guill
a7fafcd751 Add support for VIDEO as a built-in type (#7844)
* Add basic support for videos as types

This PR adds support for VIDEO as first-class types. In order to avoid
unnecessary costs, VIDEO outputs must implement the `VideoInput` ABC,
but their implementation details can vary. Included are two
implementations of this type which can be returned by other nodes:

* `VideoFromFile` - Created with either a path on disk (as a string) or
  a `io.BytesIO` containing the contents of a file in a supported format
  (like .mp4). This implementation won't actually load the video unless
  necessary. It will also avoid re-encoding when saving if possible.
* `VideoFromComponents` - Created from an image tensor and an optional
  audio tensor.

Currently, only h264 encoded videos in .mp4 containers are supported for
saving, but the plan is to add additional encodings/containers in the
near future (particularly .webm).

* Add optimization to avoid parsing entire video

* Improve type declarations to reduce warnings

* Make sure bytesIO objects can be read many times

* Fix a potential issue when saving long videos

* Fix incorrect type annotation

* Add a `LoadVideo` node to make testing easier

* Refactor new types out of the base comfy folder

I've created a new `comfy_api` top-level module. The intention is that
anything within this folder would be covered by semver-style versioning
that would allow custom nodes to rely on them not introducing breaking
changes.

* Fix linting issue
2025-04-29 05:58:00 -04:00
comfyanonymous
87ca14eb5e Support HiDream E1 model. (#7857) 2025-04-28 15:01:15 -04:00
chaObserv
4cc5ea895e CFG++ for gradient estimation sampler (#7809) 2025-04-28 13:51:35 -04:00
comfyanonymous
44ddce3a45 Fix stream priority to support older pytorch. (#7856) 2025-04-28 13:07:21 -04:00
Pam
31b1662657 Save v pred zsnr metadata (#7840) 2025-04-28 13:03:21 -04:00
comfyanonymous
520b3c22e3 Use stream for casting if enabled. (#7833) 2025-04-27 05:38:11 -04:00
comfyanonymous
06146cf019 Make loras work with --async-offload (#7824) 2025-04-26 19:56:22 -04:00
comfyanonymous
0782adba34 Add experimental --async-offload lowvram weight offloading. (#7820)
This should speed up the lowvram mode a bit. It currently is only enabled when --async-offload is used but it will be enabled by default in the future if there are no problems.
2025-04-26 16:11:21 -04:00
comfyanonymous
11606962cf Add a T5TokenizerOptions node to set options for the T5 tokenizer. (#7803) 2025-04-25 19:36:00 -04:00
AustinMroz
364eb4c3f7 [NodeDef] Add documentation on widgetType (#7768)
* [NodeDef] Add documentation on widgetType

* Document required version for widgetType
2025-04-25 13:35:07 -04:00
comfyanonymous
53e7a5517f Support SimpleTuner lycoris lora format for HiDream. 2025-04-25 03:11:14 -04:00
thot experiment
78d56f3232 throw away alpha channel in clip vision preprocessor (#7769)
saves users having to explicitly discard the channel
2025-04-23 21:28:36 -04:00
comfyanonymous
74db647fe9 Fix for dino lowvram. (#7748) 2025-04-23 04:12:52 -04:00
Robin Huang
5025cdb151 [API nodes] Add backbone for supporting api nodes in ComfyUI (#7745)
* Add Ideogram generate node.

* Add staging api.

* COMFY_API_NODE_NAME node property

* switch to boolean flag and use original node name for id

* add optional to type

* Add API_NODE and common error for missing auth token (#5)

* Add Minimax Video Generation + Async Task queue polling example (#6)

* [Minimax] Show video preview and embed workflow in ouput (#7)

* [API Nodes] Send empty request body instead of empty dictionary. (#8)

* Fixed: removed function from rebase.

* Add pydantic.

* Remove uv.lock

* Remove polling operations.

* Update stubs workflow.

* Remove polling comments.

* Update stubs.

* Use pydantic v2.

* Use pydantic v2.

* Add basic OpenAITextToImage node

* Add.

* convert image to tensor.

* Improve types.

* Ruff.

* Push tests.

* Handle multi-form data.

- Don't set content-type for multi-part/form
- Use data field instead of JSON

* Change to api.comfy.org

* Handle error code 409.

* Remove nodes.

---------

Co-authored-by: bymyself <cbyrne@comfy.org>
Co-authored-by: Yoland Y <4950057+yoland68@users.noreply.github.com>
2025-04-23 02:18:08 -04:00
comfyanonymous
27d0247321 Add option for using fp8_e8m0fnu for model weights. (#7733)
Seems to break every model I have tried but worth testing?
2025-04-22 06:17:38 -04:00
Kohaku-Blueleaf
6d53f6f12b Support dora_scale on both axis (#7727) 2025-04-22 05:01:27 -04:00
Kohaku-Blueleaf
5958c5aab6 Add OFT/BOFT algorithm in weight adapter (#7725) 2025-04-22 04:59:47 -04:00
comfyanonymous
5f31968bed Fix issue with WAN VACE implementation. (#7724) 2025-04-21 23:36:12 -04:00
Kohaku-Blueleaf
10a8518868 Unified Weight Adapter system for better maintainability and future feature of Lora system (#7540) 2025-04-21 20:15:32 -04:00
comfyanonymous
8f8fa157a4 Add strength control for vace. (#7717) 2025-04-21 19:36:20 -04:00
filtered
64fa3d6fc5 Add node type hint for socketless option (#7714)
* Add node type hint for socketless option

* nit - Doc
2025-04-21 16:13:00 -04:00
comfyanonymous
e3c1b181a2 Support for WAN VACE preview model. (#7711)
* Support for WAN VACE preview model.

* Remove print.
2025-04-21 14:40:29 -04:00
comfyanonymous
a834f992f0 Slightly better fix for #7687 2025-04-20 11:33:27 -04:00
comfyanonymous
da3d136cb7 Use empty t5 of size 128 for hidream, seems to give closer results. 2025-04-19 19:49:40 -04:00
power88
6f35198397 Hidream: Allow loading hidream text encoders in CLIPLoader and DualCLIPLoader (#7676)
* Hidream: Allow partial loading text encoders

* reformat code for ruff check.
2025-04-19 19:47:30 -04:00
comfyanonymous
da12bcd594 Fix hard crash when the spiece tokenizer path is bad. 2025-04-19 15:55:43 -04:00
comfyanonymous
b16e60eafd Set default context_img_len to 257 2025-04-17 12:42:34 -04:00
comfyanonymous
9601e9e909 Support loading WAN FLF model. 2025-04-17 12:04:48 -04:00
comfyanonymous
89aec08623 Don't hardcode length of context_img in wan code. 2025-04-17 06:25:39 -04:00
comfyanonymous
7223e38263 Make hidream work with any latent resolution. 2025-04-16 18:34:14 -04:00
comfyanonymous
f26cf7d5fa Limit T5 to 128 tokens for HiDream: #7620 2025-04-16 18:07:55 -04:00
comfyanonymous
5ea8378fc6 Reuse code from flux model. 2025-04-16 17:43:55 -04:00
Chenlei Hu
3bcade9e24 [Type] Mark input options NotRequired (#7614) 2025-04-16 15:41:00 -04:00
comfyanonymous
554e371cad Fix issue on old torch. 2025-04-16 04:53:56 -04:00
comfyanonymous
720a954e00 Basic support for hidream i1 model. 2025-04-15 17:35:05 -04:00
comfyanonymous
921181d22b Cleanup. 2025-04-15 12:13:28 -04:00
comfyanonymous
df67f25c28 More flexible long clip support.
Add clip g long clip support.

Text encoder refactor.

Support llama models with different vocab sizes.
2025-04-15 10:32:21 -04:00
comfyanonymous
04d5cdb6a2 add RMSNorm to comfy.ops 2025-04-14 18:00:33 -04:00
chaObserv
c4116ffaea Add SEEDS (stage 2 & 3 DP) sampler (#7580)
* Add seeds stage 2 & 3 (DP) sampler

* Change the name to SEEDS in comment
2025-04-12 18:36:08 -04:00
catboxanon
2f65dfe6b0 Add CublasOps support (#7574)
* CublasOps support

* Guard CublasOps behind --fast arg
2025-04-12 18:29:15 -04:00
Chargeuk
1b28367616 Dependency Aware Node Caching for low RAM/VRAM machines (#7509)
* add dependency aware cache that removed a cached node as soon as all of its decendents have executed. This allows users with lower RAM to run workflows they would otherwise not be able to run. The downside is that every workflow will fully run each time even if no nodes have changed.

* remove test code

* tidy code
2025-04-11 06:55:51 -04:00
Chenlei Hu
a8fd307f35 Deprecate InputTypeOptions.defaultInput (#7551)
* Deprecate InputTypeOptions.defaultInput

* nit

* nit
2025-04-10 06:57:06 -04:00
Jedrzej Kosinski
6227fc9edf Add prepare_sampling wrapper allowing custom nodes to more accurately report noise_shape (#7500) 2025-04-09 09:43:35 -04:00