Initial commit: Add custom_nodes and modifications

.gitignore

@@ -5,17 +5,17 @@ __pycache__/
 !/input/example.png
 /models/
 /temp/
-/custom_nodes/
+#/custom_nodes/
-!custom_nodes/example_node.py.example
+#!custom_nodes/example_node.py.example
 extra_model_paths.yaml
 /.vs
 .vscode/
 .idea/
 venv/
 .venv/
-/web/extensions/*
+#/web/extensions/*
-!/web/extensions/logging.js.example
+#!/web/extensions/logging.js.example
-!/web/extensions/core/
+#!/web/extensions/core/
 /tests-ui/data/object_info.json
 /user/
 *.log

custom_nodes/ComfyUI-GGUF/.github/workflows/registry.yaml

@@ -0,0 +1,21 @@
+name: ComfyUI Registry publish
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - stable
+    paths:
+      - "pyproject.toml"
+
+jobs:
+  publish-node:
+    name: ComfyUI Registry publish
+    runs-on: ubuntu-latest
+    if: github.event.repository.fork == false
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+      - name: Publish Custom Node
+        uses: Comfy-Org/publish-node-action@main
+        with:
+          personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}

custom_nodes/ComfyUI-GGUF/.gitignore

@@ -0,0 +1,167 @@
+*.bin
+*.gguf
+*.safetensors
+tools/llama.cpp*
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

custom_nodes/ComfyUI-GGUF/.tracking

@@ -0,0 +1,17 @@
+.github/workflows/registry.yaml
+.gitignore
+LICENSE
+README.md
+__init__.py
+dequant.py
+loader.py
+nodes.py
+ops.py
+pyproject.toml
+requirements.txt
+tools/README.md
+tools/convert.py
+tools/fix_5d_tensors.py
+tools/fix_lines_ending.py
+tools/lcpp.patch
+tools/read_tensors.py

custom_nodes/ComfyUI-GGUF/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

custom_nodes/ComfyUI-GGUF/README.md

@@ -0,0 +1,49 @@
+# ComfyUI-GGUF
+GGUF Quantization support for native ComfyUI models
+
+This is currently very much WIP. These custom nodes provide support for model files stored in the GGUF format popularized by [llama.cpp](https://github.com/ggerganov/llama.cpp).
+
+While quantization wasn't feasible for regular UNET models (conv2d), transformer/DiT models such as flux seem less affected by quantization. This allows running it in much lower bits per weight variable bitrate quants on low-end GPUs. For further VRAM savings, a node to load a quantized version of the T5 text encoder is also included.
+
+![Comfy_Flux1_dev_Q4_0_GGUF_1024](https://github.com/user-attachments/assets/70d16d97-c522-4ef4-9435-633f128644c8)
+
+Note: The "Force/Set CLIP Device" is **NOT** part of this node pack. Do not install it if you only have one GPU. Do not set it to cuda:0 then complain about OOM errors if you do not undestand what it is for. There is not need to copy the workflow above, just use your own workflow and replace the stock "Load Diffusion Model" with the "Unet Loader (GGUF)" node.
+
+## Installation
+
+> [!IMPORTANT]  
+> Make sure your ComfyUI is on a recent-enough version to support custom ops when loading the UNET-only.
+
+To install the custom node normally, git clone this repository into your custom nodes folder (`ComfyUI/custom_nodes`) and install the only dependency for inference (`pip install --upgrade gguf`)
+
+```
+git clone https://github.com/city96/ComfyUI-GGUF
+```
+
+To install the custom node on a standalone ComfyUI release, open a CMD inside the "ComfyUI_windows_portable" folder (where your `run_nvidia_gpu.bat` file is) and use the following commands:
+
+```
+git clone https://github.com/city96/ComfyUI-GGUF ComfyUI/custom_nodes/ComfyUI-GGUF
+.\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt
+```
+
+On MacOS sequoia, torch 2.4.1 seems to be required, as 2.6.X nightly versions cause a "M1 buffer is not large enough" error. See [this issue](https://github.com/city96/ComfyUI-GGUF/issues/107) for more information/workarounds.
+
+## Usage
+
+Simply use the GGUF Unet loader found under the `bootleg` category. Place the .gguf model files in your `ComfyUI/models/unet` folder.
+
+LoRA loading is experimental but it should work with just the built-in LoRA loader node(s).
+
+Pre-quantized models:
+
+- [flux1-dev GGUF](https://huggingface.co/city96/FLUX.1-dev-gguf)
+- [flux1-schnell GGUF](https://huggingface.co/city96/FLUX.1-schnell-gguf)
+- [stable-diffusion-3.5-large GGUF](https://huggingface.co/city96/stable-diffusion-3.5-large-gguf)
+- [stable-diffusion-3.5-large-turbo GGUF](https://huggingface.co/city96/stable-diffusion-3.5-large-turbo-gguf)
+
+Initial support for quantizing T5 has also been added recently, these can be used using the various `*CLIPLoader (gguf)` nodes which can be used inplace of the regular ones. For the CLIP model, use whatever model you were using before for CLIP. The loader can handle both types of files - `gguf` and regular `safetensors`/`bin`.
+
+- [t5_v1.1-xxl GGUF](https://huggingface.co/city96/t5-v1_1-xxl-encoder-gguf)
+
+See the instructions in the [tools](https://github.com/city96/ComfyUI-GGUF/tree/main/tools) folder for how to create your own quants.

custom_nodes/ComfyUI-GGUF/__init__.py

@@ -0,0 +1,9 @@
+# only import if running as a custom node
+try:
+    import comfy.utils
+except ImportError:
+    pass
+else:
+    from .nodes import NODE_CLASS_MAPPINGS
+    NODE_DISPLAY_NAME_MAPPINGS = {k:v.TITLE for k,v in NODE_CLASS_MAPPINGS.items()}
+    __all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']

custom_nodes/ComfyUI-GGUF/dequant.py

@@ -0,0 +1,248 @@
+# (c) City96 || Apache-2.0 (apache.org/licenses/LICENSE-2.0)
+import gguf
+import torch
+from tqdm import tqdm
+
+
+TORCH_COMPATIBLE_QTYPES = (None, gguf.GGMLQuantizationType.F32, gguf.GGMLQuantizationType.F16)
+
+def is_torch_compatible(tensor):
+    return tensor is None or getattr(tensor, "tensor_type", None) in TORCH_COMPATIBLE_QTYPES
+
+def is_quantized(tensor):
+    return not is_torch_compatible(tensor)
+
+def dequantize_tensor(tensor, dtype=None, dequant_dtype=None):
+    qtype = getattr(tensor, "tensor_type", None)
+    oshape = getattr(tensor, "tensor_shape", tensor.shape)
+
+    if qtype in TORCH_COMPATIBLE_QTYPES:
+        return tensor.to(dtype)
+    elif qtype in dequantize_functions:
+        dequant_dtype = dtype if dequant_dtype == "target" else dequant_dtype
+        return dequantize(tensor.data, qtype, oshape, dtype=dequant_dtype).to(dtype)
+    else:
+        # this is incredibly slow
+        tqdm.write(f"Falling back to numpy dequant for qtype: {qtype}")
+        new = gguf.quants.dequantize(tensor.cpu().numpy(), qtype)
+        return torch.from_numpy(new).to(tensor.device, dtype=dtype)
+
+def dequantize(data, qtype, oshape, dtype=None):
+    """
+    Dequantize tensor back to usable shape/dtype
+    """
+    block_size, type_size = gguf.GGML_QUANT_SIZES[qtype]
+    dequantize_blocks = dequantize_functions[qtype]
+
+    rows = data.reshape(
+        (-1, data.shape[-1])
+    ).view(torch.uint8)
+
+    n_blocks = rows.numel() // type_size
+    blocks = rows.reshape((n_blocks, type_size))
+    blocks = dequantize_blocks(blocks, block_size, type_size, dtype)
+    return blocks.reshape(oshape)
+
+def to_uint32(x):
+    # no uint32 :(
+    x = x.view(torch.uint8).to(torch.int32)
+    return (x[:, 0] | x[:, 1] << 8 | x[:, 2] << 16 | x[:, 3] << 24).unsqueeze(1)
+
+def split_block_dims(blocks, *args):
+    n_max = blocks.shape[1]
+    dims = list(args) + [n_max - sum(args)]
+    return torch.split(blocks, dims, dim=1)
+
+# Full weights #
+def dequantize_blocks_BF16(blocks, block_size, type_size, dtype=None):
+    return (blocks.view(torch.int16).to(torch.int32) << 16).view(torch.float32)
+
+# Legacy Quants #
+def dequantize_blocks_Q8_0(blocks, block_size, type_size, dtype=None):
+    d, x = split_block_dims(blocks, 2)
+    d = d.view(torch.float16).to(dtype)
+    x = x.view(torch.int8)
+    return (d * x)
+
+def dequantize_blocks_Q5_1(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    d, m, qh, qs = split_block_dims(blocks, 2, 2, 4)
+    d = d.view(torch.float16).to(dtype)
+    m = m.view(torch.float16).to(dtype)
+    qh = to_uint32(qh)
+
+    qh = qh.reshape((n_blocks, 1)) >> torch.arange(32, device=d.device, dtype=torch.int32).reshape(1, 32)
+    ql = qs.reshape((n_blocks, -1, 1, block_size // 2)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape(1, 1, 2, 1)
+    qh = (qh & 1).to(torch.uint8)
+    ql = (ql & 0x0F).reshape((n_blocks, -1))
+
+    qs = (ql | (qh << 4))
+    return (d * qs) + m
+
+def dequantize_blocks_Q5_0(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    d, qh, qs = split_block_dims(blocks, 2, 4)
+    d  = d.view(torch.float16).to(dtype)
+    qh = to_uint32(qh)
+
+    qh = qh.reshape(n_blocks, 1) >> torch.arange(32, device=d.device, dtype=torch.int32).reshape(1, 32)
+    ql = qs.reshape(n_blocks, -1, 1, block_size // 2) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape(1, 1, 2, 1)
+
+    qh = (qh & 1).to(torch.uint8)
+    ql = (ql & 0x0F).reshape(n_blocks, -1)
+
+    qs = (ql | (qh << 4)).to(torch.int8) - 16
+    return (d * qs)
+
+def dequantize_blocks_Q4_1(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    d, m, qs = split_block_dims(blocks, 2, 2)
+    d = d.view(torch.float16).to(dtype)
+    m = m.view(torch.float16).to(dtype)
+
+    qs = qs.reshape((n_blocks, -1, 1, block_size // 2)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape(1, 1, 2, 1)
+    qs = (qs & 0x0F).reshape(n_blocks, -1)
+
+    return (d * qs) + m
+
+def dequantize_blocks_Q4_0(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    d, qs = split_block_dims(blocks, 2)
+    d  = d.view(torch.float16).to(dtype)
+
+    qs = qs.reshape((n_blocks, -1, 1, block_size // 2)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape((1, 1, 2, 1))
+    qs = (qs & 0x0F).reshape((n_blocks, -1)).to(torch.int8) - 8
+    return (d * qs)
+
+# K Quants #
+QK_K = 256
+K_SCALE_SIZE = 12
+
+def get_scale_min(scales):
+    n_blocks = scales.shape[0]
+    scales = scales.view(torch.uint8)
+    scales = scales.reshape((n_blocks, 3, 4))
+
+    d, m, m_d = torch.split(scales, scales.shape[-2] // 3, dim=-2)
+
+    sc = torch.cat([d & 0x3F, (m_d & 0x0F) | ((d >> 2) & 0x30)], dim=-1)
+    min = torch.cat([m & 0x3F, (m_d >> 4) | ((m >> 2) & 0x30)], dim=-1)
+
+    return (sc.reshape((n_blocks, 8)), min.reshape((n_blocks, 8)))
+
+def dequantize_blocks_Q6_K(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    ql, qh, scales, d, = split_block_dims(blocks, QK_K // 2, QK_K // 4, QK_K // 16)
+
+    scales = scales.view(torch.int8).to(dtype)
+    d = d.view(torch.float16).to(dtype)
+    d = (d * scales).reshape((n_blocks, QK_K // 16, 1))
+
+    ql = ql.reshape((n_blocks, -1, 1, 64)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape((1, 1, 2, 1))
+    ql = (ql & 0x0F).reshape((n_blocks, -1, 32))
+    qh = qh.reshape((n_blocks, -1, 1, 32)) >> torch.tensor([0, 2, 4, 6], device=d.device, dtype=torch.uint8).reshape((1, 1, 4, 1))
+    qh = (qh & 0x03).reshape((n_blocks, -1, 32))
+    q = (ql | (qh << 4)).to(torch.int8) - 32
+    q = q.reshape((n_blocks, QK_K // 16, -1))
+
+    return (d * q).reshape((n_blocks, QK_K))
+
+def dequantize_blocks_Q5_K(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    d, dmin, scales, qh, qs = split_block_dims(blocks, 2, 2, K_SCALE_SIZE, QK_K // 8)
+
+    d = d.view(torch.float16).to(dtype)
+    dmin = dmin.view(torch.float16).to(dtype)
+
+    sc, m = get_scale_min(scales)
+
+    d = (d * sc).reshape((n_blocks, -1, 1))
+    dm = (dmin * m).reshape((n_blocks, -1, 1))
+
+    ql = qs.reshape((n_blocks, -1, 1, 32)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape((1, 1, 2, 1))
+    qh = qh.reshape((n_blocks, -1, 1, 32)) >> torch.tensor([i for i in range(8)], device=d.device, dtype=torch.uint8).reshape((1, 1, 8, 1))
+    ql = (ql & 0x0F).reshape((n_blocks, -1, 32))
+    qh = (qh & 0x01).reshape((n_blocks, -1, 32))
+    q = (ql | (qh << 4))
+
+    return (d * q - dm).reshape((n_blocks, QK_K))
+
+def dequantize_blocks_Q4_K(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    d, dmin, scales, qs = split_block_dims(blocks, 2, 2, K_SCALE_SIZE)
+    d = d.view(torch.float16).to(dtype)
+    dmin = dmin.view(torch.float16).to(dtype)
+
+    sc, m = get_scale_min(scales)
+
+    d = (d * sc).reshape((n_blocks, -1, 1))
+    dm = (dmin * m).reshape((n_blocks, -1, 1))
+
+    qs = qs.reshape((n_blocks, -1, 1, 32)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape((1, 1, 2, 1))
+    qs = (qs & 0x0F).reshape((n_blocks, -1, 32))
+
+    return (d * qs - dm).reshape((n_blocks, QK_K))
+
+def dequantize_blocks_Q3_K(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    hmask, qs, scales, d = split_block_dims(blocks, QK_K // 8, QK_K // 4, 12)
+    d = d.view(torch.float16).to(dtype)
+
+    lscales, hscales = scales[:, :8], scales[:, 8:]
+    lscales = lscales.reshape((n_blocks, 1, 8)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape((1, 2, 1))
+    lscales = lscales.reshape((n_blocks, 16))
+    hscales = hscales.reshape((n_blocks, 1, 4)) >> torch.tensor([0, 2, 4, 6], device=d.device, dtype=torch.uint8).reshape((1, 4, 1))
+    hscales = hscales.reshape((n_blocks, 16))
+    scales = (lscales & 0x0F) | ((hscales & 0x03) << 4)
+    scales = (scales.to(torch.int8) - 32)
+
+    dl = (d * scales).reshape((n_blocks, 16, 1))
+
+    ql = qs.reshape((n_blocks, -1, 1, 32)) >> torch.tensor([0, 2, 4, 6], device=d.device, dtype=torch.uint8).reshape((1, 1, 4, 1))
+    qh = hmask.reshape(n_blocks, -1, 1, 32) >> torch.tensor([i for i in range(8)], device=d.device, dtype=torch.uint8).reshape((1, 1, 8, 1))
+    ql = ql.reshape((n_blocks, 16, QK_K // 16)) & 3
+    qh = (qh.reshape((n_blocks, 16, QK_K // 16)) & 1) ^ 1
+    q = (ql.to(torch.int8) - (qh << 2).to(torch.int8))
+
+    return (dl * q).reshape((n_blocks, QK_K))
+
+def dequantize_blocks_Q2_K(blocks, block_size, type_size, dtype=None):
+    n_blocks = blocks.shape[0]
+
+    scales, qs, d, dmin = split_block_dims(blocks, QK_K // 16, QK_K // 4, 2)
+    d = d.view(torch.float16).to(dtype)
+    dmin = dmin.view(torch.float16).to(dtype)
+
+    # (n_blocks, 16, 1)
+    dl = (d * (scales & 0xF)).reshape((n_blocks, QK_K // 16, 1))
+    ml = (dmin * (scales >> 4)).reshape((n_blocks, QK_K // 16, 1))
+
+    shift = torch.tensor([0, 2, 4, 6], device=d.device, dtype=torch.uint8).reshape((1, 1, 4, 1))
+
+    qs = (qs.reshape((n_blocks, -1, 1, 32)) >> shift) & 3
+    qs = qs.reshape((n_blocks, QK_K // 16, 16))
+    qs = dl * qs - ml
+
+    return qs.reshape((n_blocks, -1))
+
+dequantize_functions = {
+    gguf.GGMLQuantizationType.BF16: dequantize_blocks_BF16,
+    gguf.GGMLQuantizationType.Q8_0: dequantize_blocks_Q8_0,
+    gguf.GGMLQuantizationType.Q5_1: dequantize_blocks_Q5_1,
+    gguf.GGMLQuantizationType.Q5_0: dequantize_blocks_Q5_0,
+    gguf.GGMLQuantizationType.Q4_1: dequantize_blocks_Q4_1,
+    gguf.GGMLQuantizationType.Q4_0: dequantize_blocks_Q4_0,
+    gguf.GGMLQuantizationType.Q6_K: dequantize_blocks_Q6_K,
+    gguf.GGMLQuantizationType.Q5_K: dequantize_blocks_Q5_K,
+    gguf.GGMLQuantizationType.Q4_K: dequantize_blocks_Q4_K,
+    gguf.GGMLQuantizationType.Q3_K: dequantize_blocks_Q3_K,
+    gguf.GGMLQuantizationType.Q2_K: dequantize_blocks_Q2_K,
+}

custom_nodes/ComfyUI-GGUF/loader.py

@@ -0,0 +1,259 @@
+# (c) City96 || Apache-2.0 (apache.org/licenses/LICENSE-2.0)
+import warnings
+import logging
+import torch
+import gguf
+
+from .ops import GGMLTensor
+from .dequant import is_quantized, dequantize_tensor
+
+IMG_ARCH_LIST = {"flux", "sd1", "sdxl", "sd3", "aura", "hidream", "cosmos", "ltxv", "hyvid", "wan", "lumina2", "qwen_image"}
+TXT_ARCH_LIST = {"t5", "t5encoder", "llama", "qwen2vl"}
+
+def get_orig_shape(reader, tensor_name):
+    field_key = f"comfy.gguf.orig_shape.{tensor_name}"
+    field = reader.get_field(field_key)
+    if field is None:
+        return None
+    # Has original shape metadata, so we try to decode it.
+    if len(field.types) != 2 or field.types[0] != gguf.GGUFValueType.ARRAY or field.types[1] != gguf.GGUFValueType.INT32:
+        raise TypeError(f"Bad original shape metadata for {field_key}: Expected ARRAY of INT32, got {field.types}")
+    return torch.Size(tuple(int(field.parts[part_idx][0]) for part_idx in field.data))
+
+def get_field(reader, field_name, field_type):
+    field = reader.get_field(field_name)
+    if field is None:
+        return None
+    elif field_type == str:
+        # extra check here as this is used for checking arch string
+        if len(field.types) != 1 or field.types[0] != gguf.GGUFValueType.STRING:
+            raise TypeError(f"Bad type for GGUF {field_name} key: expected string, got {field.types!r}")
+        return str(field.parts[field.data[-1]], encoding="utf-8")
+    elif field_type in [int, float, bool]:
+        return field_type(field.parts[field.data[-1]])
+    else:
+        raise TypeError(f"Unknown field type {field_type}")
+
+def get_list_field(reader, field_name, field_type):
+    field = reader.get_field(field_name)
+    if field is None:
+        return None
+    elif field_type == str:
+        return tuple(str(field.parts[part_idx], encoding="utf-8") for part_idx in field.data)
+    elif field_type in [int, float, bool]:
+        return tuple(field_type(field.parts[part_idx][0]) for part_idx in field.data)
+    else:
+        raise TypeError(f"Unknown field type {field_type}")
+
+def gguf_sd_loader(path, handle_prefix="model.diffusion_model.", return_arch=False, is_text_model=False):
+    """
+    Read state dict as fake tensors
+    """
+    reader = gguf.GGUFReader(path)
+
+    # filter and strip prefix
+    has_prefix = False
+    if handle_prefix is not None:
+        prefix_len = len(handle_prefix)
+        tensor_names = set(tensor.name for tensor in reader.tensors)
+        has_prefix = any(s.startswith(handle_prefix) for s in tensor_names)
+
+    tensors = []
+    for tensor in reader.tensors:
+        sd_key = tensor_name = tensor.name
+        if has_prefix:
+            if not tensor_name.startswith(handle_prefix):
+                continue
+            sd_key = tensor_name[prefix_len:]
+        tensors.append((sd_key, tensor))
+
+    # detect and verify architecture
+    compat = None
+    arch_str = get_field(reader, "general.architecture", str)
+    if arch_str in [None, "pig"]:
+        if is_text_model:
+            raise ValueError(f"This text model is incompatible with llama.cpp!\nConsider using the safetensors version\n({path})")
+        compat = "sd.cpp" if arch_str is None else arch_str
+        # import here to avoid changes to convert.py breaking regular models
+        from .tools.convert import detect_arch
+        try:
+            arch_str = detect_arch(set(val[0] for val in tensors)).arch
+        except Exception as e:
+            raise ValueError(f"This model is not currently supported - ({e})")
+    elif arch_str not in TXT_ARCH_LIST and is_text_model:
+        raise ValueError(f"Unexpected text model architecture type in GGUF file: {arch_str!r}")
+    elif arch_str not in IMG_ARCH_LIST and not is_text_model:
+        raise ValueError(f"Unexpected architecture type in GGUF file: {arch_str!r}")
+
+    if compat:
+        logging.warning(f"Warning: This gguf model file is loaded in compatibility mode '{compat}' [arch:{arch_str}]")
+
+    # main loading loop
+    state_dict = {}
+    qtype_dict = {}
+    for sd_key, tensor in tensors:
+        tensor_name = tensor.name
+        # torch_tensor = torch.from_numpy(tensor.data) # mmap
+
+        # NOTE: line above replaced with this block to avoid persistent numpy warning about mmap
+        with warnings.catch_warnings():
+            warnings.filterwarnings("ignore", message="The given NumPy array is not writable")
+            torch_tensor = torch.from_numpy(tensor.data) # mmap
+
+        shape = get_orig_shape(reader, tensor_name)
+        if shape is None:
+            shape = torch.Size(tuple(int(v) for v in reversed(tensor.shape)))
+            # Workaround for stable-diffusion.cpp SDXL detection.
+            if compat == "sd.cpp" and arch_str == "sdxl":
+                if any([tensor_name.endswith(x) for x in (".proj_in.weight", ".proj_out.weight")]):
+                    while len(shape) > 2 and shape[-1] == 1:
+                        shape = shape[:-1]
+
+        # add to state dict
+        if tensor.tensor_type in {gguf.GGMLQuantizationType.F32, gguf.GGMLQuantizationType.F16}:
+            torch_tensor = torch_tensor.view(*shape)
+        state_dict[sd_key] = GGMLTensor(torch_tensor, tensor_type=tensor.tensor_type, tensor_shape=shape)
+
+        # keep track of loaded tensor types
+        tensor_type_str = getattr(tensor.tensor_type, "name", repr(tensor.tensor_type))
+        qtype_dict[tensor_type_str] = qtype_dict.get(tensor_type_str, 0) + 1
+
+    # print loaded tensor type counts
+    logging.info("gguf qtypes: " + ", ".join(f"{k} ({v})" for k, v in qtype_dict.items()))
+
+    # mark largest tensor for vram estimation
+    qsd = {k:v for k,v in state_dict.items() if is_quantized(v)}
+    if len(qsd) > 0:
+        max_key = max(qsd.keys(), key=lambda k: qsd[k].numel())
+        state_dict[max_key].is_largest_weight = True
+
+    if return_arch:
+        return (state_dict, arch_str)
+    return state_dict
+
+# for remapping llama.cpp -> original key names
+T5_SD_MAP = {
+    "enc.": "encoder.",
+    ".blk.": ".block.",
+    "token_embd": "shared",
+    "output_norm": "final_layer_norm",
+    "attn_q": "layer.0.SelfAttention.q",
+    "attn_k": "layer.0.SelfAttention.k",
+    "attn_v": "layer.0.SelfAttention.v",
+    "attn_o": "layer.0.SelfAttention.o",
+    "attn_norm": "layer.0.layer_norm",
+    "attn_rel_b": "layer.0.SelfAttention.relative_attention_bias",
+    "ffn_up": "layer.1.DenseReluDense.wi_1",
+    "ffn_down": "layer.1.DenseReluDense.wo",
+    "ffn_gate": "layer.1.DenseReluDense.wi_0",
+    "ffn_norm": "layer.1.layer_norm",
+}
+
+LLAMA_SD_MAP = {
+    "blk.": "model.layers.",
+    "attn_norm": "input_layernorm",
+    "attn_q": "self_attn.q_proj",
+    "attn_k": "self_attn.k_proj",
+    "attn_v": "self_attn.v_proj",
+    "attn_output": "self_attn.o_proj",
+    "ffn_up": "mlp.up_proj",
+    "ffn_down": "mlp.down_proj",
+    "ffn_gate": "mlp.gate_proj",
+    "ffn_norm": "post_attention_layernorm",
+    "token_embd": "model.embed_tokens",
+    "output_norm": "model.norm",
+    "output.weight": "lm_head.weight",
+}
+
+def sd_map_replace(raw_sd, key_map):
+    sd = {}
+    for k,v in raw_sd.items():
+        for s,d in key_map.items():
+            k = k.replace(s,d)
+        sd[k] = v
+    return sd
+
+def llama_permute(raw_sd, n_head, n_head_kv):
+    # Reverse version of LlamaModel.permute in llama.cpp convert script
+    sd = {}
+    permute = lambda x,h: x.reshape(h, x.shape[0] // h // 2, 2, *x.shape[1:]).swapaxes(1, 2).reshape(x.shape)
+    for k,v in raw_sd.items():
+        if k.endswith(("q_proj.weight", "q_proj.bias")):
+            v.data = permute(v.data, n_head)
+        if k.endswith(("k_proj.weight", "k_proj.bias")):
+            v.data = permute(v.data, n_head_kv)
+        sd[k] = v
+    return sd
+
+def gguf_tokenizer_loader(path, temb_shape):
+    # convert gguf tokenizer to spiece
+    logging.info("Attempting to recreate sentencepiece tokenizer from GGUF file metadata...")
+    try:
+        from sentencepiece import sentencepiece_model_pb2 as model
+    except ImportError:
+        raise ImportError("Please make sure sentencepiece and protobuf are installed.\npip install sentencepiece protobuf")
+    spm = model.ModelProto()
+
+    reader = gguf.GGUFReader(path)
+
+    if get_field(reader, "tokenizer.ggml.model", str) == "t5":
+        if temb_shape == (256384, 4096): # probably UMT5
+            spm.trainer_spec.model_type == 1 # Unigram (do we have a T5 w/ BPE?)
+        else:
+            raise NotImplementedError("Unknown model, can't set tokenizer!")
+    else:
+        raise NotImplementedError("Unknown model, can't set tokenizer!")
+
+    spm.normalizer_spec.add_dummy_prefix = get_field(reader, "tokenizer.ggml.add_space_prefix", bool)
+    spm.normalizer_spec.remove_extra_whitespaces = get_field(reader, "tokenizer.ggml.remove_extra_whitespaces", bool)
+
+    tokens = get_list_field(reader, "tokenizer.ggml.tokens", str)
+    scores = get_list_field(reader, "tokenizer.ggml.scores", float)
+    toktypes = get_list_field(reader, "tokenizer.ggml.token_type", int)
+
+    for idx, (token, score, toktype) in enumerate(zip(tokens, scores, toktypes)):
+        # # These aren't present in the original?
+        # if toktype == 5 and idx >= temb_shape[0]%1000):
+        #     continue
+
+        piece = spm.SentencePiece()
+        piece.piece = token
+        piece.score = score
+        piece.type = toktype
+        spm.pieces.append(piece)
+
+    # unsure if any of these are correct
+    spm.trainer_spec.byte_fallback = True
+    spm.trainer_spec.vocab_size = len(tokens) # split off unused?
+    spm.trainer_spec.max_sentence_length = 4096
+    spm.trainer_spec.eos_id = get_field(reader, "tokenizer.ggml.eos_token_id", int)
+    spm.trainer_spec.pad_id = get_field(reader, "tokenizer.ggml.padding_token_id", int)
+
+    logging.info(f"Created tokenizer with vocab size of {len(spm.pieces)}")
+    del reader
+    return torch.ByteTensor(list(spm.SerializeToString()))
+
+def gguf_clip_loader(path):
+    sd, arch = gguf_sd_loader(path, return_arch=True, is_text_model=True)
+    if arch in {"t5", "t5encoder"}:
+        temb_key = "token_embd.weight"
+        if temb_key in sd and sd[temb_key].shape == (256384, 4096):
+            # non-standard Comfy-Org tokenizer
+            sd["spiece_model"] = gguf_tokenizer_loader(path, sd[temb_key].shape)
+            # TODO: dequantizing token embed here is janky but otherwise we OOM due to tensor being massive.
+            logging.warning(f"Dequantizing {temb_key} to prevent runtime OOM.")
+            sd[temb_key] = dequantize_tensor(sd[temb_key], dtype=torch.float16)
+        sd = sd_map_replace(sd, T5_SD_MAP)
+    elif arch in {"llama", "qwen2vl"}:
+        # TODO: pass model_options["vocab_size"] to loader somehow
+        temb_key = "token_embd.weight"
+        if temb_key in sd and sd[temb_key].shape[0] >= (64 * 1024):
+            # See note above for T5.
+            logging.warning(f"Dequantizing {temb_key} to prevent runtime OOM.")
+            sd[temb_key] = dequantize_tensor(sd[temb_key], dtype=torch.float16)
+        sd = sd_map_replace(sd, LLAMA_SD_MAP)
+        if arch == "llama":
+            sd = llama_permute(sd, 32, 8) # L3
+    else:
+        pass
+    return sd

custom_nodes/ComfyUI-GGUF/nodes.py

@@ -0,0 +1,305 @@
+# (c) City96 || Apache-2.0 (apache.org/licenses/LICENSE-2.0)
+import torch
+import logging
+import collections
+
+import nodes
+import comfy.sd
+import comfy.lora
+import comfy.float
+import comfy.utils
+import comfy.model_patcher
+import comfy.model_management
+import folder_paths
+
+from .ops import GGMLOps, move_patch_to_device
+from .loader import gguf_sd_loader, gguf_clip_loader
+from .dequant import is_quantized, is_torch_compatible
+
+def update_folder_names_and_paths(key, targets=[]):
+    # check for existing key
+    base = folder_paths.folder_names_and_paths.get(key, ([], {}))
+    base = base[0] if isinstance(base[0], (list, set, tuple)) else []
+    # find base key & add w/ fallback, sanity check + warning
+    target = next((x for x in targets if x in folder_paths.folder_names_and_paths), targets[0])
+    orig, _ = folder_paths.folder_names_and_paths.get(target, ([], {}))
+    folder_paths.folder_names_and_paths[key] = (orig or base, {".gguf"})
+    if base and base != orig:
+        logging.warning(f"Unknown file list already present on key {key}: {base}")
+
+# Add a custom keys for files ending in .gguf
+update_folder_names_and_paths("unet_gguf", ["diffusion_models", "unet"])
+update_folder_names_and_paths("clip_gguf", ["text_encoders", "clip"])
+
+class GGUFModelPatcher(comfy.model_patcher.ModelPatcher):
+    patch_on_device = False
+
+    def patch_weight_to_device(self, key, device_to=None, inplace_update=False):
+        if key not in self.patches:
+            return
+        weight = comfy.utils.get_attr(self.model, key)
+
+        patches = self.patches[key]
+        if is_quantized(weight):
+            out_weight = weight.to(device_to)
+            patches = move_patch_to_device(patches, self.load_device if self.patch_on_device else self.offload_device)
+            # TODO: do we ever have legitimate duplicate patches? (i.e. patch on top of patched weight)
+            out_weight.patches = [(patches, key)]
+        else:
+            inplace_update = self.weight_inplace_update or inplace_update
+            if key not in self.backup:
+                self.backup[key] = collections.namedtuple('Dimension', ['weight', 'inplace_update'])(
+                    weight.to(device=self.offload_device, copy=inplace_update), inplace_update
+                )
+
+            if device_to is not None:
+                temp_weight = comfy.model_management.cast_to_device(weight, device_to, torch.float32, copy=True)
+            else:
+                temp_weight = weight.to(torch.float32, copy=True)
+
+            out_weight = comfy.lora.calculate_weight(patches, temp_weight, key)
+            out_weight = comfy.float.stochastic_rounding(out_weight, weight.dtype)
+
+        if inplace_update:
+            comfy.utils.copy_to_param(self.model, key, out_weight)
+        else:
+            comfy.utils.set_attr_param(self.model, key, out_weight)
+
+    def unpatch_model(self, device_to=None, unpatch_weights=True):
+        if unpatch_weights:
+            for p in self.model.parameters():
+                if is_torch_compatible(p):
+                    continue
+                patches = getattr(p, "patches", [])
+                if len(patches) > 0:
+                    p.patches = []
+        # TODO: Find another way to not unload after patches
+        return super().unpatch_model(device_to=device_to, unpatch_weights=unpatch_weights)
+
+    mmap_released = False
+    def load(self, *args, force_patch_weights=False, **kwargs):
+        # always call `patch_weight_to_device` even for lowvram
+        super().load(*args, force_patch_weights=True, **kwargs)
+
+        # make sure nothing stays linked to mmap after first load
+        if not self.mmap_released:
+            linked = []
+            if kwargs.get("lowvram_model_memory", 0) > 0:
+                for n, m in self.model.named_modules():
+                    if hasattr(m, "weight"):
+                        device = getattr(m.weight, "device", None)
+                        if device == self.offload_device:
+                            linked.append((n, m))
+                            continue
+                    if hasattr(m, "bias"):
+                        device = getattr(m.bias, "device", None)
+                        if device == self.offload_device:
+                            linked.append((n, m))
+                            continue
+            if linked and self.load_device != self.offload_device:
+                logging.info(f"Attempting to release mmap ({len(linked)})")
+                for n, m in linked:
+                    # TODO: possible to OOM, find better way to detach
+                    m.to(self.load_device).to(self.offload_device)
+            self.mmap_released = True
+
+    def clone(self, *args, **kwargs):
+        src_cls = self.__class__
+        self.__class__ = GGUFModelPatcher
+        n = super().clone(*args, **kwargs)
+        n.__class__ = GGUFModelPatcher
+        self.__class__ = src_cls
+        # GGUF specific clone values below
+        n.patch_on_device = getattr(self, "patch_on_device", False)
+        if src_cls != GGUFModelPatcher:
+            n.size = 0 # force recalc
+        return n
+
+class UnetLoaderGGUF:
+    @classmethod
+    def INPUT_TYPES(s):
+        unet_names = [x for x in folder_paths.get_filename_list("unet_gguf")]
+        return {
+            "required": {
+                "unet_name": (unet_names,),
+            }
+        }
+
+    RETURN_TYPES = ("MODEL",)
+    FUNCTION = "load_unet"
+    CATEGORY = "bootleg"
+    TITLE = "Unet Loader (GGUF)"
+
+    def load_unet(self, unet_name, dequant_dtype=None, patch_dtype=None, patch_on_device=None):
+        ops = GGMLOps()
+
+        if dequant_dtype in ("default", None):
+            ops.Linear.dequant_dtype = None
+        elif dequant_dtype in ["target"]:
+            ops.Linear.dequant_dtype = dequant_dtype
+        else:
+            ops.Linear.dequant_dtype = getattr(torch, dequant_dtype)
+
+        if patch_dtype in ("default", None):
+            ops.Linear.patch_dtype = None
+        elif patch_dtype in ["target"]:
+            ops.Linear.patch_dtype = patch_dtype
+        else:
+            ops.Linear.patch_dtype = getattr(torch, patch_dtype)
+
+        # init model
+        unet_path = folder_paths.get_full_path("unet", unet_name)
+        sd = gguf_sd_loader(unet_path)
+        model = comfy.sd.load_diffusion_model_state_dict(
+            sd, model_options={"custom_operations": ops}
+        )
+        if model is None:
+            logging.error("ERROR UNSUPPORTED UNET {}".format(unet_path))
+            raise RuntimeError("ERROR: Could not detect model type of: {}".format(unet_path))
+        model = GGUFModelPatcher.clone(model)
+        model.patch_on_device = patch_on_device
+        return (model,)
+
+class UnetLoaderGGUFAdvanced(UnetLoaderGGUF):
+    @classmethod
+    def INPUT_TYPES(s):
+        unet_names = [x for x in folder_paths.get_filename_list("unet_gguf")]
+        return {
+            "required": {
+                "unet_name": (unet_names,),
+                "dequant_dtype": (["default", "target", "float32", "float16", "bfloat16"], {"default": "default"}),
+                "patch_dtype": (["default", "target", "float32", "float16", "bfloat16"], {"default": "default"}),
+                "patch_on_device": ("BOOLEAN", {"default": False}),
+            }
+        }
+    TITLE = "Unet Loader (GGUF/Advanced)"
+
+class CLIPLoaderGGUF:
+    @classmethod
+    def INPUT_TYPES(s):
+        base = nodes.CLIPLoader.INPUT_TYPES()
+        return {
+            "required": {
+                "clip_name": (s.get_filename_list(),),
+                "type": base["required"]["type"],
+            }
+        }
+
+    RETURN_TYPES = ("CLIP",)
+    FUNCTION = "load_clip"
+    CATEGORY = "bootleg"
+    TITLE = "CLIPLoader (GGUF)"
+
+    @classmethod
+    def get_filename_list(s):
+        files = []
+        files += folder_paths.get_filename_list("clip")
+        files += folder_paths.get_filename_list("clip_gguf")
+        return sorted(files)
+
+    def load_data(self, ckpt_paths):
+        clip_data = []
+        for p in ckpt_paths:
+            if p.endswith(".gguf"):
+                sd = gguf_clip_loader(p)
+            else:
+                sd = comfy.utils.load_torch_file(p, safe_load=True)
+                if "scaled_fp8" in sd: # NOTE: Scaled FP8 would require different custom ops, but only one can be active
+                    raise NotImplementedError(f"Mixing scaled FP8 with GGUF is not supported! Use regular CLIP loader or switch model(s)\n({p})")
+            clip_data.append(sd)
+        return clip_data
+
+    def load_patcher(self, clip_paths, clip_type, clip_data):
+        clip = comfy.sd.load_text_encoder_state_dicts(
+            clip_type = clip_type,
+            state_dicts = clip_data,
+            model_options = {
+                "custom_operations": GGMLOps,
+                "initial_device": comfy.model_management.text_encoder_offload_device()
+            },
+            embedding_directory = folder_paths.get_folder_paths("embeddings"),
+        )
+        clip.patcher = GGUFModelPatcher.clone(clip.patcher)
+        return clip
+
+    def load_clip(self, clip_name, type="stable_diffusion"):
+        clip_path = folder_paths.get_full_path("clip", clip_name)
+        clip_type = getattr(comfy.sd.CLIPType, type.upper(), comfy.sd.CLIPType.STABLE_DIFFUSION)
+        return (self.load_patcher([clip_path], clip_type, self.load_data([clip_path])),)
+
+class DualCLIPLoaderGGUF(CLIPLoaderGGUF):
+    @classmethod
+    def INPUT_TYPES(s):
+        base = nodes.DualCLIPLoader.INPUT_TYPES()
+        file_options = (s.get_filename_list(), )
+        return {
+            "required": {
+                "clip_name1": file_options,
+                "clip_name2": file_options,
+                "type": base["required"]["type"],
+            }
+        }
+
+    TITLE = "DualCLIPLoader (GGUF)"
+
+    def load_clip(self, clip_name1, clip_name2, type):
+        clip_path1 = folder_paths.get_full_path("clip", clip_name1)
+        clip_path2 = folder_paths.get_full_path("clip", clip_name2)
+        clip_paths = (clip_path1, clip_path2)
+        clip_type = getattr(comfy.sd.CLIPType, type.upper(), comfy.sd.CLIPType.STABLE_DIFFUSION)
+        return (self.load_patcher(clip_paths, clip_type, self.load_data(clip_paths)),)
+
+class TripleCLIPLoaderGGUF(CLIPLoaderGGUF):
+    @classmethod
+    def INPUT_TYPES(s):
+        file_options = (s.get_filename_list(), )
+        return {
+            "required": {
+                "clip_name1": file_options,
+                "clip_name2": file_options,
+                "clip_name3": file_options,
+            }
+        }
+
+    TITLE = "TripleCLIPLoader (GGUF)"
+
+    def load_clip(self, clip_name1, clip_name2, clip_name3, type="sd3"):
+        clip_path1 = folder_paths.get_full_path("clip", clip_name1)
+        clip_path2 = folder_paths.get_full_path("clip", clip_name2)
+        clip_path3 = folder_paths.get_full_path("clip", clip_name3)
+        clip_paths = (clip_path1, clip_path2, clip_path3)
+        clip_type = getattr(comfy.sd.CLIPType, type.upper(), comfy.sd.CLIPType.STABLE_DIFFUSION)
+        return (self.load_patcher(clip_paths, clip_type, self.load_data(clip_paths)),)
+
+class QuadrupleCLIPLoaderGGUF(CLIPLoaderGGUF):
+    @classmethod
+    def INPUT_TYPES(s):
+        file_options = (s.get_filename_list(), )
+        return {
+            "required": {
+            "clip_name1": file_options,
+            "clip_name2": file_options,
+            "clip_name3": file_options,
+            "clip_name4": file_options,
+        }
+    }
+
+    TITLE = "QuadrupleCLIPLoader (GGUF)"
+
+    def load_clip(self, clip_name1, clip_name2, clip_name3, clip_name4, type="stable_diffusion"):
+        clip_path1 = folder_paths.get_full_path("clip", clip_name1)
+        clip_path2 = folder_paths.get_full_path("clip", clip_name2)
+        clip_path3 = folder_paths.get_full_path("clip", clip_name3)
+        clip_path4 = folder_paths.get_full_path("clip", clip_name4)
+        clip_paths = (clip_path1, clip_path2, clip_path3, clip_path4)
+        clip_type = getattr(comfy.sd.CLIPType, type.upper(), comfy.sd.CLIPType.STABLE_DIFFUSION)
+        return (self.load_patcher(clip_paths, clip_type, self.load_data(clip_paths)),)
+
+NODE_CLASS_MAPPINGS = {
+    "UnetLoaderGGUF": UnetLoaderGGUF,
+    "CLIPLoaderGGUF": CLIPLoaderGGUF,
+    "DualCLIPLoaderGGUF": DualCLIPLoaderGGUF,
+    "TripleCLIPLoaderGGUF": TripleCLIPLoaderGGUF,
+    "QuadrupleCLIPLoaderGGUF": QuadrupleCLIPLoaderGGUF,
+    "UnetLoaderGGUFAdvanced": UnetLoaderGGUFAdvanced,
+}

custom_nodes/ComfyUI-GGUF/ops.py

@@ -0,0 +1,281 @@
+# (c) City96 || Apache-2.0 (apache.org/licenses/LICENSE-2.0)
+import gguf
+import torch
+import logging
+
+import comfy.ops
+import comfy.lora
+import comfy.model_management
+from .dequant import dequantize_tensor, is_quantized
+
+def chained_hasattr(obj, chained_attr):
+    probe = obj
+    for attr in chained_attr.split('.'):
+        if hasattr(probe, attr):
+            probe = getattr(probe, attr)
+        else:
+            return False
+    return True
+
+# A bakcward and forward compatible way to get `torch.compiler.disable`.
+def get_torch_compiler_disable_decorator():
+    def dummy_decorator(*args, **kwargs):
+        def noop(x):
+            return x
+        return noop
+
+    from packaging import version
+
+    if not chained_hasattr(torch, "compiler.disable"):
+        logging.info("ComfyUI-GGUF: Torch too old for torch.compile - bypassing")
+        return dummy_decorator # torch too old
+    elif version.parse(torch.__version__) >= version.parse("2.8"):
+        logging.info("ComfyUI-GGUF: Allowing full torch compile")
+        return dummy_decorator # torch compile works
+    if chained_hasattr(torch, "_dynamo.config.nontraceable_tensor_subclasses"):
+        logging.info("ComfyUI-GGUF: Allowing full torch compile (nightly)")
+        return dummy_decorator # torch compile works, nightly before 2.8 release
+    else:
+        logging.info("ComfyUI-GGUF: Partial torch compile only, consider updating pytorch")
+        return torch.compiler.disable
+
+torch_compiler_disable = get_torch_compiler_disable_decorator()
+
+class GGMLTensor(torch.Tensor):
+    """
+    Main tensor-like class for storing quantized weights
+    """
+    def __init__(self, *args, tensor_type, tensor_shape, patches=[], **kwargs):
+        super().__init__()
+        self.tensor_type = tensor_type
+        self.tensor_shape = tensor_shape
+        self.patches = patches
+
+    def __new__(cls, *args, tensor_type, tensor_shape, patches=[], **kwargs):
+        return super().__new__(cls, *args, **kwargs)
+
+    def to(self, *args, **kwargs):
+        new = super().to(*args, **kwargs)
+        new.tensor_type = getattr(self, "tensor_type", None)
+        new.tensor_shape = getattr(self, "tensor_shape", new.data.shape)
+        new.patches = getattr(self, "patches", []).copy()
+        return new
+
+    def clone(self, *args, **kwargs):
+        return self
+
+    def detach(self, *args, **kwargs):
+        return self
+
+    def copy_(self, *args, **kwargs):
+        # fixes .weight.copy_ in comfy/clip_model/CLIPTextModel
+        try:
+            return super().copy_(*args, **kwargs)
+        except Exception as e:
+            logging.warning(f"ignoring 'copy_' on tensor: {e}")
+
+    def new_empty(self, size, *args, **kwargs):
+        # Intel Arc fix, ref#50
+        new_tensor = super().new_empty(size, *args, **kwargs)
+        return GGMLTensor(
+                new_tensor,
+                tensor_type = getattr(self, "tensor_type", None),
+                tensor_shape = size,
+                patches = getattr(self, "patches", []).copy()
+        )
+
+    @property
+    def shape(self):
+        if not hasattr(self, "tensor_shape"):
+            self.tensor_shape = self.size()
+        return self.tensor_shape
+
+class GGMLLayer(torch.nn.Module):
+    """
+    This (should) be responsible for de-quantizing on the fly
+    """
+    comfy_cast_weights = True
+    dequant_dtype = None
+    patch_dtype = None
+    largest_layer = False
+    torch_compatible_tensor_types = {None, gguf.GGMLQuantizationType.F32, gguf.GGMLQuantizationType.F16}
+
+    def is_ggml_quantized(self, *, weight=None, bias=None):
+        if weight is None:
+            weight = self.weight
+        if bias is None:
+            bias = self.bias
+        return is_quantized(weight) or is_quantized(bias)
+
+    def _load_from_state_dict(self, state_dict, prefix, *args, **kwargs):
+        weight, bias = state_dict.get(f"{prefix}weight"), state_dict.get(f"{prefix}bias")
+        # NOTE: using modified load for linear due to not initializing on creation, see GGMLOps todo
+        if self.is_ggml_quantized(weight=weight, bias=bias) or isinstance(self, torch.nn.Linear):
+            return self.ggml_load_from_state_dict(state_dict, prefix, *args, **kwargs)
+        # Not strictly required, but fixes embedding shape mismatch. Threshold set in loader.py
+        if isinstance(self, torch.nn.Embedding) and self.weight.shape[0] >= (64 * 1024):
+            return self.ggml_load_from_state_dict(state_dict, prefix, *args, **kwargs)
+        return super()._load_from_state_dict(state_dict, prefix, *args, **kwargs)
+
+    def ggml_load_from_state_dict(self, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs):
+        prefix_len = len(prefix)
+        for k,v in state_dict.items():
+            if k[prefix_len:] == "weight":
+                self.weight = torch.nn.Parameter(v, requires_grad=False)
+            elif k[prefix_len:] == "bias" and v is not None:
+                self.bias = torch.nn.Parameter(v, requires_grad=False)
+            else:
+                unexpected_keys.append(k)
+
+        # For Linear layer with missing weight
+        if self.weight is None and isinstance(self, torch.nn.Linear):
+            v = torch.zeros(self.in_features, self.out_features)
+            self.weight = torch.nn.Parameter(v, requires_grad=False)
+            missing_keys.append(prefix+"weight")
+
+        # for vram estimation (TODO: less fragile logic?)
+        if getattr(self.weight, "is_largest_weight", False):
+            self.largest_layer = True
+
+    def _save_to_state_dict(self, *args, **kwargs):
+        if self.is_ggml_quantized():
+            return self.ggml_save_to_state_dict(*args, **kwargs)
+        return super()._save_to_state_dict(*args, **kwargs)
+
+    def ggml_save_to_state_dict(self, destination, prefix, keep_vars):
+        # This is a fake state dict for vram estimation
+        weight = torch.zeros_like(self.weight, device=torch.device("meta"))
+        destination[prefix + "weight"] = weight
+        if self.bias is not None:
+            bias = torch.zeros_like(self.bias, device=torch.device("meta"))
+            destination[prefix + "bias"] = bias
+
+        # Take into account space required for dequantizing the largest tensor
+        if self.largest_layer:
+            shape = getattr(self.weight, "tensor_shape", self.weight.shape)
+            dtype = self.dequant_dtype or torch.float16
+            temp = torch.empty(*shape, device=torch.device("meta"), dtype=dtype)
+            destination[prefix + "temp.weight"] = temp
+
+        return
+        # This would return the dequantized state dict
+        destination[prefix + "weight"] = self.get_weight(self.weight)
+        if bias is not None:
+            destination[prefix + "bias"] = self.get_weight(self.bias)
+
+    def get_weight(self, tensor, dtype):
+        if tensor is None:
+            return
+
+        # consolidate and load patches to GPU in async
+        patch_list = []
+        device = tensor.device
+        for patches, key in getattr(tensor, "patches", []):
+            patch_list += move_patch_to_device(patches, device)
+
+        # dequantize tensor while patches load
+        weight = dequantize_tensor(tensor, dtype, self.dequant_dtype)
+
+        # prevent propagating custom tensor class
+        if isinstance(weight, GGMLTensor):
+            weight = torch.Tensor(weight)
+
+        # apply patches
+        if len(patch_list) > 0:
+            if self.patch_dtype is None:
+                weight = comfy.lora.calculate_weight(patch_list, weight, key)
+            else:
+                # for testing, may degrade image quality
+                patch_dtype = dtype if self.patch_dtype == "target" else self.patch_dtype
+                weight = comfy.lora.calculate_weight(patch_list, weight, key, patch_dtype)
+        return weight
+
+    @torch_compiler_disable()
+    def cast_bias_weight(s, input=None, dtype=None, device=None, bias_dtype=None):
+        if input is not None:
+            if dtype is None:
+                dtype = getattr(input, "dtype", torch.float32)
+            if bias_dtype is None:
+                bias_dtype = dtype
+            if device is None:
+                device = input.device
+
+        bias = None
+        non_blocking = comfy.model_management.device_supports_non_blocking(device)
+        if s.bias is not None:
+            bias = s.get_weight(s.bias.to(device), dtype)
+            bias = comfy.ops.cast_to(bias, bias_dtype, device, non_blocking=non_blocking, copy=False)
+
+        weight = s.get_weight(s.weight.to(device), dtype)
+        weight = comfy.ops.cast_to(weight, dtype, device, non_blocking=non_blocking, copy=False)
+        return weight, bias
+
+    def forward_comfy_cast_weights(self, input, *args, **kwargs):
+        if self.is_ggml_quantized():
+            out = self.forward_ggml_cast_weights(input, *args, **kwargs)
+        else:
+            out = super().forward_comfy_cast_weights(input, *args, **kwargs)
+
+        # non-ggml forward might still propagate custom tensor class
+        if isinstance(out, GGMLTensor):
+            out = torch.Tensor(out)
+        return out
+
+    def forward_ggml_cast_weights(self, input):
+        raise NotImplementedError
+
+class GGMLOps(comfy.ops.manual_cast):
+    """
+    Dequantize weights on the fly before doing the compute
+    """
+    class Linear(GGMLLayer, comfy.ops.manual_cast.Linear):
+        def __init__(self, in_features, out_features, bias=True, device=None, dtype=None):
+            torch.nn.Module.__init__(self)
+            # TODO: better workaround for reserved memory spike on windows
+            # Issue is with `torch.empty` still reserving the full memory for the layer
+            # Windows doesn't over-commit memory so without this 24GB+ of pagefile is used
+            self.in_features = in_features
+            self.out_features = out_features
+            self.weight = None
+            self.bias = None
+
+        def forward_ggml_cast_weights(self, input):
+            weight, bias = self.cast_bias_weight(input)
+            return torch.nn.functional.linear(input, weight, bias)
+
+    class Conv2d(GGMLLayer, comfy.ops.manual_cast.Conv2d):
+        def forward_ggml_cast_weights(self, input):
+            weight, bias = self.cast_bias_weight(input)
+            return self._conv_forward(input, weight, bias)
+
+    class Embedding(GGMLLayer, comfy.ops.manual_cast.Embedding):
+        def forward_ggml_cast_weights(self, input, out_dtype=None):
+            output_dtype = out_dtype
+            if self.weight.dtype == torch.float16 or self.weight.dtype == torch.bfloat16:
+                out_dtype = None
+            weight, _bias = self.cast_bias_weight(self, device=input.device, dtype=out_dtype)
+            return torch.nn.functional.embedding(
+                input, weight, self.padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq, self.sparse
+            ).to(dtype=output_dtype)
+
+    class LayerNorm(GGMLLayer, comfy.ops.manual_cast.LayerNorm):
+        def forward_ggml_cast_weights(self, input):
+            if self.weight is None:
+                return super().forward_comfy_cast_weights(input)
+            weight, bias = self.cast_bias_weight(input)
+            return torch.nn.functional.layer_norm(input, self.normalized_shape, weight, bias, self.eps)
+
+    class GroupNorm(GGMLLayer, comfy.ops.manual_cast.GroupNorm):
+        def forward_ggml_cast_weights(self, input):
+            weight, bias = self.cast_bias_weight(input)
+            return torch.nn.functional.group_norm(input, self.num_groups, weight, bias, self.eps)
+
+def move_patch_to_device(item, device):
+    if isinstance(item, torch.Tensor):
+        return item.to(device, non_blocking=True)
+    elif isinstance(item, tuple):
+        return tuple(move_patch_to_device(x, device) for x in item)
+    elif isinstance(item, list):
+        return [move_patch_to_device(x, device) for x in item]
+    else:
+        return item

custom_nodes/ComfyUI-GGUF/pyproject.toml

@@ -0,0 +1,14 @@
+[project]
+name = "ComfyUI-GGUF"
+description = "GGUF Quantization support for native ComfyUI models."
+version = "1.1.3" # 2.0.0 = GitHub main, 1.X.X = ComfyUI Registry
+license = { file = "LICENSE" }
+dependencies = ["gguf>=0.13.0", "sentencepiece", "protobuf"]
+
+[project.urls]
+Repository = "https://github.com/city96/ComfyUI-GGUF"
+
+[tool.comfy]
+PublisherId = "city96"
+DisplayName = "ComfyUI-GGUF"
+Icon = ""

custom_nodes/ComfyUI-GGUF/requirements.txt

@@ -0,0 +1,5 @@
+# main
+gguf>=0.13.0
+# optional - tokenizer
+sentencepiece
+protobuf

custom_nodes/ComfyUI-GGUF/tools/README.md

@@ -0,0 +1,93 @@
+## Converting initial model
+
+To convert your initial safetensors/ckpt model to FP16/BF16 GGUF, run the following command:
+
+```
+python convert.py --src E:\models\unet\flux1-dev.safetensors
+```
+Make sure `gguf>=0.13.0` is installed for this step. Optionally, specify the output gguf file with the `--dst` arg.
+
+> [!NOTE]  
+> Do not use the diffusers UNET format for flux, it won't work, use the default/reference checkpoint key format. This is due to q/k/v being merged into one qkv key.
+> You can convert it by loading it in ComfyUI and saving it using the built-in "ModelSave" node.
+
+> [!WARNING] 
+> For hunyuan video/wan 2.1, you will see a warning about 5D tensors. This means the script will save a **non functional** model to disk first, that you can quantize. I recommend saving these in a separate `raw` folder to avoid confusion.
+> 
+> After quantization, you will have to run `fix_5d_tensor.py` manually to add back the missing key that was saved by the conversion code.
+
+## Quantizing using custom llama.cpp
+
+Depending on your git settings, you may need to run the following script first in order to make sure the patch file is valid. It will convert Windows (CRLF) line endings to Unix (LF) ones.
+
+```
+python fix_lines_ending.py
+```
+
+Git clone llama.cpp into the current folder:
+
+```
+git clone https://github.com/ggerganov/llama.cpp
+```
+
+Check out the correct branch, then apply the custom patch needed to add image model support to the repo you just cloned.
+
+```
+cd llama.cpp
+git checkout tags/b3962
+git apply ..\lcpp.patch
+```
+
+Compile the llama-quantize binary. This example uses cmake, on linux you can just use make.
+
+### Visual Studio 2019, Linux, etc...
+
+```
+mkdir build
+cmake -B build
+cmake --build build --config Debug -j10 --target llama-quantize
+cd ..
+```
+
+### Visual Studio 2022
+
+```
+mkdir build
+cmake -B build -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_FLAGS="-std=c++17"
+```
+
+Edit the `llama.cpp\common\log.cpp` file, inserts two lines after the existing first line:
+
+```
+#include "log.h"
+
+#define _SILENCE_CXX23_CHRONO_DEPRECATION_WARNING
+#include <chrono>
+```
+
+Then you can build the project:
+```
+cmake --build build --config Debug -j10 --target llama-quantize
+cd ..
+```
+
+### Quantize your model
+
+
+Now you can use the newly build binary to quantize your model to the desired format:
+```
+llama.cpp\build\bin\Debug\llama-quantize.exe E:\models\unet\flux1-dev-BF16.gguf E:\models\unet\flux1-dev-Q4_K_S.gguf Q4_K_S
+```
+
+You can extract the patch again with `git diff src\llama.cpp > lcpp.patch` if you wish to change something and contribute back.
+
+> [!WARNING] 
+> For hunyuan video/wan 2.1, you will have to run `fix_5d_tensor.py` after the quantization step is done.
+>
+> Example usage:  `fix_5d_tensors.py --src E:\models\video\raw\wan2.1-t2v-1.3b-Q8_0.gguf --dst E:\models\video\wan2.1-t2v-1.3b-Q8_0.gguf`
+>
+> By default, this also saves a `fix_5d_tensors_[arch].safetensors` file in the `ComfyUI-GGUF/tools` folder, it's recommended to delete this after all models have been converted.
+
+> [!NOTE]
+> Do not quantize SDXL / SD1 / other Conv2D heavy models. If you do, make sure to **extract the UNET model first**.
+>This should be obvious, but also don't use the resulting llama-quantize binary with LLMs.

custom_nodes/ComfyUI-GGUF/tools/convert.py

@@ -0,0 +1,365 @@
+# (c) City96 || Apache-2.0 (apache.org/licenses/LICENSE-2.0)
+import os
+import gguf
+import torch
+import logging
+import argparse
+from tqdm import tqdm
+from safetensors.torch import load_file, save_file
+
+QUANTIZATION_THRESHOLD = 1024
+REARRANGE_THRESHOLD = 512
+MAX_TENSOR_NAME_LENGTH = 127
+MAX_TENSOR_DIMS = 4
+
+class ModelTemplate:
+    arch = "invalid"  # string describing architecture
+    shape_fix = False # whether to reshape tensors
+    keys_detect = []  # list of lists to match in state dict
+    keys_banned = []  # list of keys that should mark model as invalid for conversion
+    keys_hiprec = []  # list of keys that need to be kept in fp32 for some reason
+    keys_ignore = []  # list of strings to ignore keys by when found
+
+    def handle_nd_tensor(self, key, data):
+        raise NotImplementedError(f"Tensor detected that exceeds dims supported by C++ code! ({key} @ {data.shape})")
+
+class ModelFlux(ModelTemplate):
+    arch = "flux"
+    keys_detect = [
+        ("transformer_blocks.0.attn.norm_added_k.weight",),
+        ("double_blocks.0.img_attn.proj.weight",),
+    ]
+    keys_banned = ["transformer_blocks.0.attn.norm_added_k.weight",]
+
+class ModelSD3(ModelTemplate):
+    arch = "sd3"
+    keys_detect = [
+        ("transformer_blocks.0.attn.add_q_proj.weight",),
+        ("joint_blocks.0.x_block.attn.qkv.weight",),
+    ]
+    keys_banned = ["transformer_blocks.0.attn.add_q_proj.weight",]
+
+class ModelAura(ModelTemplate):
+    arch = "aura"
+    keys_detect = [
+        ("double_layers.3.modX.1.weight",),
+        ("joint_transformer_blocks.3.ff_context.out_projection.weight",),
+    ]
+    keys_banned = ["joint_transformer_blocks.3.ff_context.out_projection.weight",]
+
+class ModelHiDream(ModelTemplate):
+    arch = "hidream"
+    keys_detect = [
+        (
+            "caption_projection.0.linear.weight",
+            "double_stream_blocks.0.block.ff_i.shared_experts.w3.weight"
+        )
+    ]
+    keys_hiprec = [
+        # nn.parameter, can't load from BF16 ver
+        ".ff_i.gate.weight",
+        "img_emb.emb_pos"
+    ]
+
+class CosmosPredict2(ModelTemplate):
+    arch = "cosmos"
+    keys_detect = [
+        (
+            "blocks.0.mlp.layer1.weight",
+            "blocks.0.adaln_modulation_cross_attn.1.weight",
+        )
+    ]
+    keys_hiprec = ["pos_embedder"]
+    keys_ignore = ["_extra_state", "accum_"]
+
+class ModelHyVid(ModelTemplate):
+    arch = "hyvid"
+    keys_detect = [
+        (
+            "double_blocks.0.img_attn_proj.weight",
+            "txt_in.individual_token_refiner.blocks.1.self_attn_qkv.weight",
+        )
+    ]
+
+    def handle_nd_tensor(self, key, data):
+        # hacky but don't have any better ideas
+        path = f"./fix_5d_tensors_{self.arch}.safetensors" # TODO: somehow get a path here??
+        if os.path.isfile(path):
+            raise RuntimeError(f"5D tensor fix file already exists! {path}")
+        fsd = {key: torch.from_numpy(data)}
+        tqdm.write(f"5D key found in state dict! Manual fix required! - {key} {data.shape}")
+        save_file(fsd, path)
+
+class ModelWan(ModelHyVid):
+    arch = "wan"
+    keys_detect = [
+        (
+            "blocks.0.self_attn.norm_q.weight",
+            "text_embedding.2.weight",
+            "head.modulation",
+        )
+    ]
+    keys_hiprec = [
+        ".modulation" # nn.parameter, can't load from BF16 ver
+    ]
+
+class ModelLTXV(ModelTemplate):
+    arch = "ltxv"
+    keys_detect = [
+        (
+            "adaln_single.emb.timestep_embedder.linear_2.weight",
+            "transformer_blocks.27.scale_shift_table",
+            "caption_projection.linear_2.weight",
+        )
+    ]
+    keys_hiprec = [
+        "scale_shift_table" # nn.parameter, can't load from BF16 base quant
+    ]
+
+class ModelSDXL(ModelTemplate):
+    arch = "sdxl"
+    shape_fix = True
+    keys_detect = [
+        ("down_blocks.0.downsamplers.0.conv.weight", "add_embedding.linear_1.weight",),
+        (
+            "input_blocks.3.0.op.weight", "input_blocks.6.0.op.weight",
+            "output_blocks.2.2.conv.weight", "output_blocks.5.2.conv.weight",
+        ), # Non-diffusers
+        ("label_emb.0.0.weight",),
+    ]
+
+class ModelSD1(ModelTemplate):
+    arch = "sd1"
+    shape_fix = True
+    keys_detect = [
+        ("down_blocks.0.downsamplers.0.conv.weight",),
+        (
+            "input_blocks.3.0.op.weight", "input_blocks.6.0.op.weight", "input_blocks.9.0.op.weight",
+            "output_blocks.2.1.conv.weight", "output_blocks.5.2.conv.weight", "output_blocks.8.2.conv.weight"
+        ), # Non-diffusers
+    ]
+
+class ModelLumina2(ModelTemplate):
+    arch = "lumina2"
+    keys_detect = [
+        ("cap_embedder.1.weight", "context_refiner.0.attention.qkv.weight")
+    ]
+
+arch_list = [ModelFlux, ModelSD3, ModelAura, ModelHiDream, CosmosPredict2, 
+             ModelLTXV, ModelHyVid, ModelWan, ModelSDXL, ModelSD1, ModelLumina2]
+
+def is_model_arch(model, state_dict):
+    # check if model is correct
+    matched = False
+    invalid = False
+    for match_list in model.keys_detect:
+        if all(key in state_dict for key in match_list):
+            matched = True
+            invalid = any(key in state_dict for key in model.keys_banned)
+            break
+    assert not invalid, "Model architecture not allowed for conversion! (i.e. reference VS diffusers format)"
+    return matched
+
+def detect_arch(state_dict):
+    model_arch = None
+    for arch in arch_list:
+        if is_model_arch(arch, state_dict):
+            model_arch = arch()
+            break
+    assert model_arch is not None, "Unknown model architecture!"
+    return model_arch
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="Generate F16 GGUF files from single UNET")
+    parser.add_argument("--src", required=True, help="Source model ckpt file.")
+    parser.add_argument("--dst", help="Output unet gguf file.")
+    args = parser.parse_args()
+
+    if not os.path.isfile(args.src):
+        parser.error("No input provided!")
+
+    return args
+
+def strip_prefix(state_dict):
+    # prefix for mixed state dict
+    prefix = None
+    for pfx in ["model.diffusion_model.", "model."]:
+        if any([x.startswith(pfx) for x in state_dict.keys()]):
+            prefix = pfx
+            break
+
+    # prefix for uniform state dict
+    if prefix is None:
+        for pfx in ["net."]:
+            if all([x.startswith(pfx) for x in state_dict.keys()]):
+                prefix = pfx
+                break
+
+    # strip prefix if found
+    if prefix is not None:
+        logging.info(f"State dict prefix found: '{prefix}'")
+        sd = {}
+        for k, v in state_dict.items():
+            if prefix not in k:
+                continue
+            k = k.replace(prefix, "")
+            sd[k] = v
+    else:
+        logging.debug("State dict has no prefix")
+        sd = state_dict
+
+    return sd
+
+def load_state_dict(path):
+    if any(path.endswith(x) for x in [".ckpt", ".pt", ".bin", ".pth"]):
+        state_dict = torch.load(path, map_location="cpu", weights_only=True)
+        for subkey in ["model", "module"]:
+            if subkey in state_dict:
+                state_dict = state_dict[subkey]
+                break
+        if len(state_dict) < 20:
+            raise RuntimeError(f"pt subkey load failed: {state_dict.keys()}")
+    else:
+        state_dict = load_file(path)
+
+    return strip_prefix(state_dict)
+
+def handle_tensors(writer, state_dict, model_arch):
+    name_lengths = tuple(sorted(
+        ((key, len(key)) for key in state_dict.keys()),
+        key=lambda item: item[1],
+        reverse=True,
+    ))
+    if not name_lengths:
+        return
+    max_name_len = name_lengths[0][1]
+    if max_name_len > MAX_TENSOR_NAME_LENGTH:
+        bad_list = ", ".join(f"{key!r} ({namelen})" for key, namelen in name_lengths if namelen > MAX_TENSOR_NAME_LENGTH)
+        raise ValueError(f"Can only handle tensor names up to {MAX_TENSOR_NAME_LENGTH} characters. Tensors exceeding the limit: {bad_list}")
+    for key, data in tqdm(state_dict.items()):
+        old_dtype = data.dtype
+
+        if any(x in key for x in model_arch.keys_ignore):
+            tqdm.write(f"Filtering ignored key: '{key}'")
+            continue
+
+        if data.dtype == torch.bfloat16:
+            data = data.to(torch.float32).numpy()
+        # this is so we don't break torch 2.0.X
+        elif data.dtype in [getattr(torch, "float8_e4m3fn", "_invalid"), getattr(torch, "float8_e5m2", "_invalid")]:
+            data = data.to(torch.float16).numpy()
+        else:
+            data = data.numpy()
+
+        n_dims = len(data.shape)
+        data_shape = data.shape
+        if old_dtype == torch.bfloat16:
+            data_qtype = gguf.GGMLQuantizationType.BF16
+        # elif old_dtype == torch.float32:
+        #     data_qtype = gguf.GGMLQuantizationType.F32
+        else:
+            data_qtype = gguf.GGMLQuantizationType.F16
+
+        # The max no. of dimensions that can be handled by the quantization code is 4
+        if len(data.shape) > MAX_TENSOR_DIMS:
+            model_arch.handle_nd_tensor(key, data)
+            continue # needs to be added back later
+
+        # get number of parameters (AKA elements) in this tensor
+        n_params = 1
+        for dim_size in data_shape:
+            n_params *= dim_size
+
+        if old_dtype in (torch.float32, torch.bfloat16):
+            if n_dims == 1:
+                # one-dimensional tensors should be kept in F32
+                # also speeds up inference due to not dequantizing
+                data_qtype = gguf.GGMLQuantizationType.F32
+
+            elif n_params <= QUANTIZATION_THRESHOLD:
+                # very small tensors
+                data_qtype = gguf.GGMLQuantizationType.F32
+
+            elif any(x in key for x in model_arch.keys_hiprec):
+                # tensors that require max precision
+                data_qtype = gguf.GGMLQuantizationType.F32
+
+        if (model_arch.shape_fix                        # NEVER reshape for models such as flux
+            and n_dims > 1                              # Skip one-dimensional tensors
+            and n_params >= REARRANGE_THRESHOLD         # Only rearrange tensors meeting the size requirement
+            and (n_params / 256).is_integer()           # Rearranging only makes sense if total elements is divisible by 256
+            and not (data.shape[-1] / 256).is_integer() # Only need to rearrange if the last dimension is not divisible by 256
+        ):
+            orig_shape = data.shape
+            data = data.reshape(n_params // 256, 256)
+            writer.add_array(f"comfy.gguf.orig_shape.{key}", tuple(int(dim) for dim in orig_shape))
+
+        try:
+            data = gguf.quants.quantize(data, data_qtype)
+        except (AttributeError, gguf.QuantError) as e:
+            tqdm.write(f"falling back to F16: {e}")
+            data_qtype = gguf.GGMLQuantizationType.F16
+            data = gguf.quants.quantize(data, data_qtype)
+
+        new_name = key # do we need to rename?
+
+        shape_str = f"{{{', '.join(str(n) for n in reversed(data.shape))}}}"
+        tqdm.write(f"{f'%-{max_name_len + 4}s' % f'{new_name}'} {old_dtype} --> {data_qtype.name}, shape = {shape_str}")
+
+        writer.add_tensor(new_name, data, raw_dtype=data_qtype)
+
+def convert_file(path, dst_path=None, interact=True, overwrite=False):
+    # load & run model detection logic
+    state_dict = load_state_dict(path)
+    model_arch = detect_arch(state_dict)
+    logging.info(f"* Architecture detected from input: {model_arch.arch}")
+
+    # detect & set dtype for output file
+    dtypes = [x.dtype for x in state_dict.values()]
+    dtypes = {x:dtypes.count(x) for x in set(dtypes)}
+    main_dtype = max(dtypes, key=dtypes.get)
+
+    if main_dtype == torch.bfloat16:
+        ftype_name = "BF16"
+        ftype_gguf = gguf.LlamaFileType.MOSTLY_BF16
+    # elif main_dtype == torch.float32:
+    #     ftype_name = "F32"
+    #     ftype_gguf = None
+    else:
+        ftype_name = "F16"
+        ftype_gguf = gguf.LlamaFileType.MOSTLY_F16
+
+    if dst_path is None:
+        dst_path = f"{os.path.splitext(path)[0]}-{ftype_name}.gguf"
+    elif "{ftype}" in dst_path: # lcpp logic
+        dst_path = dst_path.replace("{ftype}", ftype_name)
+
+    if os.path.isfile(dst_path) and not overwrite:
+        if interact:
+            input("Output exists enter to continue or ctrl+c to abort!")
+        else:
+            raise OSError("Output exists and overwriting is disabled!")
+
+    # handle actual file
+    writer = gguf.GGUFWriter(path=None, arch=model_arch.arch)
+    writer.add_quantization_version(gguf.GGML_QUANT_VERSION)
+    if ftype_gguf is not None:
+        writer.add_file_type(ftype_gguf)
+
+    handle_tensors(writer, state_dict, model_arch)
+    writer.write_header_to_file(path=dst_path)
+    writer.write_kv_data_to_file()
+    writer.write_tensors_to_file(progress=True)
+    writer.close()
+
+    fix = f"./fix_5d_tensors_{model_arch.arch}.safetensors"
+    if os.path.isfile(fix):
+        logging.warning(f"\n### Warning! Fix file found at '{fix}'")
+        logging.warning(" you most likely need to run 'fix_5d_tensors.py' after quantization.")
+
+    return dst_path, model_arch
+
+if __name__ == "__main__":
+    args = parse_args()
+    convert_file(args.src, args.dst)
+

custom_nodes/ComfyUI-GGUF/tools/fix_5d_tensors.py

@@ -0,0 +1,82 @@
+# (c) City96 || Apache-2.0 (apache.org/licenses/LICENSE-2.0)
+import os
+import gguf
+import torch
+import argparse
+from tqdm import tqdm
+from safetensors.torch import load_file
+
+def get_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--src", required=True)
+    parser.add_argument("--dst", required=True)
+    parser.add_argument("--fix", required=False, help="Defaults to ./fix_5d_tensors_[arch].pt")
+    parser.add_argument("--overwrite", action="store_true")
+    args = parser.parse_args()
+
+    if not os.path.isfile(args.src):
+        parser.error(f"Invalid source file '{args.src}'")
+    if not args.overwrite and os.path.exists(args.dst):
+        parser.error(f"Output exists, use '--overwrite' ({args.dst})")
+
+    return args
+
+def get_arch_str(reader):
+    field = reader.get_field("general.architecture")
+    return str(field.parts[field.data[-1]], encoding="utf-8")
+
+def get_file_type(reader):
+    field = reader.get_field("general.file_type")
+    ft = int(field.parts[field.data[-1]])
+    return gguf.LlamaFileType(ft)
+
+if __name__ == "__main__":
+    args = get_args()
+
+    # read existing
+    reader = gguf.GGUFReader(args.src)
+    arch = get_arch_str(reader)
+    file_type = get_file_type(reader)
+    print(f"Detected arch: '{arch}' (ftype: {str(file_type)})")
+
+    # prep fix
+    if args.fix is None:
+        args.fix = f"./fix_5d_tensors_{arch}.safetensors"
+ 
+    if not os.path.isfile(args.fix):
+        raise OSError(f"No 5D tensor fix file: {args.fix}")
+
+    sd5d = load_file(args.fix)
+    sd5d = {k:v.numpy() for k,v in sd5d.items()}
+    print("5D tensors:", sd5d.keys())
+
+    # prep output
+    writer = gguf.GGUFWriter(path=None, arch=arch)
+    writer.add_quantization_version(gguf.GGML_QUANT_VERSION)
+    writer.add_file_type(file_type)
+
+    added = []
+    def add_extra_key(writer, key, data):
+        global added
+        data_qtype = gguf.GGMLQuantizationType.F32
+        data = gguf.quants.quantize(data, data_qtype)
+        tqdm.write(f"Adding key {key} ({data.shape})")
+        writer.add_tensor(key, data, raw_dtype=data_qtype)
+        added.append(key)
+
+    # main loop to add missing 5D tensor(s)
+    for tensor in tqdm(reader.tensors):
+        writer.add_tensor(tensor.name, tensor.data, raw_dtype=tensor.tensor_type)
+        key5d = tensor.name.replace(".bias", ".weight")
+        if key5d in sd5d.keys():
+            add_extra_key(writer, key5d, sd5d[key5d])
+
+    # brute force for any missed
+    for key, data in sd5d.items():
+        if key not in added:
+            add_extra_key(writer, key, data)
+
+    writer.write_header_to_file(path=args.dst)
+    writer.write_kv_data_to_file()
+    writer.write_tensors_to_file(progress=True)
+    writer.close()

custom_nodes/ComfyUI-GGUF/tools/fix_lines_ending.py

@@ -0,0 +1,31 @@
+import os
+
+files = ["lcpp.patch", "lcpp_sd3.patch"]
+
+def has_unix_line_endings(file_path):
+    try:
+        with open(file_path, 'rb') as file:
+            content = file.read()
+        return b'\r\n' not in content
+    except Exception as e:
+        print(f"Error checking '{file_path}': {e}")
+        return False
+
+def convert_to_linux_format(file_path):
+    try:
+        with open(file_path, 'rb') as file:
+            content = file.read().replace(b'\r\n', b'\n')
+        with open(file_path, 'wb') as file:
+            file.write(content)
+        print(f"'{file_path}' converted to Linux line endings (LF).")
+    except Exception as e:
+        print(f"Error processing '{file_path}': {e}")
+
+for file in files:
+    if os.path.exists(file):
+        if has_unix_line_endings(file):
+            print(f"'{file}' already has Unix line endings (LF). No conversion needed.")
+        else:
+            convert_to_linux_format(file)
+    else:
+        print(f"File '{file}' does not exist.")

custom_nodes/ComfyUI-GGUF/tools/lcpp.patch

@@ -0,0 +1,451 @@
+diff --git a/ggml/include/ggml.h b/ggml/include/ggml.h
+index de3c706f..0267c1fa 100644
+--- a/ggml/include/ggml.h
++++ b/ggml/include/ggml.h
+@@ -223,7 +223,7 @@
+ #define GGML_MAX_OP_PARAMS      64
+ 
+ #ifndef GGML_MAX_NAME
+-#   define GGML_MAX_NAME        64
++#   define GGML_MAX_NAME        128
+ #endif
+ 
+ #define GGML_DEFAULT_N_THREADS  4
+@@ -2449,6 +2449,7 @@ extern "C" {
+ 
+     // manage tensor info
+     GGML_API void gguf_add_tensor(struct gguf_context * ctx, const struct ggml_tensor * tensor);
++    GGML_API void gguf_set_tensor_ndim(struct gguf_context * ctx, const char * name, int n_dim);
+     GGML_API void gguf_set_tensor_type(struct gguf_context * ctx, const char * name, enum ggml_type type);
+     GGML_API void gguf_set_tensor_data(struct gguf_context * ctx, const char * name, const void * data, size_t size);
+ 
+diff --git a/ggml/src/ggml.c b/ggml/src/ggml.c
+index b16c462f..6d1568f1 100644
+--- a/ggml/src/ggml.c
++++ b/ggml/src/ggml.c
+@@ -22960,6 +22960,14 @@ void gguf_add_tensor(
+     ctx->header.n_tensors++;
+ }
+ 
++void gguf_set_tensor_ndim(struct gguf_context * ctx, const char * name, const int n_dim) {
++    const int idx = gguf_find_tensor(ctx, name);
++    if (idx < 0) {
++        GGML_ABORT("tensor not found");
++    }
++    ctx->infos[idx].n_dims = n_dim;
++}
++
+ void gguf_set_tensor_type(struct gguf_context * ctx, const char * name, enum ggml_type type) {
+     const int idx = gguf_find_tensor(ctx, name);
+     if (idx < 0) {
+diff --git a/src/llama.cpp b/src/llama.cpp
+index 24e1f1f0..25db4c69 100644
+--- a/src/llama.cpp
++++ b/src/llama.cpp
+@@ -205,6 +205,17 @@ enum llm_arch {
+     LLM_ARCH_GRANITE,
+     LLM_ARCH_GRANITE_MOE,
+     LLM_ARCH_CHAMELEON,
++    LLM_ARCH_FLUX,
++    LLM_ARCH_SD1,
++    LLM_ARCH_SDXL,
++    LLM_ARCH_SD3,
++    LLM_ARCH_AURA,
++    LLM_ARCH_LTXV,
++    LLM_ARCH_HYVID,
++    LLM_ARCH_WAN,
++    LLM_ARCH_HIDREAM,
++    LLM_ARCH_COSMOS,
++    LLM_ARCH_LUMINA2,
+     LLM_ARCH_UNKNOWN,
+ };
+ 
+@@ -258,6 +269,17 @@ static const std::map<llm_arch, const char *> LLM_ARCH_NAMES = {
+     { LLM_ARCH_GRANITE,         "granite"      },
+     { LLM_ARCH_GRANITE_MOE,     "granitemoe"   },
+     { LLM_ARCH_CHAMELEON,       "chameleon"    },
++    { LLM_ARCH_FLUX,            "flux"         },
++    { LLM_ARCH_SD1,             "sd1"          },
++    { LLM_ARCH_SDXL,            "sdxl"         },
++    { LLM_ARCH_SD3,             "sd3"          },
++    { LLM_ARCH_AURA,            "aura"         },
++    { LLM_ARCH_LTXV,            "ltxv"         },
++    { LLM_ARCH_HYVID,           "hyvid"        },
++    { LLM_ARCH_WAN,             "wan"          },
++    { LLM_ARCH_HIDREAM,         "hidream"      },
++    { LLM_ARCH_COSMOS,          "cosmos"       },
++    { LLM_ARCH_LUMINA2,         "lumina2"      },
+     { LLM_ARCH_UNKNOWN,         "(unknown)"    },
+ };
+ 
+@@ -1531,6 +1553,17 @@ static const std::map<llm_arch, std::map<llm_tensor, const char *>> LLM_TENSOR_N
+             { LLM_TENSOR_ATTN_K_NORM,     "blk.%d.attn_k_norm" },
+         },
+     },
++    { LLM_ARCH_FLUX,    {}},
++    { LLM_ARCH_SD1,     {}},
++    { LLM_ARCH_SDXL,    {}},
++    { LLM_ARCH_SD3,     {}},
++    { LLM_ARCH_AURA,    {}},
++    { LLM_ARCH_LTXV,    {}},
++    { LLM_ARCH_HYVID,   {}},
++    { LLM_ARCH_WAN,     {}},
++    { LLM_ARCH_HIDREAM, {}},
++    { LLM_ARCH_COSMOS,  {}},
++    { LLM_ARCH_LUMINA2, {}},
+     {
+         LLM_ARCH_UNKNOWN,
+         {
+@@ -5403,6 +5436,25 @@ static void llm_load_hparams(
+     // get general kv
+     ml.get_key(LLM_KV_GENERAL_NAME, model.name, false);
+ 
++    // Disable LLM metadata for image models
++    switch (model.arch) {
++        case LLM_ARCH_FLUX:
++        case LLM_ARCH_SD1:
++        case LLM_ARCH_SDXL:
++        case LLM_ARCH_SD3:
++        case LLM_ARCH_AURA:
++        case LLM_ARCH_LTXV:
++        case LLM_ARCH_HYVID:
++        case LLM_ARCH_WAN:
++        case LLM_ARCH_HIDREAM:
++        case LLM_ARCH_COSMOS:
++        case LLM_ARCH_LUMINA2:
++            model.ftype = ml.ftype;
++            return;
++        default:
++            break;
++    }
++
+     // get hparams kv
+     ml.get_key(LLM_KV_VOCAB_SIZE, hparams.n_vocab, false) || ml.get_arr_n(LLM_KV_TOKENIZER_LIST, hparams.n_vocab);
+ 
+@@ -18016,6 +18068,134 @@ static void llama_tensor_dequantize_internal(
+     workers.clear();
+ }
+ 
++static ggml_type img_tensor_get_type(quantize_state_internal & qs, ggml_type new_type, const ggml_tensor * tensor, llama_ftype ftype) {
++    // Special function for quantizing image model tensors
++    const std::string name = ggml_get_name(tensor);
++    const llm_arch arch = qs.model.arch;
++
++    // Sanity check
++    if (
++            (name.find("model.diffusion_model.") != std::string::npos) ||
++            (name.find("first_stage_model.") != std::string::npos) ||
++            (name.find("single_transformer_blocks.") != std::string::npos) ||
++            (name.find("joint_transformer_blocks.") != std::string::npos)
++        ) {
++            throw std::runtime_error("Invalid input GGUF file. This is not a supported UNET model");
++    }
++
++    // Unsupported quant types - exclude all IQ quants for now
++    if (ftype == LLAMA_FTYPE_MOSTLY_IQ2_XXS || ftype == LLAMA_FTYPE_MOSTLY_IQ2_XS  ||
++        ftype == LLAMA_FTYPE_MOSTLY_IQ2_S   || ftype == LLAMA_FTYPE_MOSTLY_IQ2_M  ||
++        ftype == LLAMA_FTYPE_MOSTLY_IQ3_XXS || ftype == LLAMA_FTYPE_MOSTLY_IQ1_S  ||
++        ftype == LLAMA_FTYPE_MOSTLY_IQ1_M   || ftype == LLAMA_FTYPE_MOSTLY_IQ4_NL ||
++        ftype == LLAMA_FTYPE_MOSTLY_IQ4_XS  || ftype == LLAMA_FTYPE_MOSTLY_IQ3_S  ||
++        ftype == LLAMA_FTYPE_MOSTLY_IQ3_M   || ftype == LLAMA_FTYPE_MOSTLY_Q4_0_4_4 ||
++        ftype == LLAMA_FTYPE_MOSTLY_Q4_0_4_8 || ftype == LLAMA_FTYPE_MOSTLY_Q4_0_8_8) {
++        throw std::runtime_error("Invalid quantization type for image model (Not supported)");
++    }
++
++    if ( // Rules for to_v attention
++            (name.find("attn_v.weight") != std::string::npos) ||
++            (name.find(".to_v.weight") != std::string::npos) ||
++            (name.find(".v.weight") != std::string::npos) ||
++            (name.find(".attn.w1v.weight") != std::string::npos) ||
++            (name.find(".attn.w2v.weight") != std::string::npos) ||
++            (name.find("_attn.v_proj.weight") != std::string::npos)
++        ){
++            if (ftype == LLAMA_FTYPE_MOSTLY_Q2_K) {
++                new_type = GGML_TYPE_Q3_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q3_K_M) {
++                new_type = qs.i_attention_wv < 2 ? GGML_TYPE_Q5_K : GGML_TYPE_Q4_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q3_K_L) {
++                new_type = GGML_TYPE_Q5_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q4_K_M || ftype == LLAMA_FTYPE_MOSTLY_Q5_K_M) {
++                new_type = GGML_TYPE_Q6_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q4_K_S && qs.i_attention_wv < 4) {
++                new_type = GGML_TYPE_Q5_K;
++            }
++            ++qs.i_attention_wv;
++    } else if ( // Rules for fused qkv attention
++            (name.find("attn_qkv.weight") != std::string::npos) ||
++            (name.find("attn.qkv.weight") != std::string::npos) ||
++            (name.find("attention.qkv.weight") != std::string::npos)
++        ) {
++            if (ftype == LLAMA_FTYPE_MOSTLY_Q3_K_M || ftype == LLAMA_FTYPE_MOSTLY_Q3_K_L) {
++                new_type = GGML_TYPE_Q4_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q4_K_M) {
++                new_type = GGML_TYPE_Q5_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q5_K_M) {
++                new_type = GGML_TYPE_Q6_K;
++            }
++    } else if ( // Rules for ffn
++            (name.find("ffn_down") != std::string::npos) ||
++            ((name.find("experts.") != std::string::npos) && (name.find(".w2.weight") != std::string::npos)) ||
++            (name.find(".ffn.2.weight") != std::string::npos) || // is this even the right way around?
++            (name.find(".ff.net.2.weight") != std::string::npos) ||
++            (name.find(".mlp.layer2.weight") != std::string::npos) ||
++            (name.find(".adaln_modulation_mlp.2.weight") != std::string::npos) ||
++            (name.find(".feed_forward.w2.weight") != std::string::npos)
++        ) {
++            // TODO: add back `layer_info` with some model specific logic + logic further down
++            if (ftype == LLAMA_FTYPE_MOSTLY_Q3_K_M) {
++                new_type = GGML_TYPE_Q4_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q3_K_L) {
++                new_type = GGML_TYPE_Q5_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q4_K_S) {
++                new_type = GGML_TYPE_Q5_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q4_K_M) {
++                new_type = GGML_TYPE_Q6_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q5_K_M) {
++                new_type = GGML_TYPE_Q6_K;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q4_0) {
++                new_type = GGML_TYPE_Q4_1;
++            }
++            else if (ftype == LLAMA_FTYPE_MOSTLY_Q5_0) {
++                new_type = GGML_TYPE_Q5_1;
++            }
++            ++qs.i_ffn_down;
++    }
++
++    // Sanity check for row shape
++    bool convert_incompatible_tensor = false;
++    if (new_type == GGML_TYPE_Q2_K    || new_type == GGML_TYPE_Q3_K    || new_type == GGML_TYPE_Q4_K   ||
++        new_type == GGML_TYPE_Q5_K    || new_type == GGML_TYPE_Q6_K) {
++        int nx = tensor->ne[0];
++        int ny = tensor->ne[1];
++        if (nx % QK_K != 0) {
++            LLAMA_LOG_WARN("\n\n%s : tensor cols %d x %d are not divisible by %d, required for %s", __func__, nx, ny, QK_K, ggml_type_name(new_type));
++            convert_incompatible_tensor = true;
++        } else {
++            ++qs.n_k_quantized;
++        }
++    }
++    if (convert_incompatible_tensor) {
++        // TODO: Possibly reenable this in the future
++        // switch (new_type) {
++        //     case GGML_TYPE_Q2_K:
++        //     case GGML_TYPE_Q3_K:
++        //     case GGML_TYPE_Q4_K:   new_type = GGML_TYPE_Q5_0;   break;
++        //     case GGML_TYPE_Q5_K:   new_type = GGML_TYPE_Q5_1;   break;
++        //     case GGML_TYPE_Q6_K:   new_type = GGML_TYPE_Q8_0;   break;
++        //     default: throw std::runtime_error("\nUnsupported tensor size encountered\n");
++        // }
++        new_type = GGML_TYPE_F16;
++        LLAMA_LOG_WARN(" - using fallback quantization %s\n", ggml_type_name(new_type));
++        ++qs.n_fallback;
++    }
++    return new_type;
++}
++
+ static ggml_type llama_tensor_get_type(quantize_state_internal & qs, ggml_type new_type, const ggml_tensor * tensor, llama_ftype ftype) {
+     const std::string name = ggml_get_name(tensor);
+ 
+@@ -18513,7 +18693,9 @@ static void llama_model_quantize_internal(const std::string & fname_inp, const s
+         if (llama_model_has_encoder(&model)) {
+             n_attn_layer *= 3;
+         }
+-        GGML_ASSERT((qs.n_attention_wv == n_attn_layer) && "n_attention_wv is unexpected");
++        if (model.arch != LLM_ARCH_HYVID) { // TODO: Check why this fails
++            GGML_ASSERT((qs.n_attention_wv == n_attn_layer) && "n_attention_wv is unexpected");
++        }
+     }
+ 
+     size_t total_size_org = 0;
+@@ -18547,6 +18729,51 @@ static void llama_model_quantize_internal(const std::string & fname_inp, const s
+             ctx_outs[i_split] = gguf_init_empty();
+         }
+         gguf_add_tensor(ctx_outs[i_split], tensor);
++        // SD3 pos_embed needs special fix as first dim is 1, which gets truncated here
++        if (model.arch == LLM_ARCH_SD3) {
++            const std::string name = ggml_get_name(tensor);
++            if (name == "pos_embed" && tensor->ne[2] == 1) {
++                const int n_dim = 3;
++                gguf_set_tensor_ndim(ctx_outs[i_split], "pos_embed", n_dim);
++                LLAMA_LOG_INFO("\n%s: Correcting pos_embed shape for SD3: [key:%s]\n", __func__, tensor->name);
++            }
++        }
++        // same goes for auraflow
++        if (model.arch == LLM_ARCH_AURA) {
++            const std::string name = ggml_get_name(tensor);
++            if (name == "positional_encoding" && tensor->ne[2] == 1) {
++                const int n_dim = 3;
++                gguf_set_tensor_ndim(ctx_outs[i_split], "positional_encoding", n_dim);
++                LLAMA_LOG_INFO("\n%s: Correcting positional_encoding shape for AuraFlow: [key:%s]\n", __func__, tensor->name);
++            }
++            if (name == "register_tokens" && tensor->ne[2] == 1) {
++                const int n_dim = 3;
++                gguf_set_tensor_ndim(ctx_outs[i_split], "register_tokens", n_dim);
++                LLAMA_LOG_INFO("\n%s: Correcting register_tokens shape for AuraFlow: [key:%s]\n", __func__, tensor->name);
++            }
++        }
++        // conv3d fails due to max dims - unsure what to do here as we never even reach this check
++        if (model.arch == LLM_ARCH_HYVID) {
++            const std::string name = ggml_get_name(tensor);
++            if (name == "img_in.proj.weight" && tensor->ne[5] != 1 ) {
++                throw std::runtime_error("img_in.proj.weight size failed for HyVid");
++            }
++        }
++        // All the modulation layers also have dim1, and I think conv3d fails here too but we segfaul way before that...
++        if (model.arch == LLM_ARCH_WAN) {
++            const std::string name = ggml_get_name(tensor);
++            if (name.find(".modulation") != std::string::npos && tensor->ne[2] == 1) {
++                const int n_dim = 3;
++                gguf_set_tensor_ndim(ctx_outs[i_split], tensor->name, n_dim);
++                LLAMA_LOG_INFO("\n%s: Correcting shape for Wan: [key:%s]\n", __func__, tensor->name);
++            }
++            // FLF2V model only
++            if (name == "img_emb.emb_pos") {
++                const int n_dim = 3;
++                gguf_set_tensor_ndim(ctx_outs[i_split], tensor->name, n_dim);
++                LLAMA_LOG_INFO("\n%s: Correcting shape for Wan FLF2V: [key:%s]\n", __func__, tensor->name);
++            }
++        }
+     }
+ 
+     // Set split info if needed
+@@ -18647,6 +18874,110 @@ static void llama_model_quantize_internal(const std::string & fname_inp, const s
+         // do not quantize relative position bias (T5)
+         quantize &= name.find("attn_rel_b.weight") == std::string::npos;
+ 
++        // rules for image models
++        bool image_model = false;
++        if (model.arch == LLM_ARCH_FLUX) {
++            image_model = true;
++            quantize &= name.find("txt_in.") == std::string::npos;
++            quantize &= name.find("img_in.") == std::string::npos;
++            quantize &= name.find("time_in.") == std::string::npos;
++            quantize &= name.find("vector_in.") == std::string::npos;
++            quantize &= name.find("guidance_in.") == std::string::npos;
++            quantize &= name.find("final_layer.") == std::string::npos;
++        }
++        if (model.arch == LLM_ARCH_SD1 || model.arch == LLM_ARCH_SDXL) {
++            image_model = true;
++            quantize &= name.find("class_embedding.") == std::string::npos;
++            quantize &= name.find("time_embedding.") == std::string::npos;
++            quantize &= name.find("add_embedding.") == std::string::npos;
++            quantize &= name.find("time_embed.") == std::string::npos;
++            quantize &= name.find("label_emb.") == std::string::npos;
++            quantize &= name.find("conv_in.") == std::string::npos;
++            quantize &= name.find("conv_out.") == std::string::npos;
++            quantize &= name != "input_blocks.0.0.weight";
++            quantize &= name != "out.2.weight";
++        }
++        if (model.arch == LLM_ARCH_SD3) {
++            image_model = true;
++            quantize &= name.find("final_layer.") == std::string::npos;
++            quantize &= name.find("time_text_embed.") == std::string::npos;
++            quantize &= name.find("context_embedder.") == std::string::npos;
++            quantize &= name.find("t_embedder.") == std::string::npos;
++            quantize &= name.find("y_embedder.") == std::string::npos;
++            quantize &= name.find("x_embedder.") == std::string::npos;
++            quantize &= name != "proj_out.weight";
++            quantize &= name != "pos_embed";
++        }
++        if (model.arch == LLM_ARCH_AURA) {
++            image_model = true;
++            quantize &= name.find("t_embedder.") == std::string::npos;
++            quantize &= name.find("init_x_linear.") == std::string::npos;
++            quantize &= name != "modF.1.weight";
++            quantize &= name != "cond_seq_linear.weight";
++            quantize &= name != "final_linear.weight";
++            quantize &= name != "final_linear.weight";
++            quantize &= name != "positional_encoding";
++            quantize &= name != "register_tokens";
++        }
++        if (model.arch == LLM_ARCH_LTXV) {
++            image_model = true;
++            quantize &= name.find("adaln_single.") == std::string::npos;
++            quantize &= name.find("caption_projection.") == std::string::npos;
++            quantize &= name.find("patchify_proj.") == std::string::npos;
++            quantize &= name.find("proj_out.") == std::string::npos;
++            quantize &= name.find("scale_shift_table") == std::string::npos; // last block too
++        }
++        if (model.arch == LLM_ARCH_HYVID) {
++            image_model = true;
++            quantize &= name.find("txt_in.") == std::string::npos;
++            quantize &= name.find("img_in.") == std::string::npos;
++            quantize &= name.find("time_in.") == std::string::npos;
++            quantize &= name.find("vector_in.") == std::string::npos;
++            quantize &= name.find("guidance_in.") == std::string::npos;
++            quantize &= name.find("final_layer.") == std::string::npos;
++        }
++        if (model.arch == LLM_ARCH_WAN) {
++            image_model = true;
++            quantize &= name.find("modulation.") == std::string::npos;
++            quantize &= name.find("patch_embedding.") == std::string::npos;
++            quantize &= name.find("text_embedding.") == std::string::npos;
++            quantize &= name.find("time_projection.") == std::string::npos;
++            quantize &= name.find("time_embedding.") == std::string::npos;
++            quantize &= name.find("img_emb.") == std::string::npos;
++            quantize &= name.find("head.") == std::string::npos;
++        }
++        if (model.arch == LLM_ARCH_HIDREAM) {
++            image_model = true;
++            quantize &= name.find("p_embedder.") == std::string::npos;
++            quantize &= name.find("t_embedder.") == std::string::npos;
++            quantize &= name.find("x_embedder.") == std::string::npos;
++            quantize &= name.find("final_layer.") == std::string::npos;
++            quantize &= name.find(".ff_i.gate.weight") == std::string::npos;
++            quantize &= name.find("caption_projection.") == std::string::npos;
++        }
++        if (model.arch == LLM_ARCH_COSMOS) {
++            image_model = true;
++            quantize &= name.find("p_embedder.") == std::string::npos;
++            quantize &= name.find("t_embedder.") == std::string::npos;
++            quantize &= name.find("t_embedding_norm.") == std::string::npos;
++            quantize &= name.find("x_embedder.") == std::string::npos;
++            quantize &= name.find("pos_embedder.") == std::string::npos;
++            quantize &= name.find("final_layer.") == std::string::npos;
++        }
++        if (model.arch == LLM_ARCH_LUMINA2) {
++            image_model = true;
++            quantize &= name.find("t_embedder.") == std::string::npos;
++            quantize &= name.find("x_embedder.") == std::string::npos;
++            quantize &= name.find("final_layer.") == std::string::npos;
++            quantize &= name.find("cap_embedder.") == std::string::npos;
++            quantize &= name.find("context_refiner.") == std::string::npos;
++            quantize &= name.find("noise_refiner.") == std::string::npos;
++        }
++        // ignore 3D/4D tensors for image models as the code was never meant to handle these
++        if (image_model) {
++            quantize &= ggml_n_dims(tensor) == 2;
++        }
++
+         enum ggml_type new_type;
+         void * new_data;
+         size_t new_size;
+@@ -18655,6 +18986,9 @@ static void llama_model_quantize_internal(const std::string & fname_inp, const s
+             new_type = default_type;
+ 
+             // get more optimal quantization type based on the tensor shape, layer, etc.
++            if (image_model) {
++                new_type = img_tensor_get_type(qs, new_type, tensor, ftype);
++            } else {
+             if (!params->pure && ggml_is_quantized(default_type)) {
+                 new_type = llama_tensor_get_type(qs, new_type, tensor, ftype);
+             }
+@@ -18664,6 +18998,7 @@ static void llama_model_quantize_internal(const std::string & fname_inp, const s
+             if (params->output_tensor_type < GGML_TYPE_COUNT && strcmp(tensor->name, "output.weight") == 0) {
+                 new_type = params->output_tensor_type;
+             }
++            }
+ 
+             // If we've decided to quantize to the same type the tensor is already
+             // in then there's nothing to do.

custom_nodes/ComfyUI-GGUF/tools/read_tensors.py

@@ -0,0 +1,21 @@
+#!/usr/bin/python3
+import os
+import sys
+import gguf
+
+def read_tensors(path):
+    reader = gguf.GGUFReader(path)
+    for tensor in reader.tensors:
+        if tensor.tensor_type == gguf.GGMLQuantizationType.F32:
+            continue
+        print(f"{str(tensor.tensor_type):32}: {tensor.name}")
+
+try:
+    path = sys.argv[1]
+    assert os.path.isfile(path), "Invalid path"
+    print(f"input: {path}")
+except Exception as e:
+    input(f"failed: {e}")
+else:
+    read_tensors(path)
+    input()

custom_nodes/ComfyUI-Manager

@@ -0,0 +1 @@
+Subproject commit dcb37d9c55f9422030bcd553c281bc6970357b1a

custom_nodes/ComfyUI_Comfyroll_CustomNodes

@@ -0,0 +1 @@
+Subproject commit d78b780ae43fcf8c6b7c6505e6ffb4584281ceca

custom_nodes/RES4LYF

@@ -0,0 +1 @@
+Subproject commit 7750bf7800b6ad9d670308a09989fc0c04c40cec

custom_nodes/SendToButton/__init__.py

@@ -0,0 +1,5 @@
+from .send_to_loadimage import TaggedLoadImage
+
+NODE_CLASS_MAPPINGS = {
+    "TaggedLoadImage": TaggedLoadImage
+}

custom_nodes/SendToButton/send_to_loadimage.py

@@ -0,0 +1,56 @@
+from PIL import Image
+import numpy as np
+import os
+
+import folder_paths  # ComfyUI helper for input/output dirs
+
+class TaggedLoadImage:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                # STRING + image_upload avoids "value not in list" and supports subfolders (e.g., pasted/...)
+                "image_file": ("STRING", {
+                    "default": "",
+                    "image_upload": True,
+                    "placeholder": "Select or upload an image (relative to input/)",
+                }),
+                "tag": ("STRING", {"default": "default"}),
+            },
+            "optional": {
+                # still allow an upstream IMAGE to override file selection if connected
+                "image": ("IMAGE",),
+            },
+        }
+
+    RETURN_TYPES = ("IMAGE", "MASK")
+    FUNCTION = "load_image"
+    CATEGORY = "Load"
+
+    def load_image(self, image_file, tag="default", image=None):
+        # If an upstream IMAGE is provided, prefer it
+        if image is not None:
+            if isinstance(image, np.ndarray):
+                mask = np.ones((image.shape[0], image.shape[1]), dtype=np.float32)
+                return (image, mask)
+            raise TypeError("TaggedLoadImage: unsupported IMAGE type; expected numpy array")
+
+        # Otherwise, load from the provided path (supports subfolders like pasted/..)
+        if not image_file or image_file.strip() == "":
+            raise ValueError("TaggedLoadImage: no image provided. Upload/select a file or connect an IMAGE input.")
+
+        # Resolve to full path: allow absolute paths; otherwise treat as relative to input dir
+        if os.path.isabs(image_file):
+            full_path = image_file
+        else:
+            input_dir = folder_paths.get_input_directory()
+            full_path = os.path.join(input_dir, image_file)
+
+        if not os.path.exists(full_path) or not os.path.isfile(full_path):
+            raise ValueError(f"TaggedLoadImage: file not found: {full_path}")
+
+        with Image.open(full_path) as im:
+            pil_image = im.convert("RGB")
+        arr = np.array(pil_image).astype(np.float32) / 255.0
+        mask = np.ones((arr.shape[0], arr.shape[1]), dtype=np.float32)
+        return (arr, mask)

custom_nodes/cg-image-filter/.gitattributes

@@ -0,0 +1,2 @@
+# Auto detect text files and perform LF normalization
+* text=auto

custom_nodes/cg-image-filter/.github/workflows/publish_action.yml

@@ -0,0 +1,24 @@
+name: Publish to Comfy registry
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - "pyproject.toml"
+
+permissions:
+  issues: write
+
+jobs:
+  publish-node:
+    name: Publish Custom Node to registry
+    runs-on: ubuntu-latest
+    if: ${{ github.repository_owner == 'chrisgoringe' }}
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+      - name: Publish Custom Node
+        uses: Comfy-Org/publish-node-action@v1
+        with:
+          personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }} ## Add your own personal access token to your Github Repository secrets and reference it here.

custom_nodes/cg-image-filter/.gitignore

@@ -0,0 +1,160 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

custom_nodes/cg-image-filter/.tracking

@@ -0,0 +1,39 @@
+.gitattributes
+.github/workflows/publish_action.yml
+.gitignore
+LICENSE
+README.md
+__init__.py
+image_filter.py
+image_filter_messaging.py
+images/basic.png
+images/blob.png
+images/editorchoice.png
+images/extras.png
+images/fromlist.png
+images/mask workflow.png
+images/mask.png
+images/maskedsection.png
+images/maskin.png
+images/options.png
+images/popup.png
+images/text workflow.png
+images/text.png
+images/three filters.png
+images/tiny.png
+images/versions.png
+images/workflow.png
+js/ding.mp3
+js/filter.css
+js/floating_window.css
+js/floating_window.js
+js/image_filter.js
+js/log.js
+js/mask_utils.js
+js/popup.js
+js/utils.js
+js/zoomed.css
+list_utility_nodes.py
+mask_utility_nodes.py
+pyproject.toml
+string_utility_nodes.py

custom_nodes/cg-image-filter/LICENSE

@@ -0,0 +1,201 @@
+Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

custom_nodes/cg-image-filter/README.md

@@ -0,0 +1,266 @@
+# CG Image Filter
+
+---
+
+**If you are reporting a bug, please read [how to report a bug well](#how-to-report-a-bug-well) first!**
+
+---
+
+A set of nodes designed to pause execution of the workflow to allow you to make selections and/or edits before continuing.
+There's an example workflow that illustrates all of them at the end.
+
+- ['Image Filter'](#image-filter) - pause the flow and pick which images from a set you want to proceed with
+- ['Mask Image Filter'](#mask-image-filter) - launch the mask editor for the image, and return the image and mask
+- ['Text Image Filter with Extras'](#text-image-filter-with-extras) - as 'Text Image Filter' but with three extra single line texts fields that are also returned
+
+There are also some helper nodes.
+
+If you prefer trying a workflow to reading docs, use of the nodes is illustrated in this blob (drag the image into Comfy):
+
+<img src="https://github.com/chrisgoringe/cg-image-filter/raw/main/images/blob.png" alt="Seahorse" width="200" height="200">
+
+## New in 1.6 ##
+
+- `Masked Section` node to crop images to just the masked area
+
+## New in 1.5 ##
+
+- Floating window for extras and tips
+- Mask editor node now has option for text extras
+- Keyboard and mouse navigation in zoomed view
+
+## New in 1.4 ##
+
+Video previews!
+
+## New in 1.3.2 ##
+
+- works with old mask editor as well
+- keyboard shortcuts are back
+- various minor fixes
+
+## New in 1.3 ##
+
+- pick_list to automatically select images in `Image Filter`
+- optional initial mask input to `Mask Image Filter`
+
+
+## Examples of what you might do with them
+
+- Generate an image or batch, and select which ones you want before spending the time upscaling
+- Generate an image and pick part of it to inpaint all in one go (the example workflow below does this)
+- Edit auto-generated captions before saving them
+- Iterate through a folder of images, picking a masked area to inpaint (and the inpainting prompt) for each
+- you ideas here...
+
+---
+
+## Global Options
+
+![image](images/options.png)
+
+- `If all images are identical, autosend one` - in the ImageFilter node, if all images are identical (including if there is just one image) then send
+an image without user interaction. 
+- `Clicking an image sends it` - In the ImageFilter node, clicking an image sends it instead of selecting it. Useful if you know you only ever want to send one image at most.
+- `Show a small popup instead of covering the screen` - instead of taking over the whole screen immediately, display a tiny version of the image in the top left. Click that image to go into the full screen mode. You can move the tiny image window around to where you want it by dragging the title bar.
+- `Enter the Image Filter node with an image zoomed` - instead of showing the grid of images, zoom in on one. Options are `first` or `last`
+- `Play sound when activating` - play a 'ding' sound when any of the filter nodes becomes active. You can change the sound by replacing the file `ding.mp3` in the `js` subfolder.
+- `Video Frames per Second` - when previewing vidoe(s), try to play them at this speed
+
+---
+
+## Keyboard shortcuts
+
+In `Image Filter` and `Text Image Filter` (not `Mask Image Filter`)
+
+- `Space` when hovering over an image enlarges it. See `Zoom` in the `Image Filter` section below.
+- `Escape` to click the cancel button
+- `Enter`to click the send button
+- digits `0`, `1`, etc. to click an image (zero-indexed) (select/deselect or send, depending on the value of `ClickSends`)
+
+In `Image Filter` only
+
+- `ctrl-A` to select/unselect all images
+
+---
+
+## Image Filter
+
+The image filter node pauses execution of the workflow while you choose which, if any, of the images produced, you want to progress.
+
+Insert it like this:
+
+![image](images/basic.png)
+
+When you run the workflow, and get to this point, a popup window will appear displaying the image(s) for you to select which, if any, 
+you want to progress:
+
+![image](images/popup.png)
+
+Click the images that you want to keep (their border will turn green) and then click 'Send' to continue the workflow. 
+If you don't want to keep any of the images, click 'Cancel' (or press 'escape') to terminate the workflow. 
+
+The node also has a timeout specified, and a countdown is shown on the left hand side. If you don't Send or Cancel before the timeout, 
+the node will either cancel or send all the images, depending on the option you have selected.
+
+Here's a simple use: generate a batch of images and pick which ones you want to save:
+
+![workflow](images/workflow.png)
+
+### Zoom
+
+If you hover over an image (it gets a red border) and press the space bar, it will zoom to fill the screen. 
+You can also use the `Enter the Image Filter node with an image zoomed` setting to start in this mode.
+
+While zoomed, you can navigate with keyboard or mouse:
+
+| |Mouse|Keyboard|Notes|
+|-|-|-|-|
+|Select/unselect/send|Click the zoomed image|`Arrow Up`|Will select/unselect the image, or send it if you have `Clicking an image sends it` turned on|
+|Previous/Next|Click the arrows to the left or right|`Arrow Left` `Arrow Right`|The image number is shown near the top right|
+|Unzoom| |`Space`|Back to the grid view|
+
+### Optional inputs
+
+The Latent and Mask inputs are optional. If used, they should have the same number of latents (or masks) as the image batch, 
+and the latents (or masks) corresponding to the selected images will be output. Use this if (for instance) you want to 
+select from a batch of images, but the next stage uses the latent - that way you avoid the decode-recode loss, or if you want
+to pick a mask (perhaps from options automatically generated)
+
+Other things (like prompts, segs etc.) that have been used can be selected using the `Pick from List` helper node.
+
+### pick_list
+
+Sometimes you know which images you are going to want. If you provide a comma separated list of integers in `pick_list`,
+these images will be selected without user input. Values will be taken modulo the number of images (which means you can use `-1` for the last image).
+
+You could also use this to make a larger batch of images (repeat a value and the image will appear repeated times in the output).
+
+Note that this uses zero indexing (the first image is '0').
+
+### video_frames
+
+If you set this to greater than one, the image filter node will batch up each set of n images and treat them as a video snip. 
+
+Set the FPS for playback in the global settings.
+
+This is a new, experimental feature, so please report any issues...
+
+---
+
+## Mask Image Filter
+
+Designed for a single image, when executed the Mask Image Filter node will automatically launch the mask editor. 
+
+![mask](images/mask.png)
+
+When you finish mask editing the image and mask will be output. Here's a simple use - generate an image, and then 
+mask the bit you don't like, before doing an img2img step.
+
+![mask workflow](images/mask%20workflow.png)
+
+Again, there is a timeout, and if you don't save a mask before the end of the timeout (or if you press the cancel button in the mask editor), 
+it will either cancel, or send a blank mask, depending on the option chosen.
+
+### Mask in
+
+There is an optional mask input (added in 1.3), which allows you to specify the mask when the editor is launched:
+
+![mask in](images/maskin.png)
+
+**Note that the Mask Image Filter works with the new Mask Editor; it does not work with the old one**
+
+![img](images/editorchoice.png)
+
+---
+
+## Text Image Filter
+
+Also designed for a single image, this node will show the image and a string of text; you can edit the text and then press send. 
+
+![text](images/text.png)
+
+The image and (edited) text are output. The intended use is for captioning workflows; you can read and edit each caption as it is
+generated. Here's a trivial workflow:
+
+![text workflow](images/text%20workflow.png)
+
+---
+
+# A few more things
+
+## Extras
+
+'Text Image Filter' and 'Image Filter', each provide three extra text fields, intended for short form - like specifying the denoising you want on the next step, or a prefix to save the file with.
+
+If you use the optional 'tip' input, the contents will be displayed under the extras input fields, so you can remind yourself what they are for!
+
+---
+
+## Helper Nodes
+
+### Masked Section
+
+`Masked Section` takes a mask and a (batch of) images and outputs the images cropped to the bounding box of the mask (with a minimum size). 
+Here's how you might use it to preview the parts of the image that were changed in img2img (also using `Pick from List` and `Image List from Batch` described below)
+
+![masked preview](images/maskedsection.png)
+
+### String handling
+
+- `Split String by Commas` allows you to split a text string into up to five pieces, splitting on `,`, `|`, or `^`. It also strips whitespace, so that the strings can be easily parsed, especially by...
+- `String to Int` and `String to Float` convert a string to an int or a float, with a fallback default 
+
+Together, these nodes allow you to specify lots of information in the `extras` fields. For instance, if doing an inpaint, you might have an extras field that takes the format `0.4, 20` meaning 'denoise 0.4, 20 steps'. Split the string, feed the pieces into the converters, and feed that into other nodes. Like this:
+
+![extras](images/extras.png)
+
+`Split String by Commas` produces five string outputs. If there are fewer than five terms, the extra ones will havee an empty string; if there are more than five terms the fifth output will be a comma separated list of the fifth and subsequent terms.
+
+There is also a sixth output which is a list of all the strings.
+
+### List and Batch Handling
+
+**When working with multiple images** - `Image Filter` expects a batch, `Mask Image Filter` and `Text Image Filter` need a list.
+
+So if you generate a load of images using lists of, for instance, prompts, to feed them into `Image Filter` you will want to combine them with the helper node `Batch from Image List`.
+
+If you generate a batch of images and want to use the `Mask Image Filter` or `Text Image Filter`, which only handle a single image, you will want to do the opposite - convert the batch to a list with `Image List From Batch`. Then the images will be shown sequentially.
+
+The `Image Filter` node outputs a string, `indexes`, which is a comma separated list of the indexes 
+(0 based by default, but you can pick a different start value if you need to with `pick_list_start`) of the images selected. 
+Connect this to a `Pick from List` node, and connect a list of anything to the `anything` input, and the `Pick from List` node will output a list corresponding to the selected images.
+
+For instance, if you create a set of images using a list of prompts, this can be used to select the prompts that correspond to the selected images. But you might well want to batch the images if you did use lists - see below.
+
+So something like this:
+
+![image](images/fromlist.png)
+
+
+
+---
+
+# Example Workflow
+
+![image](images/three%20filters.png)
+
+This workflow:
+- generates an image
+- uses 'Mask Image Filter' to allow you to mask part of the image for inpainting
+- uses 'Text Image Filter with Extras' to enter a prompt (and negative prompt) for the inpainting
+- inpaints
+- uses 'Image Filter' to choose which, if either, of the two images (before and after inpaint) to save
+
+The workflow is embedded in the blob in a bottle:
+
+<img src="https://github.com/chrisgoringe/cg-image-filter/raw/main/images/blob.png" alt="Seahorse" width="200" height="200">
+
+# Bugs, Ideas, and the future
+
+Take a look at the [issues list](https://github.com/chrisgoringe/cg-image-filter/issues) to see what I'm thinking of,
+to report problems, or to make suggestions.
+
+## How to report a bug well
+
+Please read [this guide](https://github.com/chrisgoringe/cg-image-filter/discussions/60) before you post a bug!

custom_nodes/cg-image-filter/__init__.py

@@ -0,0 +1,32 @@
+"""
+@author: chrisgoringe
+@title: Image Filter
+@nickname: Image Filter
+@description: A custom node that pauses the flow while you choose which image or images to pass on to the rest of the workflow. Simplified and improved version of cg-image-picker.
+"""
+
+from .image_filter import ImageFilter, MaskImageFilter, TextImageFilterWithExtras
+from .list_utility_nodes import PickFromList, BatchFromImageList, ImageListFromBatch, StringListFromStrings
+from .string_utility_nodes import SplitByCommas, StringToFloat, StringToInt, AnyListToString
+from .mask_utility_nodes import MaskedSection
+
+VERSION = "1.6.1"
+WEB_DIRECTORY = "./js"
+
+NODE_CLASS_MAPPINGS= {
+    "Image Filter": ImageFilter,
+    "Text Image Filter": TextImageFilterWithExtras,
+    "Text Image Filter with Extras": TextImageFilterWithExtras,
+    "Mask Image Filter": MaskImageFilter,
+    "Split String by Commas": SplitByCommas,
+    "String to Int": StringToInt,
+    "String to Float": StringToFloat,
+    "Pick from List": PickFromList,
+    "Any List to String": AnyListToString,
+    "String List from Strings": StringListFromStrings,
+    "Batch from Image List": BatchFromImageList,
+    "Image List From Batch": ImageListFromBatch,
+    "Masked Section": MaskedSection,
+}
+
+__all__ = ["NODE_CLASS_MAPPINGS", "WEB_DIRECTORY"]

custom_nodes/cg-image-filter/image_filter.py

@@ -0,0 +1,188 @@
+
+
+from nodes import PreviewImage, LoadImage
+from comfy.model_management import InterruptProcessingException
+import os
+import torch
+
+from .image_filter_messaging import send_and_wait, Response, TimeoutResponse
+
+HIDDEN = {
+            "prompt": "PROMPT", 
+            "extra_pnginfo": "EXTRA_PNGINFO", 
+            "uid":"UNIQUE_ID",
+            "node_identifier": "NID",
+        }
+
+class ImageFilter(PreviewImage):
+    RETURN_TYPES = ("IMAGE","LATENT","MASK","STRING","STRING","STRING","STRING")
+    RETURN_NAMES = ("images","latents","masks","extra1","extra2","extra3","indexes")
+    FUNCTION = "func"
+    CATEGORY = "image_filter"
+    OUTPUT_NODE = False
+    DESCRIPTION = "Allows you to preview images and choose which, if any to proceed with"
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "images" : ("IMAGE", ), 
+                "timeout": ("INT", {"default": 600, "min":1, "max":9999999, "tooltip": "Timeout in seconds."}),
+                "ontimeout": (["send none", "send all", "send first", "send last"], {}),
+            },
+            "optional": {
+                "latents" : ("LATENT", {"tooltip": "Optional - if provided, will be output"}),
+                "masks" : ("MASK", {"tooltip": "Optional - if provided, will be output"}),
+                "tip" : ("STRING", {"default":"", "tooltip": "Optional - if provided, will be displayed in popup window"}),
+                "extra1" : ("STRING", {"default":""}),
+                "extra2" : ("STRING", {"default":""}),
+                "extra3" : ("STRING", {"default":""}),
+                "pick_list_start" : ("INT", {"default":0, "tooltip":"The number used in pick_list for the first image"}),
+                "pick_list" : ("STRING", {"default":"", "tooltip":"If a comma separated list of integers is provided, the images with these indices will be selected automatically."}),
+                "video_frames" : ("INT", {"default":1, "min":1, "tooltip": "treat each block of n images as a video"}),
+            },
+            "hidden": HIDDEN,
+        }
+    
+    @classmethod
+    def IS_CHANGED(cls, pick_list, **kwargs):
+        return pick_list or float("NaN")
+    
+    def func(self, images, timeout, ontimeout, uid, node_identifier, tip="", extra1="", extra2="", extra3="", latents=None, masks=None, pick_list_start:int=0, pick_list:str="", video_frames:int=1, **kwargs):
+        e1, e2, e3 = extra1, extra2, extra3
+        B = images.shape[0]
+
+        if video_frames>B: video_frames=1
+            
+
+        try:    images_to_return = [ int(x.strip())%B for x in pick_list.split(',') ] if pick_list else []
+        except Exception as e: 
+            print(f"{e} parsing pick_list - will manually select")
+            images_to_return = []
+
+        if len(images_to_return) == 0:
+            all_the_same = ( B and all( (images[i]==images[0]).all() for i in range(1,B) )) 
+            urls:list[str] = self.save_images(images=images, **kwargs)['ui']['images']
+            payload = {"uid": uid, "urls":urls, "allsame":all_the_same, "extras":[extra1, extra2, extra3], "tip":tip, "video_frames":video_frames}
+
+            response:Response = send_and_wait(payload, timeout, uid, node_identifier)
+
+            if isinstance(response, TimeoutResponse):
+                if ontimeout=='send none':  images_to_return = []
+                if ontimeout=='send all':   images_to_return = [*range(len(images)//video_frames)]
+                if ontimeout=='send first': images_to_return = [0,]
+                if ontimeout=='send last':  images_to_return = [(len(images)//video_frames)-1,]
+            else:
+                e1, e2, e3 = response.get_extras([extra1, extra2, extra3])
+                images_to_return = response.selection
+
+        if images_to_return is None or len(images_to_return) == 0: raise InterruptProcessingException()
+
+        if video_frames>1:
+            images_to_return = [ key*video_frames + frm  for key in images_to_return for frm in range(video_frames)   ]
+
+        images = torch.stack(list(images[int(i)] for i in images_to_return))
+        latents = {"samples": torch.stack(list(latents['samples'][int(i)] for i in images_to_return))} if latents is not None else None
+        masks = torch.stack(list(masks[int(i)] for i in images_to_return)) if masks is not None else None
+
+        try: int(pick_list_start)
+        except: pick_list_start = '0'
+                
+        return (images, latents, masks, e1, e2, e3, ",".join(str(int(x)+int(pick_list_start)) for x in images_to_return))
+    
+class TextImageFilterWithExtras(PreviewImage):
+    RETURN_TYPES = ("IMAGE","STRING","STRING","STRING","STRING")
+    RETURN_NAMES = ("image","text","extra1","extra2","extra3")
+    FUNCTION = "func"
+    CATEGORY = "image_filter"
+    OUTPUT_NODE = False
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "image" : ("IMAGE", ), 
+                "text" : ("STRING", {"default":""}),
+                "timeout": ("INT", {"default": 600, "min":1, "max":9999999, "tooltip": "Timeout in seconds."}),
+            },
+            "optional": {
+                "mask" : ("MASK", {"tooltip": "Optional - if provided, will be overlaid on image"}),
+                "tip" : ("STRING", {"default":"", "tooltip": "Optional - if provided, will be displayed in popup window"}),
+                "extra1" : ("STRING", {"default":""}),
+                "extra2" : ("STRING", {"default":""}),
+                "extra3" : ("STRING", {"default":""}),
+                "textareaheight" : ("INT", {"default": 150, "min": 50, "max": 500, "tooltip": "Height of text area in pixels"}),
+            },
+            "hidden": HIDDEN,
+        }
+    
+    @classmethod
+    def IS_CHANGED(cls, **kwargs):
+        return float("NaN")
+    
+    def func(self, image, text, timeout, uid, node_identifier, extra1="", extra2="", extra3="", mask=None, tip="", textareaheight=None, **kwargs):
+        urls:list[str] = self.save_images(images=image, **kwargs)['ui']['images']
+        payload = {"uid": uid, "urls":urls, "text":text, "extras":[extra1, extra2, extra3], "tip":tip}
+        if textareaheight is not None: payload['textareaheight'] = textareaheight
+        if mask is not None: payload['mask_urls'] = self.save_images(images=mask_to_image(mask), **kwargs)['ui']['images']
+
+        response = send_and_wait(payload, timeout, uid, node_identifier)
+        if isinstance(response, TimeoutResponse):
+            return (image, text, extra1, extra2, extra3)
+
+        return (image, response.text, *response.get_extras([extra1, extra2, extra3])) 
+
+def mask_to_image(mask:torch.Tensor):
+    return torch.stack([mask, mask, mask, 1.0-mask], -1)
+    
+class MaskImageFilter(PreviewImage, LoadImage):
+    RETURN_TYPES = ("IMAGE","MASK","STRING","STRING","STRING")
+    RETURN_NAMES = ("image","mask","extra1","extra2","extra3")
+    FUNCTION = "func"
+    CATEGORY = "image_filter"
+    OUTPUT_NODE = False
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "image" : ("IMAGE", ), 
+                "timeout": ("INT", {"default": 600, "min":1, "max":9999999, "tooltip": "Timeout in seconds."}),
+                "if_no_mask": (["cancel", "send blank"], {}),
+            },
+            "optional": {
+                "mask" : ("MASK", {"tooltip":"optional initial mask"}),
+                "tip" : ("STRING", {"default":"", "tooltip": "Optional - if provided, will be displayed in popup window"}),
+                "extra1" : ("STRING", {"default":""}),
+                "extra2" : ("STRING", {"default":""}),
+                "extra3" : ("STRING", {"default":""}),
+            },
+            "hidden": HIDDEN,
+        }
+    
+    @classmethod
+    def IS_CHANGED(cls, **kwargs):
+        return float("NaN")
+    
+    @classmethod
+    def VALIDATE_INPUTS(cls, **kwargs): return True
+    
+    def func(self, image, timeout, uid, if_no_mask, node_identifier, mask=None, extra1="", extra2="", extra3="", tip="", **kwargs):
+        if mask is not None and mask.shape[:3] == image.shape[:3] and not torch.all(mask==0):
+            saveable = torch.cat((image, mask.unsqueeze(-1)), dim=-1)
+        else:
+            saveable = image
+
+        urls:list[str] = self.save_images(images=saveable, **kwargs)['ui']['images']
+        payload = {"uid": uid, "urls":urls, "maskedit":True, "extras":[extra1, extra2, extra3], "tip":tip}
+        response = send_and_wait(payload, timeout, uid, node_identifier)
+        
+        if (response.masked_image):
+            try:
+                return ( *(self.load_image(os.path.join('clipspace', response.masked_image)+" [input]")), *response.get_extras([extra1, extra2, extra3]) ) 
+            except FileNotFoundError:
+                pass
+
+        if if_no_mask == 'cancel': 
+            raise InterruptProcessingException()
+        return ( *(self.load_image(urls[0]['filename']+" [temp]")), *response.get_extras([extra1, extra2, extra3]) ) 

custom_nodes/cg-image-filter/image_filter_messaging.py

@@ -0,0 +1,109 @@
+from server import PromptServer
+from aiohttp import web
+from comfy.model_management import InterruptProcessingException, throw_exception_if_processing_interrupted
+import time, json
+from typing import Optional
+
+REQUEST_RESHOW = "-1"
+CANCEL = "-3"
+WAITING_FOR_RESPONSE = "-9"
+
+SPECIALS = [REQUEST_RESHOW, CANCEL, WAITING_FOR_RESPONSE]
+
+class Response:
+    def __init__(self, selection:Optional[list[int]] = None, text:Optional[str] = None,
+                        masked_image:Optional[str] = None, extras:Optional[list[str]] = None):
+        self.selection = selection
+        self.text = text
+        self.masked_image = masked_image
+        self.extras = extras
+
+    def get_extras(self,defaults:list[str]) -> list[str]:
+         return self.extras or defaults  
+
+class TimeoutResponse(Response): pass
+class CancelledResponse(Response): pass
+class RequestResponse(Response): pass
+
+class MessageState:
+    latest = None
+    unique_expected = None
+    def __init__(self, data:dict|str={}):
+        if not isinstance(data,dict): data = json.loads(data)
+        self.unique:str            = data.pop('unique', None)
+        self.special:Optional[int] = data.pop('special',None)
+        self.response:Response     = Response(**data)
+
+    @classmethod
+    def waiting_state(cls): return MessageState(data={'special':WAITING_FOR_RESPONSE})
+
+    @classmethod
+    def request_state(cls): return MessageState(data={'special':REQUEST_RESHOW})
+
+    @classmethod
+    def start_waiting(cls, unique): 
+        cls.latest = cls.waiting_state()
+        cls.unique_expected = unique
+
+    @classmethod
+    def get_response(cls) -> Response:  
+        if cls.waiting(): return TimeoutResponse()
+        if cls.latest.cancelled: return CancelledResponse()
+        if cls.latest.request: return RequestResponse()
+        return cls.latest.response
+
+    @classmethod
+    def stop_waiting(cls): cls.latest = MessageState()
+
+    @classmethod
+    def waiting(cls): return cls.latest.special == WAITING_FOR_RESPONSE
+
+    @property
+    def cancelled(self): return self.special == CANCEL
+
+    @property
+    def request(self): return self.special == REQUEST_RESHOW
+
+    @property
+    def real(self): return self.special is None
+
+
+@PromptServer.instance.routes.post('/cg-image-filter-message')
+async def cg_image_filter_message(request):
+    post     = await request.post()
+    response = post.get("response")
+    message  = MessageState(response)
+
+    if str(MessageState.unique_expected)==str(message.unique):
+        if (MessageState.waiting()):
+            MessageState.latest = message
+        else:
+            print(f"Ignoring response {response} because not waiting for one")
+    else:
+        print(f"Ignoring mismatched response {response}")
+
+    return web.json_response({})
+
+def wait_for_response(secs, uid, unique) -> Response:
+    MessageState.start_waiting(unique)
+    try:
+        end_time = time.monotonic() + secs
+        while(time.monotonic() < end_time and MessageState.waiting()): 
+            throw_exception_if_processing_interrupted()
+            PromptServer.instance.send_sync("cg-image-filter-images", {"tick": int(end_time - time.monotonic()), "uid": uid, "unique":unique})
+            time.sleep(0.5)
+        if MessageState.waiting():
+            PromptServer.instance.send_sync("cg-image-filter-images", {"timeout": True, "uid": uid, "unique":unique})
+        return MessageState.get_response()
+    finally: MessageState.stop_waiting()
+    
+def send_and_wait(payload, timeout, uid, unique) -> Response:
+    payload['uid'] = uid
+    payload['unique'] = unique
+
+    while True:
+        PromptServer.instance.send_sync("cg-image-filter-images", payload)
+        r = wait_for_response(timeout, uid, unique)
+        if isinstance(r,CancelledResponse): raise InterruptProcessingException()
+        if (not isinstance(r, RequestResponse)): return r
+      

custom_nodes/cg-image-filter/js/filter.css

@@ -0,0 +1,81 @@
+.cg_popup {
+    position: absolute;
+    width:100%;
+    height:100%;
+    background-color: rgba(0, 0, 0, 0.95);
+    z-index: 100000;
+    --text_area_height: 0px;
+}
+
+.cg_popup .grid {
+    position: absolute;
+    width:70%;
+    left:15%;
+    height:calc(100% - 60px - var(--text_area_height));
+    top:55px;
+    display: grid;
+    justify-items: center;
+    align-items: center;
+}
+
+.cg_popup .overlaygrid {
+    opacity: 0.5;
+    pointer-events: none;
+}
+
+.cg_popup .title {
+    position: absolute;
+    width:100%;
+    text-align: center;
+    top: 10px;
+    font-size: 250%;
+}
+
+.cg_popup .tip {
+    display: block;
+    padding-top: 10px;
+    line-height: 20px;
+}
+
+.cg_popup .buttons {
+    position: absolute;
+    width:14%;
+    left:86%;
+    height:90%;
+    top:5%;
+    font-size: small;
+}
+
+.cg_popup .control_text:after {
+    content:"\a";
+    white-space: pre;
+}
+
+.cg_popup .grid img {
+    margin: 2px;
+    border: 1px solid white;
+    padding: 4px;
+    max-width: 100%;
+    max-height: 100%;
+}
+
+.cg_popup .grid img.selected {
+    border: 3px solid green;
+    padding: 2px;
+}
+
+.cg_popup .grid img.hover {
+    box-shadow: 0 0 4px 4px red;
+}
+
+.cg_popup.hidden {
+    display:none !important;
+}
+
+.cg_popup .hidden {
+    display: none !important;
+}
+
+.cgfloat .hidden {
+    display: none !important;
+}

custom_nodes/cg-image-filter/js/floating_window.css

@@ -0,0 +1,94 @@
+
+.cgfloat {
+    position: absolute;
+    z-index: 999999;
+    border: 1px solid black;
+    background-color: rgba(95, 158, 160, 0.713);
+    width: fit-content;
+}
+
+.cgfloat.hidden {
+    display:none
+}
+
+.cgfloat_header {
+    width: 100%;
+    text-align: center;
+    background-color: rgba(28, 51, 52, 0.713);
+    color: whitesmoke;
+    border-bottom: 1px solid black;
+    padding:4px;
+    font-size:larger;
+}
+
+.tiny .cgfloat_header {
+    font-size: smaller;
+}
+
+.cgfloat_body {
+    min-width: fit-content;
+    display:flex;
+    flex-direction: column;
+}
+
+.cgfloat_body .row {
+    color: white;
+    padding: 0px 4px;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    border-bottom: 1px dashed rgba(0, 0, 0, 0.45);
+}
+
+/* Countdown */
+.cgfloat_body .counter_text {
+    min-width:50px;
+    text-align: right;
+}
+
+.cgfloat_body .counter_reset {
+    margin-left: 5px;
+    padding: 1px;
+}
+
+/* Buttons */
+.cgfloat_body .row.buttons {
+    justify-content: space-evenly;
+}
+
+.cgfloat_body button {
+    margin: 4px;
+    padding: 2px;
+    min-width: 70px;
+}
+
+/* Extras */
+.cgfloat_body .row.extras {
+    flex-direction: column;
+}
+
+.cgfloat_body .row.extras .extra {
+    width: 100%;
+    margin: 2px 0;
+}
+
+/* Tips */
+.cgfloat_body .row.tip {
+    margin: 4px;
+    padding: 4px 0px;
+    justify-content: flex-start;
+}
+
+/* Text Edit */
+.cgfloat_body .row.text_edit {
+    margin: 4px;
+}
+
+.cgfloat .tiny_image {
+    padding: 1px;
+    width: auto;
+    max-height: 100px;
+    object-fit: contain;
+    pointer-events: none;
+}
+

custom_nodes/cg-image-filter/js/floating_window.js

@@ -0,0 +1,62 @@
+
+export class FloatingWindow extends HTMLDivElement {
+    constructor(title, x, y, parent, movecallback) {
+        super()
+        this.movecallback = movecallback
+        this.classList.add('cgfloat')
+        this.header = document.createElement('div')
+        this.header.classList.add('cgfloat_header')
+        this.header.innerText = title
+        this.append(this.header)
+        this.body = document.createElement('div')
+        this.body.classList.add('cgfloat_body')
+        this.append(this.body)
+
+        this.header.addEventListener('mousedown',this.header_mousedown.bind(this))
+        document.addEventListener('mouseup',this.header_mouseup.bind(this))
+        document.addEventListener('mousemove',this.header_mousemove.bind(this))
+        document.addEventListener('mouseleave',this.header_mouseup.bind(this))
+        
+        this.dragging = false
+        this.move_to(x,y)
+        
+
+        if (parent) parent.append(this)
+        else document.body.append(this)
+    }
+
+    show() { this.style.display = 'block' }
+    hide() { this.style.display = 'none' }
+    set_title(title) { this.header.innerText = title }
+
+    move_to(x,y,supress) {
+        this.position = {x:x,y:y}
+        this.style.left = `${this.position.x}px`
+        this.style.top = `${this.position.y}px`
+        if (!supress) this.movecallback(x,y)
+    }
+
+    swallow(e) {
+        e.stopPropagation()
+        e.preventDefault()
+    }
+
+    header_mousedown(e) {
+        this.dragging = true
+        this.swallow(e)
+    }
+
+    header_mouseup(e) {
+        this.dragging = false
+    }
+
+    header_mousemove(e) {
+        if (this.dragging) {
+            this.move_to( this.position.x + e.movementX , this.position.y + e.movementY )
+            this.swallow(e)
+        }
+    }
+
+
+}
+customElements.define('cg-floater',  FloatingWindow, {extends: 'div'})

custom_nodes/cg-image-filter/js/image_filter.js

@@ -0,0 +1,124 @@
+import { app } from "../../scripts/app.js";
+import { api } from "../../scripts/api.js";
+
+import { create } from "./utils.js";
+import { popup } from "./popup.js";
+import { ComfyWidgets } from "../../scripts/widgets.js";
+import { FloatingWindow } from "./floating_window.js";
+
+const FILTER_TYPES = ["Image Filter","Text Image Filter","Text Image Filter with Extras","Mask Image Filter"]
+
+app.registerExtension({
+	name: "cg.image_filter",
+    settings: [
+        {
+            id: "Image Filter. Image Filter",
+            name: "Version 1.6.1",
+            type: () => {
+                const x = document.createElement('span')
+                const a = document.createElement('a')
+                a.innerText = "Report issues or request features"
+                a.href = "https://github.com/chrisgoringe/cg-image-filter/issues"
+                a.target = "_blank"
+                a.style.paddingRight = "12px"
+                x.appendChild(a)
+                return x
+            },
+        },
+        {
+            id: "Image Filter.UI.Play Sound",
+            name: "Play sound when activating",
+            type: "boolean",
+            defaultValue: true
+        },
+        {
+            id: "Image Filter.UI.Enlarge Small Images",
+            name: "Enlarge small images in grid",
+            type: "boolean",
+            defaultValue: true
+        },
+        {
+            id: "Image Filter.Actions.Click Sends",
+            name: "Clicking an image sends it",
+            tooltip: "Use if you always want to send exactly one image.",
+            type: "boolean",
+            defaultValue: false
+        },
+        {
+            id: "Image Filter.Actions.Autosend Identical",
+            name: "If all images are identical, autosend one",
+            type: "boolean",
+            defaultValue: false
+        },
+        {
+            id: "Image Filter.UI.Start Zoomed",
+            name: "Enter the Image Filter node with an image zoomed",
+            type: "combo",
+            options: [ {value:0, text:"No"}, {value:"1", text:"first"}, {value:"-1", text:"last"} ],
+            default: 0,
+        },
+        {
+            id: "Image Filter.UI.Small Window",
+            name: "Show a small popup instead of covering the screen",
+            type: "boolean",
+            tooltip: "Click the small popup to activate it",
+            defaultValue: false
+        },
+        {
+            id: "Image Filter.Z.Detailed Logging",
+            name: "Turn on detailed logging",
+            tooltip: "If you are asked to for debugging!",
+            type: "boolean",
+            defaultValue: false
+        },
+        {
+            id: "Image Filter.Video.FPS",
+            name: "Video Frames per Second",
+            type: "int",
+            defaultValue: 5,
+        }
+    ],
+    setup() {
+        create('link', null, document.getElementsByTagName('HEAD')[0], 
+            {'rel':'stylesheet', 'type':'text/css', 'href': new URL("./filter.css", import.meta.url).href } )
+        create('link', null, document.getElementsByTagName('HEAD')[0], 
+            {'rel':'stylesheet', 'type':'text/css', 'href': new URL("./floating_window.css", import.meta.url).href } )
+        create('link', null, document.getElementsByTagName('HEAD')[0], 
+            {'rel':'stylesheet', 'type':'text/css', 'href': new URL("./zoomed.css", import.meta.url).href } )
+        api.addEventListener("execution_interrupted", popup.send_cancel.bind(popup));
+        api.addEventListener("cg-image-filter-images",popup.handle_message.bind(popup));
+    },
+    async beforeRegisterNodeDef(nodeType) {
+        if (nodeType.comfyClass == "Pick from List") {
+            const onConnectionsChange = nodeType.prototype.onConnectionsChange;
+            nodeType.prototype.onConnectionsChange = function (side,slot,connect,link_info,output) {
+                if (side==1 && slot==0 && link_info && connect) {
+                    const type = this.graph._nodes_by_id[link_info.origin_id].outputs[link_info.origin_slot].type
+                    this.outputs[0].type = type
+                    this.inputs[0].type = type
+                } else if (side==1 && slot==0 && !connect) {
+                    const type = "*"
+                    this.outputs[0].type = type
+                    this.inputs[0].type = type
+                }
+                return onConnectionsChange ? onConnectionsChange.apply(this, arguments) : undefined;
+            }
+        }
+        if (FILTER_TYPES.includes(nodeType.comfyClass )) {
+            const onNodeCreated = nodeType.prototype.onNodeCreated;
+            nodeType.prototype.onNodeCreated = function () {
+                this._ni_widget = this.widgets.find((n)=>n.name=='node_identifier')
+                if (!(this._ni_widget)) {
+                    this._ni_widget = ComfyWidgets["INT"](this, "node_identifier", ["INT", { "default":0 }], app).widget
+                }
+                this._ni_widget.hidden = true
+                this._ni_widget.computeSize = () => [0,0]
+                this._ni_widget.value = Math.floor(Math.random() * 1000000)
+
+                return onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
+            }
+        }
+    },
+
+
+})

custom_nodes/cg-image-filter/js/log.js

@@ -0,0 +1,18 @@
+import { app } from "../../scripts/app.js";
+
+export class Log {
+    static log(s) { if (s) console.log(s) }
+    static error(e) { console.error(e) }
+    static detail(s) {
+        if (app.ui.settings.getSettingValue("Image Filter.Z.Detailed Logging")) Log.log(s)
+    }
+    static message_in(message, extra) {
+        if (!app.ui.settings.getSettingValue("Image Filter.Z.Detailed Logging")) return
+        if (message.detail && !message.detail.tick) Log.log(`--> ${JSON.stringify(message.detail)}` + (extra ? ` ${extra}` : ""))
+        if (message.detail && message.detail.tick) Log.log(`--> tick`)
+    }
+    static message_out(response, extra) {
+        if (!app.ui.settings.getSettingValue("Image Filter.Z.Detailed Logging")) return
+        Log.log(`"<-- ${JSON.stringify(response)}` + (extra ? ` ${extra}` : ""))
+    }
+}

custom_nodes/cg-image-filter/js/mask_utils.js

@@ -0,0 +1,43 @@
+import { app, ComfyApp } from "../../scripts/app.js";
+
+export function new_editor() {
+    return app.ui.settings.getSettingValue('Comfy.MaskEditor.UseNewEditor')
+}
+
+function get_mask_editor_element() {
+    return new_editor() ? document.getElementById('maskEditor') : document.getElementById('maskCanvas')?.parentElement
+}
+
+export function mask_editor_showing() {
+    return get_mask_editor_element() && get_mask_editor_element().style.display != 'none'
+}
+
+export function hide_mask_editor() {
+    if (mask_editor_showing()) document.getElementById('maskEditor').style.display = 'none'
+}
+
+function get_mask_editor_cancel_button() {
+    if (document.getElementById("maskEditor_topBarCancelButton")) return document.getElementById("maskEditor_topBarCancelButton")
+    return get_mask_editor_element?.parentElement?.lastChild?.childNodes[2]
+}
+
+function get_mask_editor_save_button() {
+    if (document.getElementById("maskEditor_topBarSaveButton")) return document.getElementById("maskEditor_topBarSaveButton")
+    return get_mask_editor_element?.parentElement?.lastChild?.childNodes[2]
+}
+
+export function mask_editor_listen_for_cancel(callback) {
+    const cancel_button = get_mask_editor_cancel_button()
+    if (cancel_button && !cancel_button.filter_listener_added) {
+        cancel_button.addEventListener('click', callback )
+        cancel_button.filter_listener_added = true
+    }
+}
+
+export function press_maskeditor_save() {
+    get_mask_editor_save_button()?.click()
+}
+
+export function press_maskeditor_cancel() {
+    get_mask_editor_cancel_button()?.click()
+}

custom_nodes/cg-image-filter/js/popup.js

@@ -0,0 +1,664 @@
+import { app, ComfyApp } from "../../scripts/app.js";
+import { api } from "../../scripts/api.js"
+
+import { mask_editor_listen_for_cancel, mask_editor_showing, hide_mask_editor, press_maskeditor_cancel, press_maskeditor_save, new_editor } from "./mask_utils.js";
+import { Log } from "./log.js";
+import { create } from "./utils.js";
+import { FloatingWindow } from "./floating_window.js";
+
+//const EXTENSION_NODES = ["Image Filter", "Text Image Filter", "Mask Image Filter", "Text Image Filter with Extras",]
+const POPUP_NODES = ["Image Filter", "Text Image Filter", "Text Image Filter with Extras",]
+const MASK_NODES = ["Mask Image Filter",]
+
+const REQUEST_RESHOW = "-1"
+const CANCEL = "-3"
+
+const GRID_IMAGE_SPACE = 10
+
+function get_full_url(url) {
+    return api.apiURL( `/view?filename=${encodeURIComponent(url.filename ?? v)}&type=${url.type ?? "input"}&subfolder=${url.subfolder ?? ""}&r=${Math.random()}`)
+}
+
+const State = Object.freeze({
+    INACTIVE    : 0,
+    TINY        : 1,
+    MASK        : 2,
+    FILTER      : 3,
+    TEXT        : 4,
+    ZOOMED      : 5,
+})
+
+class Popup extends HTMLSpanElement {
+    constructor() {
+        super()
+        this.audio = new Audio('extensions/cg-image-filter/ding.mp3');
+
+        this.classList.add('cg_popup')
+        
+        this.grid               = create('span', 'grid', this)
+        this.overlaygrid        = create('span', 'grid overlaygrid', this)
+        this.grid.addEventListener('click', this.on_click.bind(this))
+
+        this.zoomed             = create('span', 'zoomed', this)
+        this.zoomed_prev        = create('span', 'zoomed_prev', this.zoomed)
+        this.zoomed_prev_arrow  = create('span', 'zoomed_arrow', this.zoomed_prev, {innerHTML:"&#x21E6"})
+        this.zoomed_image       = create('img',  'zoomed_image', this.zoomed)
+        this.zoomed_next        = create('span', 'zoomed_next', this.zoomed)
+        this.zoomed_number      = create('span', 'zoomed_number', this.zoomed_next)
+        this.zoomed_next_arrow  = create('span', 'zoomed_arrow', this.zoomed_next, {innerHTML:"&#x21E8"})
+
+        this.zoomed_prev_arrow.addEventListener('click', this.zoom_prev.bind(this))
+        this.zoomed_next_arrow.addEventListener('click', this.zoom_next.bind(this))
+        this.zoomed_image.addEventListener('click', this.click_zoomed.bind(this))
+
+        this.tiny_window = new FloatingWindow('', 100, 100, null, this.tiny_moved.bind(this))
+        this.tiny_window.classList.add('tiny')
+        this.tiny_image         = create('img', 'tiny_image', this.tiny_window.body)
+        this.tiny_window.addEventListener('click', this.handle_deferred_message.bind(this))
+
+        this.floating_window = new FloatingWindow('', 100, 100, null, this.floater_moved.bind(this))
+
+        this.counter_row          = create('span', 'counter row', this.floating_window.body)
+        this.counter_reset_button = create('button', 'counter_reset', this.counter_row, {innerText:"Reset"} )
+        this.counter_text         = create('span', 'counter_text', this.counter_row)
+        this.counter_reset_button.addEventListener('click', this.request_reset.bind(this) ) 
+
+        this.extras_row = create('span', 'extras row', this.floating_window.body)
+
+        this.tip_row = create('span', 'tip row', this.floating_window.body)
+
+        this.button_row    = create('span', 'buttons row', this.floating_window.body)
+        this.send_button   = create('button', 'control', this.button_row, {innerText:"Send"} )
+        this.cancel_button = create('button', 'control', this.button_row, {innerText:"Cancel"} )
+        this.send_button.addEventListener(  'click', this.send_current_state.bind(this) )
+        this.cancel_button.addEventListener('click', this.send_cancel.bind(this) )
+
+        this.mask_button_row    = create('span', 'buttons row', this.floating_window.body)
+        this.mask_send_button   = create('button', 'control', this.mask_button_row, {innerText:"Send"} )
+        this.mask_cancel_button = create('button', 'control', this.mask_button_row, {innerText:"Cancel"} )
+        this.mask_send_button.addEventListener(  'click', press_maskeditor_save )
+        this.mask_cancel_button.addEventListener('click', press_maskeditor_cancel )
+
+        this.text_edit = create('textarea', 'text_edit row', this.floating_window.body)
+
+        this.picked = new Set()
+    
+        document.addEventListener("keydown", this.on_key_down.bind(this))
+        document.addEventListener("keypress", this.on_key_press.bind(this))
+
+        document.body.appendChild(this)
+        this.last_response_sent = 0
+        this.state = State.INACTIVE
+        this.render()
+    }
+
+    floater_moved(x,y) {
+        if (this.node?.properties) {
+            this.node.properties['filter_floater_xy'] = {x:x,y:y}
+        }
+    }
+
+    floater_position() {
+        return this.node?.properties?.['filter_floater_xy']
+    }
+
+    tiny_moved(x,y) {
+        if (this.node?.properties) {
+            this.node.properties['filter_tiny_xy'] = {x:x,y:y}
+        }
+    }
+
+    tiny_position() {
+        return this.node?.properties?.['filter_tiny_xy']
+    }
+
+    visible(item, value) {
+        if (value) item.classList.remove('hidden')
+        else item.classList.add('hidden')
+    }
+    disabled(item, value) {
+        item.disabled = value
+    }
+    highlighted(item, value) {
+        if (value) item.classList.add('highlighted')
+        else item.classList.remove('highlighted')
+    }
+
+    render() {
+        const state = this.state
+        this.visible(this, (state==State.FILTER || state==State.TEXT || state==State.ZOOMED))
+
+        this.visible(this.tiny_window, state==State.TINY)
+
+        this.visible(this.zoomed, state==State.ZOOMED)
+
+        this.visible(this.floating_window, (state==State.FILTER || state==State.ZOOMED || state==State.TEXT || state==State.MASK))
+        this.visible(this.button_row, state!=State.MASK)
+        this.disabled(this.send_button, (state==State.FILTER || state==State.ZOOMED) && this.picked.size==0)
+        this.visible(this.mask_button_row, state==State.MASK && new_editor())
+        this.visible(this.extras_row, this.n_extras>0)
+        this.visible(this.tip_row, this.tip_row.innerHTML.length>0)
+        this.visible(this.text_edit, state==State.TEXT)
+        
+        if (state==State.ZOOMED) {
+            const img_index = this.zoomed_image_holder.image_index
+            this.highlighted(this.zoomed, this.picked.has(`${img_index}`))
+            this.zoomed_number.innerHTML = `${img_index+1}/${this.n_images}`
+        }
+
+        if (state!=State.MASK) hide_mask_editor()
+    }
+
+    _send_response(msg={}, keep_open=false) {
+        /*
+        msg is a dict. Valid keys are:
+        *selection    (list[int])
+        *text         (string)
+         special      (int)
+         masked_image (string)
+        *extras       (list of strings)
+        *unique       (string)
+                (*) are added
+        */
+        if (Date.now()-this.last_response_sent < 1000) {
+            Log.message_out(msg, "(throttled)")
+            return
+        }
+
+        const unique = this.node?._ni_widget?.value
+        if (!unique) {
+            if (this.node) Log.error(`Node ${this.node.id} has no _ni_widget when trying to send ${msg}`)
+            else Log.error(`No node when trying to send ${msg}`)
+            return
+        }
+        msg.unique = `${unique}`
+
+        if (!msg.special) {
+            if (this.n_extras>0) {
+                msg.extras = []
+                Array.from(this.extras_row.children).forEach((e)=>{ msg.extras.push(e.value) })
+            }
+            if (this.state==State.FILTER || this.state==State.ZOOMED) msg.selection = Array.from(this.picked)
+            if (this.state==State.TEXT) msg.text = this.text_edit.value
+            
+            this.last_response_sent = Date.now()
+        }
+
+        try {
+            const body = new FormData();
+            body.append('response', JSON.stringify(msg));
+            api.fetchApi("/cg-image-filter-message", { method: "POST", body, });
+            Log.message_out(msg)
+        } catch (e) { 
+            Log.error(e) 
+        } finally { 
+            if (!keep_open) this.close() 
+        }
+
+    }
+
+    send_current_state() { 
+        if (this.state == State.TEXT) {
+            this._send_response()
+        } else {
+            this._send_response()
+        }
+    }
+    
+    send_cancel() { this._send_response({special:CANCEL}) }
+
+    request_reset() { this._send_response({special:REQUEST_RESHOW}, true) }
+
+    close() { 
+        this.state = State.INACTIVE
+        this.render()
+    }
+
+    maybe_play_sound() { if (app.ui.settings.getSettingValue("Image Filter.UI.Play Sound")) this.audio.play(); }
+
+    handle_message(message) { 
+        Log.message_in(message)
+        Log.log( this._handle_message(message, false) )
+        this.render()
+    }
+
+    handle_deferred_message(e) {
+        Log.message_in(this.saved_message, "(deferred)")
+        Log.log( this._handle_message(this.saved_message, true) )
+        this.render()
+    }
+
+    autosend() {
+        return (app.ui.settings.getSettingValue("Image Filter.Actions.Autosend Identical") && this.allsame)
+    }
+
+    on_new_node(nd) {
+        this.node = nd
+        const fp = this.floater_position()
+        if (fp) this.floating_window.move_to(fp.x, fp.y, true)
+        const tp = this.tiny_position()
+        if (tp) this.tiny_window.move_to(tp.x, tp.y, true)
+    }
+
+    find_node(uid) {
+        const bits = uid.split(':')
+        if (bits.length==1) {
+            return app.graph._nodes_by_id[uid]
+        } else {
+            var graph = app.graph
+            var node
+            bits.forEach((bit)=>{
+                node = graph._nodes_by_id[bit]
+                graph = node.subgraph
+            })
+        }
+        return node
+    }
+
+    _handle_message(message, using_saved) {
+        const detail = message.detail
+        const uid = detail.uid
+        const the_node = this.find_node(uid)
+
+        if (this.node!=the_node) this.on_new_node(the_node)
+
+        if (!this.node) return console.log(`Message was for ${uid} which doesn't exist`)
+        if (this.node._ni_widget?.value != message.detail.unique) return console.log(`Message unique id wasn't mine`)
+
+        if (detail.tick) {
+            this.counter_text.innerText = `${detail.tick}s`
+            if (this.state==State.INACTIVE) this.request_reset()
+            return
+        }
+
+        if (detail.timeout) {
+            this.close()
+            return `Timeout` 
+        }
+
+        if (this.handling_message) return `Ignoring message because we're already handling a message`
+
+        this.set_title(this.node.title ?? "Image Filter")
+        this.allsame = detail.allsame || false
+        if (detail.tip) this.tip_row.innerHTML = detail.tip.replace(/(?:\r\n|\r|\n)/g, '<br/>')
+        else this.tip_row.innerHTML = ""
+
+        if (this.state==State.INACTIVE && app.ui.settings.getSettingValue("Image Filter.UI.Small Window") && !using_saved && !this.autosend()) {
+            this.state = State.TINY
+            this.saved_message = message
+            this.tiny_image.src = get_full_url(message.detail.urls[message.detail.urls.length-1])
+            this.maybe_play_sound()
+            return `Deferring message and showing small window`
+        }
+
+        try {
+            this.handling_message = true
+            this.n_extras = detail.extras ? message.detail.extras.length : 0 
+            this.extras_row.innerHTML = ''
+            for (let i=0; i<this.n_extras; i++) { create('input', 'extra', this.extras_row, {value:detail.extras[i]}) }
+            
+            if (!using_saved && !this.autosend()) this.maybe_play_sound()
+
+            if (detail.maskedit)   this.handle_maskedit(detail) 
+            else if (detail.urls)  this.handle_urls(detail)
+
+        } finally { this.handling_message = false }  
+    }
+
+
+
+    window_not_showing(uid) {
+        const node = this.find_node(uid)
+        return (
+            (POPUP_NODES.includes(node.type) && this.classList.contains('hidden')) ||
+            (MASK_NODES.includes(node.type) && !mask_editor_showing())
+        )
+    }
+
+    set_title(title) {
+        this.floating_window.set_title(title)
+        var pos = this.floater_position()
+        if (pos) this.floating_window.move_to(pos.x, pos.y)
+        pos = this.tiny_position()
+        if (pos) this.tiny_window.move_to(pos.x, pos.y)
+        this.tiny_window.set_title(title)
+    }
+
+    handle_maskedit(detail) {
+        this.state = State.MASK
+
+        //this.node = this.find_node(detail.uid)
+        this.node.imgs = []
+        detail.urls.forEach((url, i)=>{ 
+            this.node.imgs.push( new Image() );
+            this.node.imgs[i].src = api.apiURL( `/view?filename=${encodeURIComponent(url.filename)}&type=${url.type}&subfolder=${url.subfolder}`) 
+        })
+        ComfyApp.copyToClipspace(this.node)
+        ComfyApp.clipspace_return_node = this.node
+        ComfyApp.open_maskeditor()
+        this.seen_editor = false
+        setTimeout(this.wait_while_mask_editing.bind(this), 200)
+    }
+
+    wait_while_mask_editing() {
+        if (!this.seen_editor && mask_editor_showing()) {
+            mask_editor_listen_for_cancel( this.send_cancel.bind(this) )
+            this.render()
+            this.seen_editor = true
+        }
+
+        if (mask_editor_showing()) {
+            setTimeout(this.wait_while_mask_editing.bind(this), 100)
+        } else {
+            this._send_response({masked_image:this.extract_filename(this.node.imgs[0].src)})
+        } 
+    }
+
+    extract_filename(url_string) {
+        return (new URL(url_string)).searchParams.get('filename')
+    }
+
+    handle_urls(detail) {
+        this.video_frames = detail.video_frames || 1
+
+        // do this after the extras are set up so that we send the right extras
+        if (this.autosend()) {
+            return this._send_response({selection:[0,]})
+        }
+
+        this.autozoom_pending = false
+        if (detail.text != null) {
+            this.state = State.TEXT
+            //this.text_edit.innerHTML = detail.text
+            this.text_edit.value = detail.text
+            if (detail.textareaheight) this.text_edit.style.height = `${detail.textareaheight}px`
+        } else {
+            if (this.state != State.FILTER && this.state != State.ZOOMED && app.ui.settings.getSettingValue("Image Filter.UI.Start Zoomed")!=0) {
+                this.autozoom_pending = true
+            }
+            this.state = State.FILTER
+        }
+
+        this.n_images = detail.urls?.length
+
+        this.laidOut = -1
+
+        this.picked = new Set()
+        if (this.n_images==1) this.picked.add('0')
+
+        this.grid.innerHTML = ''
+        this.overlaygrid.innerHTML = ''
+        var latestImage = null
+
+        detail.urls.forEach((url, i)=>{
+            console.log(url)
+            if (i%this.video_frames == 0) {
+                const thisImage = create('img', null, this.grid, {src:get_full_url(url)})
+                latestImage = thisImage
+                latestImage.onload = this.layout.bind(this)
+                latestImage.image_index = i/this.video_frames
+                latestImage.addEventListener('mouseover', (e)=>this.on_mouse_enter(thisImage))
+                latestImage.addEventListener('mouseout', (e)=>this.on_mouse_out(thisImage))
+                latestImage.frames = [get_full_url(url),]
+            } else {
+                latestImage.frames.push(get_full_url(url))
+            }
+            if (detail.mask_urls) { create('img', null, this.overlaygrid, {src:get_full_url(detail.mask_urls[i])})}
+
+        })
+
+        this.layout()
+
+        if (this.video_frames>1) {
+            this.frame = 0
+            setTimeout(this.advance_videos.bind(this), 1000)
+        }
+        
+    }
+
+    advance_videos() {
+        if (this.state == State.INACTIVE) return
+        
+        this.frame = (this.frame+1)%this.video_frames
+        Array.from(this.grid.children).forEach((img)=>{img.src = img.frames[this.frame]})
+
+        const fps = app.ui.settings.getSettingValue("Image Filter.Video.FPS")
+        const delay = (fps>0) ? 1000/fps : 1000
+        setTimeout(this.advance_videos.bind(this), delay)
+    }
+
+    on_mouse_enter(img) {
+        this.mouse_is_over = img
+        this.redraw()
+    }
+
+    on_mouse_out(img) {
+        this.mouse_is_over = null
+        this.redraw()       
+    }
+
+    zoom_auto() {
+        this.autozoom_pending = false
+        if (app.ui.settings.getSettingValue("Image Filter.UI.Start Zoomed")==1) {
+            this.zoomed_image_holder = this.grid.firstChild 
+        } else if (app.ui.settings.getSettingValue("Image Filter.UI.Start Zoomed")==-1) {
+            this.zoomed_image_holder = this.grid.lastChild 
+        } else {
+            return
+        }
+        if (this.zoomed_image_holder.image_index>=0) {
+            this.state = State.ZOOMED
+            return this.show_zoomed()
+        } 
+    }
+    zoom_next() {
+        this.zoomed_image_holder = this.zoomed_image_holder.nextSibling || this.zoomed_image_holder.parentNode.firstChild
+        this.show_zoomed()   
+    } 
+    zoom_prev() {
+        this.zoomed_image_holder = this.zoomed_image_holder.previousSibling || this.zoomed_image_holder.parentNode.lastChild  
+        this.show_zoomed()     
+    }
+    click_zoomed() {
+        const fake_event = { target:this.zoomed_image_holder}
+        this.on_click(fake_event)
+        this.show_zoomed()   
+    }
+    show_zoomed() {
+        this.zoomed_image.src = this.zoomed_image_holder.src
+        return this.render() 
+    }
+    eat_event(e) {
+        e.stopPropagation()
+        e.preventDefault()
+    }
+
+    on_key_press(e) {
+        if (document.activeElement?.type=='text' || document.activeElement?.type=='textarea') {
+            if (this.floating_window.contains(document.activeElement) || this.contains(document.activeElement)) return
+        }
+        if (this.state!=State.INACTIVE && this.state!=State.TINY) {
+            this.eat_event(e)
+        }
+    }
+
+    on_key_down(e) {
+        if (document.activeElement?.type=='text' || document.activeElement?.type=='textarea') {
+            if (this.floating_window.contains(document.activeElement) || this.contains(document.activeElement)) return
+            if (this.state==State.INACTIVE && this.state==State.TINY) return
+        }
+        if (this.state==State.FILTER || this.state==State.TEXT) {
+            if (e.key=='Enter') {
+                this.send_current_state()
+                return this.eat_event(e)
+            }
+            if (e.key=='Escape') {
+                this.send_cancel()
+                return this.eat_event(e)
+            }
+            if (`${parseInt(e.key)}`==e.key) {
+                this.select_unselect(parseInt(e.key))
+                this.render()
+                return this.eat_event(e)
+            }
+        }
+
+        if (this.state==State.FILTER) {
+            if (e.key==' ' && this.mouse_is_over) {
+                this.state = State.ZOOMED
+                this.zoomed_image_holder = this.mouse_is_over
+                //this.on_mouse_out(this.mouse_is_over)
+                this.eat_event(e)
+                return this.show_zoomed()
+            }
+            if (e.key=='a' && e.ctrlKey) {
+                if (this.picked.size>this.n_images/2) {
+                    this.picked.clear()
+                    console.log('unselect all')
+                } else {
+                    this.picked.clear()
+                    for (var i=0; i<this.n_images; i++) {
+                        this.picked.add(`${i}`)
+                    }
+                    console.log('select all')
+                }
+                this.eat_event(e)
+                return this.redraw() 
+            }
+        }
+        
+        if (this.state==State.ZOOMED) {
+            if (e.key==' ') {
+                this.state = State.FILTER
+                this.zoomed_image_holder = null
+                this.eat_event(e)
+                return this.render() 
+            } else if (e.key=='ArrowUp') {
+                this.click_zoomed()
+                return this.eat_event(e)
+            } else if (e.key=='ArrowDown') {
+                // select or unselect    
+            } else if (e.key=='ArrowRight') {
+                this.zoom_next()
+                return this.eat_event(e)   
+            } else if (e.key=='ArrowLeft') {
+                this.zoom_prev()
+                return this.eat_event(e)        
+            }
+        }
+    }
+
+    select_unselect(n) {
+        if (n<0 || n>this.n_images) {
+            return
+        }
+        const s = `${n}`
+        if (app.ui.settings.getSettingValue("Image Filter.Actions.Click Sends")) {
+            this.picked.add(s)
+            this._send_response()
+        } else {
+            if (this.picked.has(s)) this.picked.delete(s)
+            else this.picked.add(s)
+            this.redraw()
+        }
+    }
+
+    on_click(e) {
+        if (e.target.image_index != undefined) {
+            this.select_unselect(e.target.image_index)
+        }
+    }
+
+    layout(norepeat) {
+        const box = this.grid.getBoundingClientRect()
+        if (this.laidOut==box.width) return
+
+        const im_w = this.grid.firstChild.naturalWidth
+        const im_h = this.grid.firstChild.naturalHeight
+        
+        if (!im_w || !im_h || !box.width || !box.height) {
+            if (!norepeat) setTimeout(this.layout.bind(this), 100, [true,])
+            return
+        } else {
+            var best_scale = 0
+            var best_pick
+            var per_row
+            for (per_row=1; per_row<=this.n_images; per_row++) {
+                const rows = Math.ceil(this.n_images/per_row)
+                const scale = Math.min( box.width/(im_w*per_row), box.height/(im_h*rows) )
+                if (scale>best_scale) {
+                    best_scale = scale
+                    best_pick = per_row
+                }
+            }
+            this.per_row = best_pick
+            this.laidOut = box.width
+        }
+
+        this.rows = Math.ceil(this.n_images/this.per_row)    
+        const w = (box.width / this.per_row)-GRID_IMAGE_SPACE
+        const h = (box.height / this.rows)-GRID_IMAGE_SPACE
+
+        var template_columns = ''
+        for (let i=0; i<this.per_row; i++) template_columns += ` ${w+GRID_IMAGE_SPACE}px`  
+        var template_rows = ''
+        for (let i=0; i<this.rows; i++) template_rows += ` ${h+GRID_IMAGE_SPACE}px`
+        this.grid.style.gridTemplateColumns = template_columns
+        this.grid.style.gridTemplateRows = template_rows
+        this.overlaygrid.style.gridTemplateColumns = template_columns
+        this.overlaygrid.style.gridTemplateRows = template_rows
+
+        Array.from(this.grid.children).forEach((c,i)=>{
+            c.style.gridArea = `${Math.floor(i/this.per_row) + 1} / ${i%this.per_row + 1} /  auto / auto`; 
+        })
+        Array.from(this.overlaygrid.children).forEach((c,i)=>{
+            c.style.gridArea = `${Math.floor(i/this.per_row) + 1} / ${i%this.per_row + 1} /  auto / auto`; 
+        })
+
+        this.redraw()
+        setTimeout(this.rescale_images.bind(this), 100)
+
+        if (this.autozoom_pending) {
+            this.zoom_auto()
+        }
+    }
+
+    rescale_images() {
+        /*const justify = /*this.per_row > 1 ? "start" : "center"
+        const align = /*this.rows > 1 ? "start" : "center"
+        this.grid.style.justifyItems = justify
+        this.grid.style.alignItems = align
+        this.overlaygrid.style.justifyItems = justify
+        this.overlaygrid.style.alignItems = align*/
+
+        const box = this.grid.getBoundingClientRect()
+        const sub = this.grid.firstChild.getBoundingClientRect()
+        const w_used = (sub.width+GRID_IMAGE_SPACE)*this.per_row / box.width
+        const h_used = (sub.height+GRID_IMAGE_SPACE)*this.rows / box.height
+        const could_zoom = 1.0 / Math.max(w_used, h_used)
+        if (could_zoom>1 && app.ui.settings.getSettingValue("Image Filter.UI.Enlarge Small Images")) {
+            Array.from(this.grid.children).forEach((img)=>{
+                img.style.width = `${sub.width*could_zoom}px`
+            })
+            Array.from(this.overlaygrid.children).forEach((img)=>{
+                img.style.width = `${sub.width*could_zoom}px`
+            })
+        }
+    
+    }
+
+    redraw() {
+        Array.from(this.grid.children).forEach((c,i)=>{
+            if (this.picked.has(`${i}`)) c.classList.add('selected')
+            else c.classList.remove('selected')
+
+            if (c == this.mouse_is_over) c.classList.add('hover')
+            else c.classList.remove('hover')
+        }) 
+    }
+
+}
+
+customElements.define('cg-imgae-filter-popup', Popup, {extends: 'span'})
+
+export const popup = new Popup()

custom_nodes/cg-image-filter/js/utils.js

@@ -0,0 +1,8 @@
+
+export function create( tag, clss, parent, properties ) {
+    const nd = document.createElement(tag);
+    if (clss)       clss.split(" ").forEach((s) => nd.classList.add(s))
+    if (parent)     parent.appendChild(nd);
+    if (properties) Object.assign(nd, properties);
+    return nd;
+}

custom_nodes/cg-image-filter/js/zoomed.css

@@ -0,0 +1,61 @@
+
+.cg_popup .zoomed {
+    position: absolute;
+    width: 100%;
+    left: 0;
+    height: 100%;
+    right: 0;
+    display: flex;
+    justify-content: space-evenly;
+    align-items: center;
+    z-index: 120000;
+    background-color: rgba(29, 29, 29, 0.886);
+}
+
+.cg_popup .zoomed_prev, .zoomed_next {
+    flex-grow: 1;
+    
+}
+
+.cg_popup .zoomed_prev {
+    text-align: right;
+}
+
+.cg_popup .zoomed_arrow {
+    font-size: 48px;
+    border-radius: 40px;
+    padding: 0px 8px 4px 8px;
+    margin: 6px;
+    border: 1px solid rgb(128, 128, 128, 1.0);
+}
+
+.cg_popup .zoomed_arrow:hover {
+    border: 1px solid rgb(255, 128, 128, 1.0);
+}
+
+
+.cg_popup .zoomed_image {
+    max-width: calc(100% - 140px);
+    height: 100%;
+    object-fit: contain;
+    border: 1px solid white;
+    padding: 4px;
+}
+
+.cg_popup .zoomed_number {
+    position: absolute;
+    top: 12px;
+    margin-left: 12px;
+    color: white;
+    padding: 10px 8px 8px 8px;
+    border-radius: 4px;
+    z-index: 110000;
+    background-color: rgba(128, 128, 128, 0.5);
+    border: 1px solid rgba(128, 128, 128, 1.0);
+}
+
+.cg_popup .zoomed.highlighted .zoomed_image {
+    border: 3px solid green;
+    padding: 2px;
+}
+

custom_nodes/cg-image-filter/list_utility_nodes.py

@@ -0,0 +1,88 @@
+import torch
+from comfy.comfy_types.node_typing import IO
+    
+class BatchFromImageList:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { "images": ("IMAGE", ), } }
+    INPUT_IS_LIST = True
+    RETURN_TYPES = ("IMAGE", )
+    FUNCTION = "func"
+
+    CATEGORY = "image_filter/helpers"
+
+    def func(self, images):
+        if len(images) <= 1:
+            return (images[0],)
+        else:
+            return (torch.cat(list(i for i in images), dim=0),)
+        
+class ImageListFromBatch:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { "images": ("IMAGE", ), } }
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = [True,]
+    RETURN_TYPES = ("IMAGE", )
+    FUNCTION = "func"
+
+    CATEGORY = "image_filter/helpers"
+
+    def func(self, images):
+        image_list = list( i.unsqueeze(0) for i in images )
+        return (image_list,) 
+    
+class StringListFromStrings:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "s0": ("STRING", {"default":""}), 
+                "s1": ("STRING", {"default":""}), 
+            },
+            "optional": {    
+                "s2": ("STRING", {"default":""}), 
+                "s3": ("STRING", {"default":""}), 
+                }
+            
+            }
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = [True,]
+    RETURN_TYPES = ("STRING", )
+    FUNCTION = "func"
+
+    CATEGORY = "image_filter/helpers"
+
+    def func(self, s0,s1,s2=None,s3=None):
+        lst = [s0,s1]
+        if s2: lst.append(s2)
+        if s3: lst.append(s3)
+        return (lst,) 
+
+
+class PickFromList:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "anything" : (IO.ANY, ), 
+                "indexes": ("STRING", {"default": ""})
+            },
+        }
+    RETURN_TYPES = (IO.ANY,)
+    RETURN_NAMES = ("picks",)
+
+    FUNCTION = "func"
+    CATEGORY = "image_filter/helpers"
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = [True,]
+
+    def func(self, anything, indexes):
+        try:
+            if len(anything)==1 and isinstance(anything[0],list): anything = anything[0]
+            indexes = [int(x.strip()) for x in indexes[0].split(',') if x.strip()]
+        except Exception as e:
+            print(e)
+            indexes = []
+
+        return ([anything[i] for i in indexes], )

custom_nodes/cg-image-filter/mask_utility_nodes.py

@@ -0,0 +1,39 @@
+import torch
+
+class MaskedSection:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "mask": ("MASK",),
+                "image": ("IMAGE",),
+                "minimum": ("INT", {"default":512, "min":16, "max":4096})
+            }
+        }
+    
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "func"
+    CATEGORY = "image_filter/helpers"
+    
+    def func(self, mask:torch.Tensor, image, minimum=512):
+        mbb = mask.squeeze()
+        H,W = mbb.shape
+        masked = mbb > 0.5
+
+        non_zero_positions = torch.nonzero(masked)
+        if len(non_zero_positions) < 2: return (image,)
+
+        min_x = int(torch.min(non_zero_positions[:, 1]))
+        max_x = int(torch.max(non_zero_positions[:, 1]))
+        min_y = int(torch.min(non_zero_positions[:, 0]))
+        max_y = int(torch.max(non_zero_positions[:, 0]))
+
+        if (x:=(minimum-(max_x-min_x))//2)>0:
+            min_x = max(min_x-x, 0)
+            max_x = min(max_x+x, W)
+        if (y:=(minimum-(max_y-min_y))//2)>0:
+            min_y = max(min_y-y, 0)
+            max_y = min(max_y+y, H)       
+
+        return (image[:,min_y:max_y,min_x:max_x,:],)
+

custom_nodes/cg-image-filter/pyproject.toml

@@ -0,0 +1,13 @@
+[project]
+name = "cg-image-filter"
+description = "A set of custom nodes that pause a workflow while you select images, add masks, or edit text."
+version = "1.6.1"
+license = { file = "LICENSE" }
+
+[project.urls]
+Repository = "https://github.com/chrisgoringe/cg-image-filter"
+
+[tool.comfy]
+PublisherId = "chrisgoringe"
+DisplayName = "cg-image-filter"
+Icon = ""

custom_nodes/cg-image-filter/string_utility_nodes.py

@@ -0,0 +1,83 @@
+from comfy.comfy_types.node_typing import IO
+
+class SplitByCommas:
+    RETURN_TYPES = ("STRING","STRING","STRING","STRING","STRING","STRING")
+    FUNCTION = "func"
+    CATEGORY = "image_filter/helpers"
+    OUTPUT_NODE = False
+    OUTPUT_IS_LIST = [False, False, False, False, False, True]
+
+    DESCRIPTION = "Split the input string into up to five pieces. Splits on commas (or | or ^) and then strips whitespace from front and end."
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { "string" : ("STRING", {"default":""}), },
+            "optional": { "split": ([",", "|", "^"], {}), },
+        }
+    
+    def func(self, string:str, split:str=","):
+        bits = [r.strip() for r in string.split(split)] 
+        as_list = [b for b in bits]
+        if len(bits)<=5: 
+            bits += ["",]*(5-len(bits))
+        else:
+            bits = bits[:4] + [",".join(bits[4:]),]
+
+        bits.append(as_list)
+        return tuple(bits)
+    
+class AnyListToString:
+    RETURN_TYPES = ("STRING",)
+    FUNCTION = "func"
+    CATEGORY = "image_filter/helpers"
+    INPUT_IS_LIST  = True
+    OUTPUT_IS_LIST = (False,) 
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "anything" : (IO.ANY, ), 
+                "join" : ("STRING", {"default":""}), 
+            }
+        }
+    
+    def func(self, anything, join:str):
+        return ( join[0].join( [f"{x}" for x in anything] ), )
+    
+class StringToInt:
+    RETURN_TYPES = ("INT",)
+    FUNCTION = "func"
+    CATEGORY = "image_filter/helpers"
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "string" : ("STRING", {"default":"", "forceInput":True, "tooltip":"whitespace will be stripped before parsing"}), 
+                "default" : ("INT", {"default":0, "tooltip":"used if the string can't be parsed as an integer"}), 
+            }
+        }
+    
+    def func(self, string:str, default:int):
+        try:    return (int(string.strip()),)
+        except: return (default,)
+
+class StringToFloat:
+    RETURN_TYPES = ("FLOAT",)
+    FUNCTION = "func"
+    CATEGORY = "image_filter/helpers"
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "string" : ("STRING", {"default":"", "forceInput":True, "tooltip":"whitespace will be stripped before parsing"}), 
+                "default" : ("FLOAT", {"default":0, "tooltip":"used if the string can't be parsed as a float"}), 
+            }
+        }
+    
+    def func(self, string:str, default:float):
+        try:    return (float(string.strip()),)
+        except: return (default,)

custom_nodes/cg-use-everywhere/.gitattributes

@@ -0,0 +1,2 @@
+# Auto detect text files and perform LF normalization
+* text=auto

custom_nodes/cg-use-everywhere/.github/workflows/publish_action.yml

@@ -0,0 +1,24 @@
+name: Publish to Comfy registry
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - "pyproject.toml"
+
+permissions:
+  issues: write
+
+jobs:
+  publish-node:
+    name: Publish Custom Node to registry
+    runs-on: ubuntu-latest
+    if: ${{ github.repository_owner == 'chrisgoringe' }}
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+      - name: Publish Custom Node
+        uses: Comfy-Org/publish-node-action@v1
+        with:
+          personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }} ## Add your own personal access token to your Github Repository secrets and reference it here.

custom_nodes/cg-use-everywhere/.gitignore

@@ -0,0 +1,155 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintainted in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+.DS_Store
+workflow.pastel.json
+workflow.pfixed.json

custom_nodes/cg-use-everywhere/.tracking

@@ -0,0 +1,68 @@
+.gitattributes
+.github/workflows/publish_action.yml
+.gitignore
+LICENSE
+README-old.md
+README.md
+__init__.py
+docs/116.png
+docs/ComfyUI_temp_zbfdv_00012_.png
+docs/PE.png
+docs/UE3.png
+docs/UEQ.png
+docs/UEQportrait.png
+docs/auto.gif
+docs/bypass_catch1.png
+docs/bypass_catch2.png
+docs/clashes.png
+docs/conditioning.png
+docs/connectable.png
+docs/connected.png
+docs/connection-ui.png
+docs/deprecated.md
+docs/group.png
+docs/highway.png
+docs/image.png
+docs/imagex.png
+docs/kSampler.png
+docs/mouseOver.gif
+docs/off.png
+docs/on.png
+docs/options.png
+docs/options116.png
+docs/portrait.png
+docs/priority.gif
+docs/regex.png
+docs/restrictions.png
+docs/run.png
+docs/sampler and sigma.png
+docs/separate.png
+docs/showlinks.png
+docs/test-workflow-screenshot.png
+docs/test-workflow.json
+docs/test-workflow.png
+docs/unconnected.png
+docs/workflow.png
+js/floating_window.js
+js/i18n.js
+js/tooltip_window.js
+js/ue.css
+js/ue_debug.js
+js/ue_properties.js
+js/ue_properties_editor.js
+js/use_everywhere.js
+js/use_everywhere_apply.js
+js/use_everywhere_cache.js
+js/use_everywhere_classes.js
+js/use_everywhere_graph_analysis.js
+js/use_everywhere_settings.js
+js/use_everywhere_subgraph_utils.js
+js/use_everywhere_ui.js
+js/use_everywhere_utilities.js
+pyproject.toml
+tests/compare.png
+tests/test.md
+tests/test.png
+tests/test2.png
+use_everywhere.py
+workflow_fixer.py

custom_nodes/cg-use-everywhere/LICENSE

@@ -0,0 +1,201 @@
+Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

custom_nodes/cg-use-everywhere/README-old.md

@@ -0,0 +1,335 @@
+# UE Nodes
+
+Love this node? [Buy me a coffee!](https://www.buymeacoffee.com/chrisgoringe)
+
+Getting started? Download the test workflow below and see how it works.
+
+Problems? Jump down to [logging and debugging](https://github.com/chrisgoringe/cg-use-everywhere/blob/main/README.md#loggingdebugging)
+
+Ideas for how to improve the nodes (or bug reports) - [raise an issue](https://github.com/chrisgoringe/cg-use-everywhere/issues)
+
+Shameless plug for my other nodes -> Check out [Image Picker](https://github.com/chrisgoringe/cg-image-filter) for another way to make some workflows smoother. And leave a star if you like something!
+
+---
+
+# Important!
+
+I've merged v7 of UE to coincide with the update to ComfyUI, but the instructions below have not been updated.
+
+Please see [this discussion](https://github.com/chrisgoringe/cg-use-everywhere/discussions/334) (but ignore the bits about branches).
+
+---
+
+Over the next few releases I am aiming to reduce the UE zoo down to a single node that can do everything, but is as easy (or easier) to use than the current nodes.
+
+The first step in this process is merging the `AnythingEverywhere` and `AnythingEverywhere?` nodes by moving the regex restrictions into a separate dialog that can be used by either. In future iterations the group and color restrictions will also move to this dialog (so people find them!)
+
+---
+
+# 6.3 Update for ComfyUI front end (version TBC) - subgraphs
+
+With the addition of subgraphs, group nodes are no longer supported in UE from 6.3 onwards.
+
+Instead, UE 6.3 has the following support for subgraphs:
+
+- UE nodes can be used in the main graph and in subgraphs. 
+- UE nodes only broadcast within the graph or subgraph they are in. 
+- Subgraph nodes are just nodes. You can connect UE nodes to their outputs, and UE links will connect to their inputs.
+- Within a subgraph you can connect from the input panel to a UE node.
+- UE nodes within the subgraph will *not* broadcast to the output panel.
+- When you convert a set of nodes to a subgraph, UE links will work as long as:
+  - the UE node (Control) and the node feeding it (Source) are both in the subgraph, or
+  - neither the Source nor Control are in the subgraph, or
+  - the Source is in the subgraph, and the Control is not, and no Target nodes are in the subgraph
+
+
+## Subgraph creation
+
+There are three nodes involved in every UE link: 
+- Source (the link sending the data), 
+- Control (the UE node connected to the source), 
+- Target (the node that is receiving the data as an input)
+
+This is how those cases are treated:
+
+|Support|Source|Control|Target||
+|-|-|-|-|-|
+|Yes|Graph|Graph|Graph|Nothing changes|
+|Yes|Graph|Graph|Subgraph|The subgraph will have inputs for the data; in the subgraph the input panel is connected to the Target with a real link|
+|No|Graph|Subgraph|Graph|Not supported|
+|No*|Graph|Subgraph|Subgraph|Not supported|
+|Yes|Subgraph|Graph|Graph|The subgraph will be connected to the Control|
+|No|Subgraph|Graph|Subgraph|Not supported|
+|Yes|Subgraph|Subgraph|Graph|The Source will be connected to the Control *and* the output panel in the subgraph, the output will be connected to the Target with a real link|
+|Yes|Subgraph|Subgraph|Subgraph|All nodes will be connected in the subgraph as they were in the graph|
+
+No* indicates a case that does not work, but might get implemented.
+
+No indicates a case I'm unlikely ever to support
+
+---
+
+## Test workflow
+
+|This workflow uses all five nodes, and can be used to test (and understand!) the nodes. You wouldn't build it like this, it's just an example...|Here's an image with the workflow in|
+|-|-|
+|![screen](docs/test-workflow-screenshot.png)|![image](docs/test-workflow.png)|
+
+Or [the workflow as json](docs/test-workflow.json)
+
+## Current known limitations
+
+There are some situations that UE nodes can't cope with at present. Here are some I know about, and possible workarounds.
+
+### Pythonssss Preset Text
+
+[pythonsssss](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) custom nodes are great, but there are some limitations in using them with UE nodes. In particular, you can't feed the output of a Preset Text node directly into a UE node (see https://github.com/chrisgoringe/cg-use-everywhere/issues/154). 
+
+### Group nodes
+
+UE nodes mostly work with group nodes. But there are a couple of important things to note:
+
+- when you create a group node the input names and node names can change. This might break UE? regex connections.
+
+## Latest updates
+
+6.3 June/July 2025
+- support for ComfyUI subgraphs
+
+6.1 (May 2025)
+- Mostly some bug-fixes (#216, #217, #300)
+- New option to show UE links differently without animation for better performance
+
+6.0 (April 2025)
+- Very major rebuild to deal with latest version of Comfy
+
+## Installing
+
+Use Comfy Manager. If you really want to do it manually, just clone this repository in your custom_nodes directory.
+
+## Anything Everywhere (start here!)
+
+The `Anything Everywhere` node has a single input, initially labelled 'anything'. Connect anything to it (directly - not via a reroute), and the input name changes to match the input type. Disconnect and it goes back to 'anything'.
+
+When you run the prompt, any unconnected input, anywhere in the workflow, which matches that type, will act as if it were connected to the same input. 
+
+To visualise what it's being connected to, right-click on the background canvas and select `Toggle UE Link Visibility`.
+
+## Anything Everywhere? - control matching with regex rules
+
+This node adds two widgets - title_regex and input_regex. It will only send to inputs which match. So in the example, title_regex is 'Preview' so the image is sent to the Preview Image node but not the Save Image node. Note that you can rename node and input titles, which can help! 
+
+(From 4.6 you can also specify a group regex to only match inputs on nodes which are in groups that match the regex.)
+
+![regex](docs/regex.png)
+
+*The matches are regular expressions, not string matches.* Most simple strings will work (matching any part of the title or input name), but some characters have special meanings (including various sorts of brackets, ^, $, /, and . in particular) so just avoid them if you aren't regex-inclined.
+
+Using regex means you can use `^prompt` to match `prompt` at the beginning of the title only, to avoid matching `negative_prompt`.
+
+Regex 101 - `^` means 'the start', `$` means 'the end', `.` matches any single character, `.*` matches anything of any length (including zero). For more than that, visit [regex101](https://regex101.com/) (the flavour you want is ECMAScript, though that probably won't matter).
+
+### Can I make the regex an input instead of a widget?
+
+Sort of.
+
+Because the regex needs to be known before the workflow is submitted (in order to calculate the links), you can't pass a string into the `Anything Everywhere?` node and expect it to work. The *only* thing that is supported is if the input comes *directly* from a node which sets it with a string widget. The `Simple String` node that is included in this pack will work. 
+
+|This works|This doesn't. And never will.|
+|-|-|
+|![Alt text](docs/image.png)|![no](docs/imagex.png)|
+
+
+## Seed Everywhere
+
+Seed Everywhere connects to any unconnected INT input with `seed` in the input name (seed, noise_seed, etc), and it has the control_after_generate feature. So if you convert the seed widgets to inputs you can use the same seed everywhere.
+
+## Anything Everywhere3 - One node, three inputs.
+
+Really just three `Anything Everywhere` nodes packaged together.  Designed for the outputs of Checkpoint Loader. 
+
+![UE3](docs/UE3.png)
+
+## Prompts Everywhere - two strings or conditionings
+
+Prompt Everywhere has two inputs. They will be sent with regex matching rules of `(^prompt|^positive)` and `neg` respectively. These should match the various versions of names that get used for prompts and negative prompts or conditionings.
+
+|strings|conditionings|
+|-|-|
+|![pe](docs/PE.png)|![pe](docs/conditioning.png)
+
+# Primitives and COMBOs and the like
+
+UE nodes don't work with primitives and COMBOs (the data type used for dropdown lists, which are also a type of primitive within Comfy). It's unlikely they ever will. 
+
+If you want to use UE to control sampler or sigma, you can do this with the built in `SamplerCustom` nodes:
+
+![sample and sigma](docs/sampler%20and%20sigma.png)
+
+For more on this, see [this discussion](https://github.com/chrisgoringe/cg-use-everywhere/issues/69)
+
+# Other features
+
+## Third Party Integration
+
+At the suggestion of [@fighting-tx](https://github.com/fighting-tx), 
+I've added a method that third party nodes can use if they want to see the prompt as generated by UE. 
+It's attached to the `app` object, so you can check if it is present and use it something like this:
+
+```js
+var prompt
+if (app.ue_modified_prompt) {
+  prompt = await app.ue_modified_prompt()
+} else {
+  prompt = await original_graphToPrompt.apply(app)
+}
+```
+
+Other methods could be exposed if there is interest - raise an issue if you'd like to see something. 
+
+## Reject links
+
+Right click on a node and you can set it to reject UE links
+
+## Show links - visualisation and animation.
+
+If you want to see the UE links, you can turn them on and off by right-clicking on the canvas. For finer control, the main settings menu has options to show links when the mouse moves over the node at either end, or when one of those nodes is selected.
+
+The links can be animated to distinguish them from normal links - this animation can take the form of moving dots, a pulsing glow, or both. This may impact performance in some cases - note that the pulse animation requires less processing than the moving dots. Control this in the main settings menu.
+
+By default the animations turn off when the workflow is running to minimise impact on CPU/GPU - you can change this in the settings too.
+
+## Convert to real links
+
+If you want to share a workflow without UE nodes being required, or to save an API version of a workflow, you can replace the virtual links created by UE nodes with real links (and remove the UE nodes).
+
+This can be done for a single node by right-clicking on it and selecting `Convert to real links`, or for all UE nodes in a workflow by right-clicking the background and selecting `Convert all UEs to real links`.
+
+## Shift drag
+
+Shift click on an output node and drag then release to get an autocreate menu. This replaces the default behaviour (which gives you a search box), so you can disable it with the `Anything Everywhere replace search` setting.
+
+![auto](docs/auto.gif)
+
+## Group and color restriction
+
+UE nodes can be restricted to send only to nodes of the same color, or only to nodes that *aren't* the same color.
+
+They can also be restricted to send only to nodes in the same group (any group in common), or only to nodes that aren't in the same group.
+
+Right-click on the node and select `Group restrictions` or `Color restrictions`. UE nodes which are restricted (in either or both ways) have a green circle in the top-left corner. 
+
+## Highway nodes
+
+Trung 0246's [Highway nodes](https://github.com/Trung0246/ComfyUI-0246) are a pretty cool way of piping data around. You can target them with an `Anything Everywhere?` node by using an `input_regex` which matches the unconnected input name with the '+', like this:
+![highway](docs/highway.png)
+
+This is new, so please report any issues!
+
+## Loop checking
+
+By default workflows are checked for loops before they are submitted (because UE can introduce them, and a loop results in a bad python outcome). If a loop is detected you'll get a JavaScript warning showing you the node ids involved. However, especially if there are other custom nodes involved, it's possible that the check will miss a loop, or flag one that isn't real.
+
+If you get a warning and don't believe there is a loop (having checked the node ids listed!) you can turn loop checking off in the main settings menu. If something flagged as a loop runs fine, please [raise an issue](https://github.com/chrisgoringe/cg-use-everywhere/issues) and include the workflow in the report (save the json and zip it, because GitHub doesn't accept .json files). Likewise if a loop doesn't get caught.
+
+I've written code for the core Comfy backend to catch loops, maybe it'll be included - [PR for ComfyUI](https://github.com/comfyanonymous/ComfyUI/pull/1652) - or maybe they have another plan.
+
+## Priorities
+
+If there is more than one sending node that matches an input, the basic rules is that the more specific node wins. The order of priorities is:
+
+- `Anything Everywhere?` 
+- `Seed Everywhere` and `Prompts Everywhere`
+- `Anything Everywhere`
+- `Anything Everywhere3`
+
+For nodes of the same time, those with colour restrictions and group restriction are prioritised (colour+group > colour > group > none).
+
+If two nodes with the same priority both match *neither will connect* - better to fail fast than have an ambiguous outcome. If there are ambiguous matches you can display them using `Show UE broadcast clashes` (right-click on background - the option only appears if there are clashes).
+
+## See what is sent
+
+The nodes which only have one output can also gain a text box showing exactly what passed through the node. You need to turn this on if you want it - it's in the main settings, 'Anything Everywhere node details'.
+
+## Logging/Debugging
+
+The JavaScript console (press f12 in some browsers) has logging information about what is being connected. You can change the level of detail by finding the file `[comfy_install]/custom_nodes/cg-use-everywhere/js/use_everywhre_utilities.js` and near the top finding this bit:
+```javascript
+    static ERROR       = 0; // actual errors
+    static PROBLEM     = 1; // things that stop the workflow working
+    static INFORMATION = 2; // record of good things
+    static DETAIL      = 3; // details
+
+    static LEVEL = Logger.PROBLEM;
+    static TRACE = false;   // most of the method calls
+```
+Change the `LEVEL` to `Logger.INFORMATION` for more, or `Logger.DETAIL` for even more; set `TRACE` to `true` for some other debugging information.
+
+If you have a problem, pressing f12 to see the JavaScript console can often help. The following steps are really helpful in making a good bug report:
+
+- update to the latest version
+- restart ComfyUI
+- clear the canvas
+- close the browser
+- open a new Comfy window (with no workflow), look in console (f12) to see if there were any errors as ComfyUI started up
+- load your workflow, and look again
+- run, and look again
+
+The other thing worth trying is clearing out all the custom node javascript from where it gets copied when ComfyUI starts:
+
+- stop Comfy
+- go to [comfy root]/web/extensions     (*not* under custom_nodes)
+- remove everything there EXCEPT for `core`. Leave `core` (it's ComfyUI stuff)
+- restart Comfy (all custom nodes will reinstall their javascript at startup)
+
+If you find a bug, please [raise an issue](https://github.com/chrisgoringe/cg-use-everywhere/issues) - if you can include the workflow, that's a huge help (you'll need to save it as .txt, or zip the .json file, because GitHub doesn't accept .json).
+
+## Cautions
+
+Bypassing and disabling nodes works, but with one catch. If you have a UE nodes that does matching (`Anything Everywhere?` and `Prompt Everywhere`) and you bypass the node it matches to, the link won't be made. So
+
+|If you use a ? node to send to a node...|...and bypass the recipient, it doesn't get connected |
+|-|-|
+|![1](docs/bypass_catch1.png)|![2](docs/bypass_catch2.png)|
+
+This is unlikely to be fixed, but should be fairly easy to avoid!
+
+
+---
+# Some older information
+
+# Update for ComfyUI front end 1.16 and above
+
+ComfyUI front end 1.16 made a major change to the way that inputs and widgets work, which had a significant impact on the UE nodes.
+
+A widget is no longer converted to an input in order to connect to it; instead, you just connect an output to it, and it converts.
+
+This is quite cool - except that it meant that there was no way for UE to tell if a widget was supposed to be an empty input for UE to feed.
+
+So there is a new mechanism for this. Here's a picture of a node that I have right-clicked:
+
+![116](docs/116.png)
+
+Things to notice:
+
+- There is a new menu `UE Connectable Widgets` which lists all the widgets. Those marked as connectable have a green bar to the left of the name.
+  - Select a widget to toggle its value
+- Next to the widget, where an input would be, the connectable widgets have a graphical indication of their state.
+  - The smaller dark circle (next to steps) indicate a connectable node which is not connected (widget value will be used)
+  - The light circle (next to cfg) and greying out the widget indicates that a UE node will provide the value
+  - No icon (all the other widgets) indicates that they are not UE connectable
+- Other inputs still have the indications of normal connection or UE connection (here model is a UE connection)
+
+You'll probably find that turning the `showlinks` option on helps, and the `highlight` option is required for those widget indicators which help a lot.
+
+![options116](docs/options116.png)
+
+Hopefully old workflows will be automatically converted when you load them. Hopefully.
+
+## Naming widgets
+
+The `Prompts Everywhere`, `Seed Everywhere`, and `Anything Everywhere?` all make use of the name of an input. On some occasions this might mean you want to rename an input.
+
+For an input that can't be a widget, that's trivial - right click on the input dot and select `Rename Slot`. The name will be updated, and the new name displayed next to the dot.
+
+If an input *can* be a widget, you can rename it in the same way, but the new name *does not get displayed*. The name has changed, and the UE nodes will use the new name, but the old name is still shown. I consider this to be a bug in the Comfy front-end, and have raised an issue [here](https://github.com/Comfy-Org/ComfyUI_frontend/issues/3654). Unfortunately it appears that Comfy UI is going to remove the ability to rename a widget entirely, so best to avoid using widget input names if at all possible.

custom_nodes/cg-use-everywhere/README.md

@@ -0,0 +1,220 @@
+# UE Nodes
+
+Love this node? [Buy me a coffee!](https://www.buymeacoffee.com/chrisgoringe)
+
+Getting started? Download the test workflow below and see how it works.
+
+Problems? Jump down to [logging and debugging](https://github.com/chrisgoringe/cg-use-everywhere/blob/main/README.md#loggingdebugging)
+
+Ideas for how to improve the nodes (or bug reports) - [raise an issue](https://github.com/chrisgoringe/cg-use-everywhere/issues)
+
+Shameless plug for my other nodes -> Check out [Image Picker](https://github.com/chrisgoringe/cg-image-filter) for another way to make some workflows smoother. And leave a star if you like something!
+
+---
+
+# Anything Everywhere v7
+
+Version 7 is a major update to the Anything Everywhere nodes, so the documentation below is all new. If you are looking for the old docs, you can find them [here](https://github.com/chrisgoringe/cg-use-everywhere/README-old).
+
+If you are new to Anything Everywhere, skip to [Anything Everywhere]().
+
+## Major changes
+
+If you used Anything Everywhere prior to v7, the major improvements are:
+
+- The `Anything Everywhere3` and `Anything Everywhere?` nodes are deprecated, as their features are now part of the standard `Anything Everywhere` node.
+- `Anything Everywhere` nodes now have dynamic inputs, so you can plug as many different things into them as you like.
+  - At present you can only connect one input of any given data type, but this restriction should go away in `7.1`
+- All the restrictions on what nodes data will be sent to are now in a restrictions editor, that can be accessed through the right click menu of the node, or by double-clicking the body of the node.
+  - In the restrictions editor you can set title, input, and group regexes, color restrictions, group restrictions, and priority (for when two nodes both match)
+  - The green circle is used to indicate that _any_ restrictions are in place; if you hover over a node with restrictions they will appear in a tooltip
+- Subgraphs are supported (in the majority of cases). Yay subgraphs! Seriously, they are _so_ much better than group nodes.
+
+There are a couple of features that have been removed:
+
+- Group nodes are no longer supported, as they are deprecated in ComfyUI in favour of the new subgraphs, which are supported (in most configurations)
+- The `Simple String` mechanism to provide an input to the regex of an `Anything Everywhere?` node is no longer supported
+  - Other UI mechanisms to address this need are under consideration
+
+## Upgrade considerations
+
+Other than the limitations noted, old workflows _should_ load and work out of the box, 
+with `Anything Everywhere3` and `Anything Everywhere?` nodes automatically converted to `Anything Everywhere` nodes with the appropriate restrictions applied.
+
+However, there may be edge cases that don't work; if you have any problems, please [raise an issue](https://github.com/chrisgoringe/cg-use-everywhere/issues).
+
+You will _not_ be able to use workflows saved using v7 with older versions of ComfyUI or older versions of UE.
+
+# Anything Everywhere
+
+## Anything Everywhere 
+
+The `Anything Everywhere` node takes one or more inputs (currently limited to one input of any data type) and sends the data to other nodes that need it. 
+When you connect an input, a new one automatically appears.
+
+By default the data will be sent to any `input` of the same data type which does not have a connection.
+
+`Anything Everywhere` does _not_ send to `widgets` by default, but if you right-click on the node that you want to receive the data you can specify which widgets should accept UE connections. 
+
+![uec](docs/connectable.png)
+
+You can also constrain where the data gets send through  _restrictions_ applied to the `Anything Everywhere` node. 
+These restrictions can be accessed by double-clicking the body of the node, or through the right-click menu.
+
+![restrictions](docs/restrictions.png)
+
+The first three entries are [regex](https://regex101.com/) patterns. 
+The node will only send data to another node if the regex matches the receiving node title, the name of the input, or the name of a group the receiving node is in, respectively.
+
+The Group and Colour restrictions will contrain the node to only send to nodes in (or not in) the same group, and of the same (or different) colour.
+
+If you select multiple restrictions, all must be satisfied for the node to send.
+
+If any restrictions are applied, the `Anything Everywhere` node gets a green circle in the top left hand corner, and a tooltip if yuo hover the mouse over it.
+
+The final line in the restrictions box is the Priority, which has an automatically calculated value which you can choose to override. 
+If two more more `Anything Everywhere` nodes match the same input, the higher priority node is used. If there is a tie, _no connection is made_.
+When there is a tie, if you right-click on the canvas you will find an option to show which nodes are the problem.
+
+## Seed Everywhere
+
+Seed Everywhere connects to any unconnected INT input with `seed` in the input name (seed, noise_seed, etc), and it has the control_after_generate feature. So if you convert the seed widgets to inputs you can use the same seed everywhere.
+
+Hopefully this node will soon be retired, but if it is, workflows using it will be automatically converted.
+
+## Prompts Everywhere
+
+Prompt Everywhere has two inputs. They will be sent with regex matching rules of `(^prompt|^positive)` and `neg` respectively. These should match the various versions of names that get used for prompts and negative prompts or conditionings. The regexes can be edited, but additional inputs are not created. 
+
+|strings|conditionings|
+|-|-|
+|![pe](docs/PE.png)|![pe](docs/conditioning.png)
+
+Hopefully this node will soon be retired, but if it is, workflows using it will be automatically converted.
+
+## Primitives and COMBOs and the like
+
+UE nodes don't work with primitives and COMBOs (the data type used for dropdown lists, which are also a type of primitive within Comfy). It's unlikely they ever will. 
+
+For more on this, see [this discussion](https://github.com/chrisgoringe/cg-use-everywhere/issues/69)
+
+# Options
+
+In the main settings menu, you will find the Use Everywhere options:
+
+![options](docs/options.png)
+
+The top set, `Graphics`, modify the visual appearance only. 
+
+The bottom set, `Options`, modify behaviour:
+
+- Block workflow validation. This prevents other nodes from complaining about the lack of connections, or creating them. If you turn this off, there may be unexpected consequences.
+- Logging. Increase the logging level if you are asked to help debug.
+- Check loops before submitting will attempt to check for loops in the workflow created by Use Everywhere links. Comfy is much better at handling this now, so I would suggest leaving it off.
+- Connect to bypassed nodes. When off, Use Everywhere will not connect to a bypassed node, and will attempt to work out whether an input is connected when upstream nodes are bypassed. 
+
+*I recommend turning `Connect to bypassed nodes` on; the default is off for backward compatibility.*
+
+# Other features
+
+## Third Party Integration
+
+At the suggestion of [@fighting-tx](https://github.com/fighting-tx), 
+I've added a method that third party nodes can use if they want to see the prompt as generated by UE. 
+It's attached to the `app` object, so you can check if it is present and use it something like this:
+
+```js
+var prompt
+if (app.ue_modified_prompt) {
+  prompt = await app.ue_modified_prompt()
+} else {
+  prompt = await original_graphToPrompt.apply(app)
+}
+```
+
+Other methods could be exposed if there is interest - raise an issue if you'd like to see something. 
+
+## Reject links
+
+Right click on a node and you can set it to reject UE links
+
+## Show links - visualisation and animation.
+
+If you want to see the UE links, you can turn them on and off by right-clicking on the canvas. For finer control, the main settings menu has options to show links when the mouse moves over the node at either end, or when one of those nodes is selected.
+
+The links can be animated to distinguish them from normal links - this animation can take the form of moving dots, a pulsing glow, or both. This may impact performance in some cases - note that the pulse animation requires less processing than the moving dots. Control this in the main settings menu.
+
+By default the animations turn off when the workflow is running to minimise impact on CPU/GPU - you can change this in the settings too.
+
+## Convert to real links
+
+If you want to share a workflow without UE nodes being required, or to save an API version of a workflow, you can replace the virtual links created by UE nodes with real links (and remove the UE nodes).
+
+This can be done for a single node by right-clicking on it and selecting `Convert to real links`, or for all UE nodes in a workflow by right-clicking the background and selecting `Convert all UEs to real links`.
+
+---
+
+# Roadmap
+
+In the near future I hope to do the following:
+
+- Add a mechanism to support multiple inputs of the same type
+  - Once this is done, `Prompts Everywhere` will be retired
+- Add a mechanism to retire `Seed Everywhere` (or auto convert to an integer node connected to a `Anything Everywhere` node)
+- Add a clean way to localise for non-English language use
+- Enable individual inputs to reject connections
+- Negative regexes ('must not match')
+- Possibly add a global variable system (so multiple modes can be changed with a single modification)
+
+Feel free to [make suggestions](https://github.com/chrisgoringe/cg-use-everywhere/issues)
+
+---
+
+# More detailed notes on a few things
+
+## Naming widgets
+
+The restrictions make use of the name of an input. On some occasions this might mean you want to rename an input. 
+
+For an input that can't be a widget, that's trivial - right click on the input dot and select `Rename Slot`. The name will be updated, and the new name displayed next to the dot.
+
+If an input *can* be a widget, you cannot rename it. This is a decision taken by the Comfy team (see [here](https://github.com/Comfy-Org/ComfyUI_frontend/issues/3654)).
+
+## Subgraph creation
+
+There are three nodes involved in every UE link: 
+- Source (the link sending the data), 
+- Control (the UE node connected to the source), 
+- Target (the node that is receiving the data as an input)
+
+This is how those cases are treated:
+
+|Support|Source|Control|Target||
+|-|-|-|-|-|
+|Yes|Graph|Graph|Graph|Nothing changes|
+|Yes|Graph|Graph|Subgraph|The subgraph will have inputs for the data; in the subgraph the input panel is connected to the Target with a real link|
+|No|Graph|Subgraph|Graph|Not supported|
+|No*|Graph|Subgraph|Subgraph|Not supported|
+|Yes|Subgraph|Graph|Graph|The subgraph will be connected to the Control|
+|No|Subgraph|Graph|Subgraph|Not supported|
+|Yes|Subgraph|Subgraph|Graph|The Source will be connected to the Control *and* the output panel in the subgraph, the output will be connected to the Target with a real link|
+|Yes|Subgraph|Subgraph|Subgraph|All nodes will be connected in the subgraph as they were in the graph|
+
+No* indicates a case that does not work, but might get implemented.
+
+No indicates a case I'm unlikely ever to support
+
+## Reporting a bug well
+
+If you are having problems, the better information you give me, the more chance I can fix it! 
+
+Read the list below and include anything that seems relevant. If you can't get the information, that's ok, it just makes it less likely I'll be able to work out what's going on!
+
+- **describe what you did, what you expected, and what happened**
+- if you have a simple workflow that recreates the problem, that's a huge help
+- Comfy version information (in Settings - About, it looks like this:)
+![versions](https://github.com/user-attachments/assets/a16812a3-94c9-4eb7-8e0e-0d8b5319cd8f)
+- include what all your settings are - in the Comfy settings select `AE` in the left menu
+- check your version of the node (in the settings from version 6.0.4, before that look in the `cg-use-everywhere` folder in the file `__init__.py`
+- press f12 and see if there are any errors in the javascript console that look like they might be relevant
+- look at the server (python) console log and see if there are any errors there

custom_nodes/cg-use-everywhere/__init__.py

@@ -0,0 +1,30 @@
+from .use_everywhere import SeedEverywhere, AnythingEverywherePrompts
+
+UE_VERSION = "7.0.1"
+
+NODE_CLASS_MAPPINGS = { "Seed Everywhere": SeedEverywhere }
+
+from .use_everywhere import AnythingEverywhere, AnythingSomewhere, AnythingEverywhereTriplet, SimpleString
+NODE_CLASS_MAPPINGS["Anything Everywhere"] = AnythingEverywhere
+NODE_CLASS_MAPPINGS["Anything Everywhere3"] = AnythingEverywhereTriplet
+NODE_CLASS_MAPPINGS["Anything Everywhere?"] = AnythingSomewhere
+NODE_CLASS_MAPPINGS["Prompts Everywhere"] = AnythingEverywherePrompts
+NODE_CLASS_MAPPINGS["Simple String"] = SimpleString
+
+import os, shutil
+import folder_paths
+
+# temporary code to remove old javascript installs
+module_js_directory = os.path.join(os.path.dirname(os.path.realpath(__file__)), "js")
+application_root_directory = os.path.dirname(folder_paths.__file__)
+old_code_location = os.path.join(application_root_directory, "web", "extensions", "use_everywhere")
+if os.path.exists(old_code_location):
+    shutil.rmtree(old_code_location)
+
+old_code_location = os.path.join(application_root_directory, "web", "extensions", "cg-nodes", "use_everywhere.js")
+if os.path.exists(old_code_location):
+    os.remove(old_code_location)
+# end of temporary code
+
+WEB_DIRECTORY = "./js"
+__all__ = ["NODE_CLASS_MAPPINGS", "WEB_DIRECTORY"]

custom_nodes/cg-use-everywhere/docs/deprecated.md

@@ -0,0 +1,33 @@
+
+# Deprecated Nodes
+
+This is the old documentation, in case you have a workflow still using the deprecated nodes. 
+
+
+UE nodes are "Use Everywhere". Put a UE node into your workflow, connect its input, and every node with an unconnected input of the same type will act as if connected to it. 
+
+CLIP, IMAGE, MODEL, VAE, CONDITIONING, or LATENT (want something else? Edit `__init__.py` line 3.)
+
+Update: added INT, MASK, and CHECKPOIMNT - which combines MODEL, CLIP, and VAE, and a special node for SEEDs.
+
+| Model, clip, vae, latent and image are all being automagically connected. | Drop this image into ComfyUI to get a working workflow. |
+|-|-|
+|![workflow](./workflow.png)|![portrait](./portrait.png)|
+
+## UE? Nodes
+
+UE? nodes are like UE Nodes, but add two widgets, 'title' and 'input'. These are Regular Expressions, and the node will only send to nodes where the node Title and the unconnected input name match. 
+
+It doesn't need to be a complete match - the logic is `regex.match(name) || regex.match(title)`, so if you want to match the exact name `seed`, you'll need something like `^seed$` as your regex.
+
+Regex 101 - ^ means 'the start', $ means 'the end', '.' matches anything, '.*' matches any number of anything. For more than that, visit [regex101](https://regex101.com/) (the flavour you want is ECMAScript, though that probably won't matter).
+
+| So you can do things like: | Drop this image into ComfyUI to get a working workflow. |
+|-|-|
+|![this](./UEQ.png)|![drop](./UEQportrait.png)|
+
+## Widget?
+
+A UE or UE? node with just one output can have the output converted to a widget. But the combination ones can't. Also note that if you convert it to a widget, you can't then change the title
+
+Why not? because the code gets the data type from the input (weirdly the prompt doesn't contain the data type on outputs), and it's not available if it's a widget, because reasons, so the hack is to get the data type from what comes after `UE ` in the title...