mirror of
https://git.datalinker.icu/comfyanonymous/ComfyUI
synced 2025-12-09 14:04:26 +08:00
Compare commits
7 Commits
fe0d5b9e4c
...
839c78e4e1
| Author | SHA1 | Date | |
|---|---|---|---|
|
839c78e4e1 | ||
|
|
fd271dedfd | ||
|
|
c3c6313fc7 | ||
|
|
85c4b4ae26 | ||
|
|
058f084371 | ||
|
|
ec7f65187d | ||
|
|
aa2ab2415c |
@ -14,6 +14,8 @@
|
|||||||
[![][github-downloads-shield]][github-downloads-link]
|
[![][github-downloads-shield]][github-downloads-link]
|
||||||
[![][github-downloads-latest-shield]][github-downloads-link]
|
[![][github-downloads-latest-shield]][github-downloads-link]
|
||||||
|
|
||||||
|
[中文版 README](README_CN.md)
|
||||||
|
|
||||||
[matrix-shield]: https://img.shields.io/badge/Matrix-000000?style=flat&logo=matrix&logoColor=white
|
[matrix-shield]: https://img.shields.io/badge/Matrix-000000?style=flat&logo=matrix&logoColor=white
|
||||||
[matrix-url]: https://app.element.io/#/room/%23comfyui_space%3Amatrix.org
|
[matrix-url]: https://app.element.io/#/room/%23comfyui_space%3Amatrix.org
|
||||||
[website-shield]: https://img.shields.io/badge/ComfyOrg-4285F4?style=flat
|
[website-shield]: https://img.shields.io/badge/ComfyOrg-4285F4?style=flat
|
||||||
|
|||||||
455
README_CN.md
Normal file
455
README_CN.md
Normal file
@ -0,0 +1,455 @@
|
|||||||
|
<div align="center">
|
||||||
|
|
||||||
|
# ComfyUI
|
||||||
|
**最强大、模块化的可视化 AI 引擎和应用。**
|
||||||
|
|
||||||
|
|
||||||
|
[![Website][website-shield]][website-url]
|
||||||
|
[![Dynamic JSON Badge][discord-shield]][discord-url]
|
||||||
|
[![Twitter][twitter-shield]][twitter-url]
|
||||||
|
[![Matrix][matrix-shield]][matrix-url]
|
||||||
|
<br>
|
||||||
|
[![][github-release-shield]][github-release-link]
|
||||||
|
[![][github-release-date-shield]][github-release-link]
|
||||||
|
[![][github-downloads-shield]][github-downloads-link]
|
||||||
|
[![][github-downloads-latest-shield]][github-downloads-link]
|
||||||
|
|
||||||
|
[matrix-shield]: https://img.shields.io/badge/Matrix-000000?style=flat&logo=matrix&logoColor=white
|
||||||
|
[matrix-url]: https://app.element.io/#/room/%23comfyui_space%3Amatrix.org
|
||||||
|
[website-shield]: https://img.shields.io/badge/ComfyOrg-4285F4?style=flat
|
||||||
|
[website-url]: https://www.comfy.org/
|
||||||
|
<!-- Workaround to display total user from https://github.com/badges/shields/issues/4500#issuecomment-2060079995 -->
|
||||||
|
[discord-shield]: https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fdiscord.com%2Fapi%2Finvites%2Fcomfyorg%3Fwith_counts%3Dtrue&query=%24.approximate_member_count&logo=discord&logoColor=white&label=Discord&color=green&suffix=%20total
|
||||||
|
[discord-url]: https://www.comfy.org/discord
|
||||||
|
[twitter-shield]: https://img.shields.io/twitter/follow/ComfyUI
|
||||||
|
[twitter-url]: https://x.com/ComfyUI
|
||||||
|
|
||||||
|
[github-release-shield]: https://img.shields.io/github/v/release/comfyanonymous/ComfyUI?style=flat&sort=semver
|
||||||
|
[github-release-link]: https://github.com/comfyanonymous/ComfyUI/releases
|
||||||
|
[github-release-date-shield]: https://img.shields.io/github/release-date/comfyanonymous/ComfyUI?style=flat
|
||||||
|
[github-downloads-shield]: https://img.shields.io/github/downloads/comfyanonymous/ComfyUI/total?style=flat
|
||||||
|
[github-downloads-latest-shield]: https://img.shields.io/github/downloads/comfyanonymous/ComfyUI/latest/total?style=flat&label=downloads%40latest
|
||||||
|
[github-downloads-link]: https://github.com/comfyanonymous/ComfyUI/releases
|
||||||
|
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
ComfyUI 让你能够使用基于图形/节点/流程图的界面设计和执行高级的 Stable Diffusion 管道。支持 Windows、Linux 和 macOS。
|
||||||
|
|
||||||
|
## 快速开始
|
||||||
|
|
||||||
|
#### [桌面应用程序](https://www.comfy.org/download)
|
||||||
|
- 最简单的入门方式。
|
||||||
|
- 支持 Windows 和 macOS。
|
||||||
|
|
||||||
|
#### [Windows 便携包](#安装)
|
||||||
|
- 获取最新提交，完全便携。
|
||||||
|
- 支持 Windows。
|
||||||
|
|
||||||
|
#### [手动安装](#手动安装-windows-linux)
|
||||||
|
支持所有操作系统和 GPU 类型（NVIDIA、AMD、Intel、Apple Silicon、Ascend）。
|
||||||
|
|
||||||
|
## [示例](https://comfyanonymous.github.io/ComfyUI_examples/)
|
||||||
|
查看 [示例工作流](https://comfyanonymous.github.io/ComfyUI_examples/) 了解 ComfyUI 能做什么。
|
||||||
|
|
||||||
|
## 特性
|
||||||
|
- 节点/图形/流程图界面，无需编写任何代码即可实验和创建复杂的 Stable Diffusion 工作流。
|
||||||
|
- 图像模型
|
||||||
|
- SD1.x, SD2.x ([unCLIP](https://comfyanonymous.github.io/ComfyUI_examples/unclip/))
|
||||||
|
- [SDXL](https://comfyanonymous.github.io/ComfyUI_examples/sdxl/), [SDXL Turbo](https://comfyanonymous.github.io/ComfyUI_examples/sdturbo/)
|
||||||
|
- [Stable Cascade](https://comfyanonymous.github.io/ComfyUI_examples/stable_cascade/)
|
||||||
|
- [SD3 和 SD3.5](https://comfyanonymous.github.io/ComfyUI_examples/sd3/)
|
||||||
|
- Pixart Alpha 和 Sigma
|
||||||
|
- [AuraFlow](https://comfyanonymous.github.io/ComfyUI_examples/aura_flow/)
|
||||||
|
- [HunyuanDiT](https://comfyanonymous.github.io/ComfyUI_examples/hunyuan_dit/)
|
||||||
|
- [Flux](https://comfyanonymous.github.io/ComfyUI_examples/flux/)
|
||||||
|
- [Lumina Image 2.0](https://comfyanonymous.github.io/ComfyUI_examples/lumina2/)
|
||||||
|
- [HiDream](https://comfyanonymous.github.io/ComfyUI_examples/hidream/)
|
||||||
|
- [Qwen Image](https://comfyanonymous.github.io/ComfyUI_examples/qwen_image/)
|
||||||
|
- [Hunyuan Image 2.1](https://comfyanonymous.github.io/ComfyUI_examples/hunyuan_image/)
|
||||||
|
- [Flux 2](https://comfyanonymous.github.io/ComfyUI_examples/flux2/)
|
||||||
|
- [Z Image](https://comfyanonymous.github.io/ComfyUI_examples/z_image/)
|
||||||
|
- 图像编辑模型
|
||||||
|
- [Omnigen 2](https://comfyanonymous.github.io/ComfyUI_examples/omnigen/)
|
||||||
|
- [Flux Kontext](https://comfyanonymous.github.io/ComfyUI_examples/flux/#flux-kontext-image-editing-model)
|
||||||
|
- [HiDream E1.1](https://comfyanonymous.github.io/ComfyUI_examples/hidream/#hidream-e11)
|
||||||
|
- [Qwen Image Edit](https://comfyanonymous.github.io/ComfyUI_examples/qwen_image/#edit-model)
|
||||||
|
- 视频模型
|
||||||
|
- [Stable Video Diffusion](https://comfyanonymous.github.io/ComfyUI_examples/video/)
|
||||||
|
- [Mochi](https://comfyanonymous.github.io/ComfyUI_examples/mochi/)
|
||||||
|
- [LTX-Video](https://comfyanonymous.github.io/ComfyUI_examples/ltxv/)
|
||||||
|
- [Hunyuan Video](https://comfyanonymous.github.io/ComfyUI_examples/hunyuan_video/)
|
||||||
|
- [Wan 2.1](https://comfyanonymous.github.io/ComfyUI_examples/wan/)
|
||||||
|
- [Wan 2.2](https://comfyanonymous.github.io/ComfyUI_examples/wan22/)
|
||||||
|
- [Hunyuan Video 1.5](https://docs.comfy.org/tutorials/video/hunyuan/hunyuan-video-1-5)
|
||||||
|
- 音频模型
|
||||||
|
- [Stable Audio](https://comfyanonymous.github.io/ComfyUI_examples/audio/)
|
||||||
|
- [ACE Step](https://comfyanonymous.github.io/ComfyUI_examples/audio/)
|
||||||
|
- 3D 模型
|
||||||
|
- [Hunyuan3D 2.0](https://docs.comfy.org/tutorials/3d/hunyuan3D-2)
|
||||||
|
- 异步队列系统
|
||||||
|
- 许多优化：仅重新执行执行之间更改的工作流部分。
|
||||||
|
- 智能内存管理：可以通过智能卸载在低至 1GB vram 的 GPU 上自动运行大型模型。
|
||||||
|
- 即使没有 GPU 也可以工作：```--cpu```（慢）
|
||||||
|
- 可以加载 ckpt 和 safetensors：一体化检查点或独立扩散模型、VAE 和 CLIP 模型。
|
||||||
|
- 安全加载 ckpt、pt、pth 等文件。
|
||||||
|
- Embeddings/Textual inversion
|
||||||
|
- [Loras (regular, locon and loha)](https://comfyanonymous.github.io/ComfyUI_examples/lora/)
|
||||||
|
- [Hypernetworks](https://comfyanonymous.github.io/ComfyUI_examples/hypernetworks/)
|
||||||
|
- 从生成的 PNG、WebP 和 FLAC 文件加载完整的工作流（带有种子）。
|
||||||
|
- 将工作流保存/加载为 Json 文件。
|
||||||
|
- 节点界面可用于创建复杂的工作流，如 [Hires fix](https://comfyanonymous.github.io/ComfyUI_examples/2_pass_txt2img/) 或更高级的工作流。
|
||||||
|
- [区域合成](https://comfyanonymous.github.io/ComfyUI_examples/area_composition/)
|
||||||
|
- [Inpainting](https://comfyanonymous.github.io/ComfyUI_examples/inpaint/) 支持常规和 inpainting 模型。
|
||||||
|
- [ControlNet 和 T2I-Adapter](https://comfyanonymous.github.io/ComfyUI_examples/controlnet/)
|
||||||
|
- [放大模型 (ESRGAN, ESRGAN variants, SwinIR, Swin2SR, etc...)](https://comfyanonymous.github.io/ComfyUI_examples/upscale_models/)
|
||||||
|
- [GLIGEN](https://comfyanonymous.github.io/ComfyUI_examples/gligen/)
|
||||||
|
- [模型合并](https://comfyanonymous.github.io/ComfyUI_examples/model_merging/)
|
||||||
|
- [LCM 模型和 Loras](https://comfyanonymous.github.io/ComfyUI_examples/lcm/)
|
||||||
|
- 使用 [TAESD](#如何显示高质量预览) 进行潜在预览
|
||||||
|
- 完全离线工作：除非你想要，否则核心永远不会下载任何东西。
|
||||||
|
- 可选的 API 节点，通过在线 [Comfy API](https://docs.comfy.org/tutorials/api-nodes/overview) 使用外部提供商的付费模型。
|
||||||
|
- [配置文件](extra_model_paths.yaml.example) 用于设置模型的搜索路径。
|
||||||
|
|
||||||
|
工作流示例可以在 [示例页面](https://comfyanonymous.github.io/ComfyUI_examples/) 找到。
|
||||||
|
|
||||||
|
## 发布流程
|
||||||
|
|
||||||
|
ComfyUI 遵循每周发布周期，目标是周一，但由于模型发布或代码库的重大更改，这经常会发生变化。有三个相互关联的存储库：
|
||||||
|
|
||||||
|
1. **[ComfyUI Core](https://github.com/comfyanonymous/ComfyUI)**
|
||||||
|
- 大约每周发布一个新的稳定版本（例如 v0.7.0）。
|
||||||
|
- 稳定版本标签之外的提交可能非常不稳定，并破坏许多自定义节点。
|
||||||
|
- 作为桌面版本的基础。
|
||||||
|
|
||||||
|
2. **[ComfyUI Desktop](https://github.com/Comfy-Org/desktop)**
|
||||||
|
- 使用最新的稳定核心版本构建新版本。
|
||||||
|
|
||||||
|
3. **[ComfyUI Frontend](https://github.com/Comfy-Org/ComfyUI_frontend)**
|
||||||
|
- 每周前端更新合并到核心存储库中。
|
||||||
|
- 即将发布的核心版本的功能被冻结。
|
||||||
|
- 下一个发布周期的开发继续进行。
|
||||||
|
|
||||||
|
## 快捷键
|
||||||
|
|
||||||
|
| 快捷键 | 说明 |
|
||||||
|
|------------------------------------|--------------------------------------------------------------------------------------------------------------------|
|
||||||
|
| `Ctrl` + `Enter` | 将当前图形排队生成 |
|
||||||
|
| `Ctrl` + `Shift` + `Enter` | 将当前图形作为第一个排队生成 |
|
||||||
|
| `Ctrl` + `Alt` + `Enter` | 取消当前生成 |
|
||||||
|
| `Ctrl` + `Z`/`Ctrl` + `Y` | 撤销/重做 |
|
||||||
|
| `Ctrl` + `S` | 保存工作流 |
|
||||||
|
| `Ctrl` + `O` | 加载工作流 |
|
||||||
|
| `Ctrl` + `A` | 选择所有节点 |
|
||||||
|
| `Alt `+ `C` | 折叠/展开选定的节点 |
|
||||||
|
| `Ctrl` + `M` | 静音/取消静音选定的节点 |
|
||||||
|
| `Ctrl` + `B` | 绕过选定的节点（就像节点从图中移除并且连线重新连接通过一样） |
|
||||||
|
| `Delete`/`Backspace` | 删除选定的节点 |
|
||||||
|
| `Ctrl` + `Backspace` | 删除当前图形 |
|
||||||
|
| `Space` | 按住并移动光标时移动画布 |
|
||||||
|
| `Ctrl`/`Shift` + `Click` | 将点击的节点添加到选择中 |
|
||||||
|
| `Ctrl` + `C`/`Ctrl` + `V` | 复制并粘贴选定的节点（不保持与未选定节点输出的连接） |
|
||||||
|
| `Ctrl` + `C`/`Ctrl` + `Shift` + `V` | 复制并粘贴选定的节点（保持从未选定节点的输出到粘贴节点的输入的连接） |
|
||||||
|
| `Shift` + `Drag` | 同时移动多个选定的节点 |
|
||||||
|
| `Ctrl` + `D` | 加载默认图形 |
|
||||||
|
| `Alt` + `+` | 画布放大 |
|
||||||
|
| `Alt` + `-` | 画布缩小 |
|
||||||
|
| `Ctrl` + `Shift` + LMB + 垂直拖动 | 画布放大/缩小 |
|
||||||
|
| `P` | 固定/取消固定选定的节点 |
|
||||||
|
| `Ctrl` + `G` | 组合选定的节点 |
|
||||||
|
| `Q` | 切换队列的可见性 |
|
||||||
|
| `H` | 切换历史记录的可见性 |
|
||||||
|
| `R` | 刷新图形 |
|
||||||
|
| `F` | 显示/隐藏菜单 |
|
||||||
|
| `.` | 适应视图到选择（未选择任何内容时适应整个图形） |
|
||||||
|
| 双击 LMB | 打开节点快速搜索面板 |
|
||||||
|
| `Shift` + 拖动 | 一次移动多条连线 |
|
||||||
|
| `Ctrl` + `Alt` + LMB | 断开点击插槽的所有连线 |
|
||||||
|
|
||||||
|
macOS 用户可以将 `Ctrl` 替换为 `Cmd`
|
||||||
|
|
||||||
|
# 安装
|
||||||
|
|
||||||
|
## Windows 便携版
|
||||||
|
|
||||||
|
在 [发布页面](https://github.com/comfyanonymous/ComfyUI/releases) 上有一个适用于 Windows 的便携式独立构建，应该可以在 Nvidia GPU 上运行，或者仅在 CPU 上运行。
|
||||||
|
|
||||||
|
### [直接下载链接](https://github.com/comfyanonymous/ComfyUI/releases/latest/download/ComfyUI_windows_portable_nvidia.7z)
|
||||||
|
|
||||||
|
只需下载，使用 [7-Zip](https://7-zip.org) 解压，或者在最近的 Windows 版本上使用资源管理器解压并运行。对于较小的模型，通常只需要将 checkpoints（巨大的 ckpt/safetensors 文件）放在：ComfyUI\models\checkpoints 中，但许多较大的模型有多个文件。请务必按照说明了解将它们放在 ComfyUI\models\ 的哪个子文件夹中。
|
||||||
|
|
||||||
|
如果解压有问题，请右键单击文件 -> 属性 -> 解除锁定
|
||||||
|
|
||||||
|
如果无法启动，请更新您的 Nvidia 驱动程序。
|
||||||
|
|
||||||
|
#### 替代下载：
|
||||||
|
|
||||||
|
[适用于 AMD GPU 的实验性便携版](https://github.com/comfyanonymous/ComfyUI/releases/latest/download/ComfyUI_windows_portable_amd.7z)
|
||||||
|
|
||||||
|
[带有 pytorch cuda 12.8 和 python 3.12 的便携版](https://github.com/comfyanonymous/ComfyUI/releases/latest/download/ComfyUI_windows_portable_nvidia_cu128.7z)。
|
||||||
|
|
||||||
|
[带有 pytorch cuda 12.6 和 python 3.12 的便携版](https://github.com/comfyanonymous/ComfyUI/releases/latest/download/ComfyUI_windows_portable_nvidia_cu126.7z)（支持 Nvidia 10 系列及更旧的 GPU）。
|
||||||
|
|
||||||
|
#### 如何在另一个 UI 和 ComfyUI 之间共享模型？
|
||||||
|
|
||||||
|
请参阅 [配置文件](extra_model_paths.yaml.example) 以设置模型的搜索路径。在独立的 Windows 构建中，您可以在 ComfyUI 目录中找到此文件。将此文件重命名为 extra_model_paths.yaml 并使用您喜欢的文本编辑器进行编辑。
|
||||||
|
|
||||||
|
|
||||||
|
## [comfy-cli](https://docs.comfy.org/comfy-cli/getting-started)
|
||||||
|
|
||||||
|
您可以使用 comfy-cli 安装并启动 ComfyUI：
|
||||||
|
```bash
|
||||||
|
pip install comfy-cli
|
||||||
|
comfy install
|
||||||
|
```
|
||||||
|
|
||||||
|
## 手动安装 (Windows, Linux)
|
||||||
|
|
||||||
|
Python 3.14 可以工作，但您可能会遇到 torch 编译节点的问题。自由线程变体仍然缺少一些依赖项。
|
||||||
|
|
||||||
|
Python 3.13 支持得很好。如果您在 3.13 上遇到某些自定义节点依赖项的问题，可以尝试 3.12。
|
||||||
|
|
||||||
|
### 说明：
|
||||||
|
|
||||||
|
Git clone 此仓库。
|
||||||
|
|
||||||
|
将您的 SD checkpoints（巨大的 ckpt/safetensors 文件）放在：models/checkpoints
|
||||||
|
|
||||||
|
将您的 VAE 放在：models/vae
|
||||||
|
|
||||||
|
|
||||||
|
### AMD GPU (Linux)
|
||||||
|
|
||||||
|
AMD 用户如果尚未安装，可以使用 pip 安装 rocm 和 pytorch，这是安装稳定版本的命令：
|
||||||
|
|
||||||
|
```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.4```
|
||||||
|
|
||||||
|
这是安装带有 ROCm 7.0 的 nightly 版本的命令，可能会有一些性能改进：
|
||||||
|
|
||||||
|
```pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm7.1```
|
||||||
|
|
||||||
|
|
||||||
|
### AMD GPU（实验性：Windows 和 Linux），仅限 RDNA 3, 3.5 和 4。
|
||||||
|
|
||||||
|
这些硬件支持比上面的构建少，但它们可以在 Windows 上运行。您还需要安装特定于您的硬件的 pytorch 版本。
|
||||||
|
|
||||||
|
RDNA 3 (RX 7000 系列):
|
||||||
|
|
||||||
|
```pip install --pre torch torchvision torchaudio --index-url https://rocm.nightlies.amd.com/v2/gfx110X-dgpu/```
|
||||||
|
|
||||||
|
RDNA 3.5 (Strix halo/Ryzen AI Max+ 365):
|
||||||
|
|
||||||
|
```pip install --pre torch torchvision torchaudio --index-url https://rocm.nightlies.amd.com/v2/gfx1151/```
|
||||||
|
|
||||||
|
RDNA 4 (RX 9000 系列):
|
||||||
|
|
||||||
|
```pip install --pre torch torchvision torchaudio --index-url https://rocm.nightlies.amd.com/v2/gfx120X-all/```
|
||||||
|
|
||||||
|
### Intel GPU (Windows 和 Linux)
|
||||||
|
|
||||||
|
Intel Arc GPU 用户可以使用 pip 安装具有 torch.xpu 支持的原生 PyTorch。更多信息可以在 [这里](https://pytorch.org/docs/main/notes/get_start_xpu.html) 找到。
|
||||||
|
|
||||||
|
1. 要安装 PyTorch xpu，请使用以下命令：
|
||||||
|
|
||||||
|
```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/xpu```
|
||||||
|
|
||||||
|
这是安装 Pytorch xpu nightly 的命令，可能会有一些性能改进：
|
||||||
|
|
||||||
|
```pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/xpu```
|
||||||
|
|
||||||
|
### NVIDIA
|
||||||
|
|
||||||
|
Nvidia 用户应使用此命令安装稳定的 pytorch：
|
||||||
|
|
||||||
|
```pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130```
|
||||||
|
|
||||||
|
这是安装 pytorch nightly 的命令，可能会有性能改进。
|
||||||
|
|
||||||
|
```pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu130```
|
||||||
|
|
||||||
|
#### 故障排除
|
||||||
|
|
||||||
|
如果您收到 "Torch not compiled with CUDA enabled" 错误，请使用以下命令卸载 torch：
|
||||||
|
|
||||||
|
```pip uninstall torch```
|
||||||
|
|
||||||
|
并再次使用上面的命令安装它。
|
||||||
|
|
||||||
|
### 依赖项
|
||||||
|
|
||||||
|
通过在 ComfyUI 文件夹中打开终端并运行以下命令来安装依赖项：
|
||||||
|
|
||||||
|
```pip install -r requirements.txt```
|
||||||
|
|
||||||
|
在此之后，您应该已经安装了所有内容，并可以继续运行 ComfyUI。
|
||||||
|
|
||||||
|
### 其他：
|
||||||
|
|
||||||
|
#### Apple Mac silicon
|
||||||
|
|
||||||
|
您可以在任何最近的 macOS 版本上在 Apple Mac silicon (M1 或 M2) 上安装 ComfyUI。
|
||||||
|
|
||||||
|
1. 安装 pytorch nightly。有关说明，请阅读 [Mac 上的加速 PyTorch 训练](https://developer.apple.com/metal/pytorch/) Apple 开发者指南（确保安装最新的 pytorch nightly）。
|
||||||
|
1. 按照 Windows 和 Linux 的 [ComfyUI 手动安装](#手动安装-windows-linux) 说明进行操作。
|
||||||
|
1. 安装 ComfyUI [依赖项](#依赖项)。如果您安装了另一个 Stable Diffusion UI，[您可能能够重用依赖项](#我已经安装了另一个-stable-diffusion-ui-我真的必须安装所有这些依赖项吗)。
|
||||||
|
1. 通过运行 `python main.py` 启动 ComfyUI
|
||||||
|
|
||||||
|
> **注意**：请记住将您的模型、VAE、LoRA 等添加到相应的 Comfy 文件夹中，如 [ComfyUI 手动安装](#手动安装-windows-linux) 中所述。
|
||||||
|
|
||||||
|
#### Ascend NPU
|
||||||
|
|
||||||
|
对于与 Ascend Extension for PyTorch (torch_npu) 兼容的模型。首先，确保您的环境满足 [安装](https://ascend.github.io/docs/sources/ascend/quick_install.html) 页面上列出的先决条件。这是针对您的平台和安装方法的逐步指南：
|
||||||
|
|
||||||
|
1. 如果需要，首先安装 torch-npu 安装页面中指定的推荐或更新的 Linux 内核版本。
|
||||||
|
2. 按照针对您特定平台的说明，继续安装 Ascend Basekit，其中包括驱动程序、固件和 CANN。
|
||||||
|
3. 接下来，按照 [安装](https://ascend.github.io/docs/sources/pytorch/install.html#pytorch) 页面上的特定于平台的说明安装 torch-npu 的必要包。
|
||||||
|
4. 最后，遵循 Linux 的 [ComfyUI 手动安装](#手动安装-windows-linux) 指南。安装所有组件后，您可以如前所述运行 ComfyUI。
|
||||||
|
|
||||||
|
#### Cambricon MLU
|
||||||
|
|
||||||
|
对于与 Cambricon Extension for PyTorch (torch_mlu) 兼容的模型。这是针对您的平台和安装方法的逐步指南：
|
||||||
|
|
||||||
|
1. 按照 [安装](https://www.cambricon.com/docs/sdk_1.15.0/cntoolkit_3.7.2/cntoolkit_install_3.7.2/index.html) 上的特定于平台的说明安装 Cambricon CNToolkit
|
||||||
|
2. 接下来，按照 [安装](https://www.cambricon.com/docs/sdk_1.15.0/cambricon_pytorch_1.17.0/user_guide_1.9/index.html) 上的说明安装 PyTorch(torch_mlu)
|
||||||
|
3. 通过运行 `python main.py` 启动 ComfyUI
|
||||||
|
|
||||||
|
#### Iluvatar Corex
|
||||||
|
|
||||||
|
对于与 Iluvatar Extension for PyTorch 兼容的模型。这是针对您的平台和安装方法的逐步指南：
|
||||||
|
|
||||||
|
1. 按照 [安装](https://support.iluvatar.com/#/DocumentCentre?id=1&nameCenter=2&productId=520117912052801536) 上的特定于平台的说明安装 Iluvatar Corex Toolkit
|
||||||
|
2. 通过运行 `python main.py` 启动 ComfyUI
|
||||||
|
|
||||||
|
|
||||||
|
## [ComfyUI-Manager](https://github.com/Comfy-Org/ComfyUI-Manager/tree/manager-v4)
|
||||||
|
|
||||||
|
**ComfyUI-Manager** 是一个扩展，允许您轻松安装、更新和管理 ComfyUI 的自定义节点。
|
||||||
|
|
||||||
|
### 设置
|
||||||
|
|
||||||
|
1. 安装管理器依赖项：
|
||||||
|
```bash
|
||||||
|
pip install -r manager_requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
2. 运行 ComfyUI 时使用 `--enable-manager` 标志启用管理器：
|
||||||
|
```bash
|
||||||
|
python main.py --enable-manager
|
||||||
|
```
|
||||||
|
|
||||||
|
### 命令行选项
|
||||||
|
|
||||||
|
| 标志 | 描述 |
|
||||||
|
|------|-------------|
|
||||||
|
| `--enable-manager` | 启用 ComfyUI-Manager |
|
||||||
|
| `--enable-manager-legacy-ui` | 使用旧版管理器 UI 而不是新 UI（需要 `--enable-manager`） |
|
||||||
|
| `--disable-manager-ui` | 禁用管理器 UI 和端点，同时保留后台功能，如安全检查和计划安装完成（需要 `--enable-manager`） |
|
||||||
|
|
||||||
|
|
||||||
|
# 运行
|
||||||
|
|
||||||
|
```python main.py```
|
||||||
|
|
||||||
|
### 对于 ROCm 未正式支持的 AMD 卡
|
||||||
|
|
||||||
|
如果有问题，请尝试使用此命令运行它：
|
||||||
|
|
||||||
|
对于 6700, 6600 和可能的其他 RDNA2 或更旧的卡：```HSA_OVERRIDE_GFX_VERSION=10.3.0 python main.py```
|
||||||
|
|
||||||
|
对于 AMD 7600 和可能的其他 RDNA3 卡：```HSA_OVERRIDE_GFX_VERSION=11.0.0 python main.py```
|
||||||
|
|
||||||
|
### AMD ROCm 提示
|
||||||
|
|
||||||
|
您可以使用此命令在某些 AMD GPU 上的 ComfyUI 中启用最近 pytorch 的实验性内存高效注意力，它应该已经在 RDNA3 上默认启用。如果这提高了您 GPU 上最新 pytorch 的速度，请报告它，以便我可以默认启用它。
|
||||||
|
|
||||||
|
```TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 python main.py --use-pytorch-cross-attention```
|
||||||
|
|
||||||
|
您也可以尝试设置此环境变量 `PYTORCH_TUNABLEOP_ENABLED=1`，这可能会加快速度，但代价是非常慢的初始运行。
|
||||||
|
|
||||||
|
# 注意事项
|
||||||
|
|
||||||
|
只有具有所有正确输入的输出的图形部分才会被执行。
|
||||||
|
|
||||||
|
只有从每次执行到下一次执行发生更改的图形部分才会被执行，如果您提交相同的图形两次，只有第一次会被执行。如果您更改图形的最后一部分，只有您更改的部分和依赖于它的部分会被执行。
|
||||||
|
|
||||||
|
将生成的 png 拖到网页上或加载它将为您提供完整的工作流，包括用于创建它的种子。
|
||||||
|
|
||||||
|
您可以使用 () 来更改单词或短语的强调，例如：(good code:1.2) 或 (bad code:0.8)。() 的默认强调是 1.1。要在实际提示中使用 () 字符，请像 \\( 或 \\) 一样转义它们。
|
||||||
|
|
||||||
|
您可以使用 {day|night} 进行通配符/动态提示。使用此语法 "{wild|card|test}" 将在每次您排队提示时由前端随机替换为 "wild"、"card" 或 "test"。要在实际提示中使用 {} 字符，请像：\\{ or \\} 一样转义它们。
|
||||||
|
|
||||||
|
动态提示还支持 C 风格的注释，如 `// comment` 或 `/* comment */`。
|
||||||
|
|
||||||
|
要在文本提示中使用文本反转概念/嵌入，请将它们放在 models/embeddings 目录中，并在 CLIPTextEncode 节点中像这样使用它们（您可以省略 .pt 扩展名）：
|
||||||
|
|
||||||
|
```embedding:embedding_filename.pt```
|
||||||
|
|
||||||
|
|
||||||
|
## 如何显示高质量预览？
|
||||||
|
|
||||||
|
使用 ```--preview-method auto``` 启用预览。
|
||||||
|
|
||||||
|
默认安装包括一种快速的潜在预览方法，分辨率较低。要使用 [TAESD](https://github.com/madebyollin/taesd) 启用更高质量的预览，请下载 [taesd_decoder.pth, taesdxl_decoder.pth, taesd3_decoder.pth 和 taef1_decoder.pth](https://github.com/madebyollin/taesd/) 并将它们放在 `models/vae_approx` 文件夹中。安装完成后，重新启动 ComfyUI 并使用 `--preview-method taesd` 启动它以启用高质量预览。
|
||||||
|
|
||||||
|
## 如何使用 TLS/SSL？
|
||||||
|
通过运行以下命令生成自签名证书（不适合共享/生产使用）和密钥：`openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes -subj "/C=XX/ST=StateName/L=CityName/O=CompanyName/OU=CompanySectionName/CN=CommonNameOrHostname"`
|
||||||
|
|
||||||
|
使用 `--tls-keyfile key.pem --tls-certfile cert.pem` 启用 TLS/SSL，应用程序现在将可以通过 `https://...` 而不是 `http://...` 访问。
|
||||||
|
|
||||||
|
> 注意：Windows 用户可以使用 [alexisrolland/docker-openssl](https://github.com/alexisrolland/docker-openssl) 或 [第三方二进制发行版](https://wiki.openssl.org/index.php/Binaries) 之一来运行上面的命令示例。
|
||||||
|
<br/><br/>如果您使用容器，请注意卷挂载 `-v` 可以是相对路径，因此 `... -v ".\:/openssl-certs" ...` 将在命令提示符或 powershell 终端的当前目录中创建密钥和证书文件。
|
||||||
|
|
||||||
|
## 支持和开发频道
|
||||||
|
|
||||||
|
[Discord](https://comfy.org/discord): 尝试 #help 或 #feedback 频道。
|
||||||
|
|
||||||
|
[Matrix space: #comfyui_space:matrix.org](https://app.element.io/#/room/%23comfyui_space%3Amatrix.org) (它像 discord 但是开源的)。
|
||||||
|
|
||||||
|
另请参阅：[https://www.comfy.org/](https://www.comfy.org/)
|
||||||
|
|
||||||
|
## 前端开发
|
||||||
|
|
||||||
|
截至 2024 年 8 月 15 日，我们已过渡到新的前端，该前端现在托管在一个单独的存储库中：[ComfyUI Frontend](https://github.com/Comfy-Org/ComfyUI_frontend)。此存储库现在在 `web/` 目录下托管编译后的 JS（来自 TS/Vue）。
|
||||||
|
|
||||||
|
### 报告问题和请求功能
|
||||||
|
|
||||||
|
对于与前端相关的任何错误、问题或功能请求，请使用 [ComfyUI Frontend 存储库](https://github.com/Comfy-Org/ComfyUI_frontend)。这将帮助我们更有效地管理和解决前端特定的问题。
|
||||||
|
|
||||||
|
### 使用最新前端
|
||||||
|
|
||||||
|
新前端现在是 ComfyUI 的默认前端。但是，请注意：
|
||||||
|
|
||||||
|
1. 主 ComfyUI 存储库中的前端每两周更新一次。
|
||||||
|
2. 每日发布在单独的前端存储库中可用。
|
||||||
|
|
||||||
|
要使用最新的前端版本：
|
||||||
|
|
||||||
|
1. 对于最新的每日发布，使用此命令行参数启动 ComfyUI：
|
||||||
|
|
||||||
|
```
|
||||||
|
--front-end-version Comfy-Org/ComfyUI_frontend@latest
|
||||||
|
```
|
||||||
|
|
||||||
|
2. 对于特定版本，将 `latest` 替换为所需的版本号：
|
||||||
|
|
||||||
|
```
|
||||||
|
--front-end-version Comfy-Org/ComfyUI_frontend@1.2.2
|
||||||
|
```
|
||||||
|
|
||||||
|
这种方法允许您轻松地在稳定的每两周发布和前沿的每日更新之间切换，甚至可以切换到特定版本进行测试。
|
||||||
|
|
||||||
|
### 访问旧版前端
|
||||||
|
|
||||||
|
如果您因任何原因需要使用旧版前端，可以使用以下命令行参数访问它：
|
||||||
|
|
||||||
|
```
|
||||||
|
--front-end-version Comfy-Org/ComfyUI_legacy_frontend@latest
|
||||||
|
```
|
||||||
|
|
||||||
|
这将使用保存在 [ComfyUI Legacy Frontend 存储库](https://github.com/Comfy-Org/ComfyUI_legacy_frontend) 中的旧版前端快照。
|
||||||
|
|
||||||
|
# QA
|
||||||
|
|
||||||
|
### 我应该为此购买哪种 GPU？
|
||||||
|
|
||||||
|
[请参阅此页面以获取一些建议](https://github.com/comfyanonymous/ComfyUI/wiki/Which-GPU-should-I-buy-for-ComfyUI)
|
||||||
@ -5,9 +5,9 @@ from typing import Type, TYPE_CHECKING
|
|||||||
from comfy_api.internal import ComfyAPIBase
|
from comfy_api.internal import ComfyAPIBase
|
||||||
from comfy_api.internal.singleton import ProxiedSingleton
|
from comfy_api.internal.singleton import ProxiedSingleton
|
||||||
from comfy_api.internal.async_to_sync import create_sync_class
|
from comfy_api.internal.async_to_sync import create_sync_class
|
||||||
from comfy_api.latest._input import ImageInput, AudioInput, MaskInput, LatentInput, VideoInput
|
from ._input import ImageInput, AudioInput, MaskInput, LatentInput, VideoInput
|
||||||
from comfy_api.latest._input_impl import VideoFromFile, VideoFromComponents
|
from ._input_impl import VideoFromFile, VideoFromComponents
|
||||||
from comfy_api.latest._util import VideoCodec, VideoContainer, VideoComponents, MESH, VOXEL
|
from ._util import VideoCodec, VideoContainer, VideoComponents, MESH, VOXEL
|
||||||
from . import _io_public as io
|
from . import _io_public as io
|
||||||
from . import _ui_public as ui
|
from . import _ui_public as ui
|
||||||
# from comfy_api.latest._resources import _RESOURCES as resources #noqa: F401
|
# from comfy_api.latest._resources import _RESOURCES as resources #noqa: F401
|
||||||
@ -80,7 +80,7 @@ class ComfyExtension(ABC):
|
|||||||
async def on_load(self) -> None:
|
async def on_load(self) -> None:
|
||||||
"""
|
"""
|
||||||
Called when an extension is loaded.
|
Called when an extension is loaded.
|
||||||
This should be used to initialize any global resources neeeded by the extension.
|
This should be used to initialize any global resources needed by the extension.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
|
|||||||
@ -4,7 +4,7 @@ from fractions import Fraction
|
|||||||
from typing import Optional, Union, IO
|
from typing import Optional, Union, IO
|
||||||
import io
|
import io
|
||||||
import av
|
import av
|
||||||
from comfy_api.util import VideoContainer, VideoCodec, VideoComponents
|
from .._util import VideoContainer, VideoCodec, VideoComponents
|
||||||
|
|
||||||
class VideoInput(ABC):
|
class VideoInput(ABC):
|
||||||
"""
|
"""
|
||||||
|
|||||||
@ -3,14 +3,14 @@ from av.container import InputContainer
|
|||||||
from av.subtitles.stream import SubtitleStream
|
from av.subtitles.stream import SubtitleStream
|
||||||
from fractions import Fraction
|
from fractions import Fraction
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
from comfy_api.latest._input import AudioInput, VideoInput
|
from .._input import AudioInput, VideoInput
|
||||||
import av
|
import av
|
||||||
import io
|
import io
|
||||||
import json
|
import json
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import math
|
import math
|
||||||
import torch
|
import torch
|
||||||
from comfy_api.latest._util import VideoContainer, VideoCodec, VideoComponents
|
from .._util import VideoContainer, VideoCodec, VideoComponents
|
||||||
|
|
||||||
|
|
||||||
def container_to_output_format(container_format: str | None) -> str | None:
|
def container_to_output_format(container_format: str | None) -> str | None:
|
||||||
|
|||||||
@ -26,7 +26,7 @@ if TYPE_CHECKING:
|
|||||||
from comfy_api.input import VideoInput
|
from comfy_api.input import VideoInput
|
||||||
from comfy_api.internal import (_ComfyNodeInternal, _NodeOutputInternal, classproperty, copy_class, first_real_override, is_class,
|
from comfy_api.internal import (_ComfyNodeInternal, _NodeOutputInternal, classproperty, copy_class, first_real_override, is_class,
|
||||||
prune_dict, shallow_clone_class)
|
prune_dict, shallow_clone_class)
|
||||||
from comfy_api.latest._resources import Resources, ResourcesLocal
|
from ._resources import Resources, ResourcesLocal
|
||||||
from comfy_execution.graph_utils import ExecutionBlocker
|
from comfy_execution.graph_utils import ExecutionBlocker
|
||||||
from ._util import MESH, VOXEL
|
from ._util import MESH, VOXEL
|
||||||
|
|
||||||
|
|||||||
@ -22,7 +22,7 @@ import folder_paths
|
|||||||
|
|
||||||
# used for image preview
|
# used for image preview
|
||||||
from comfy.cli_args import args
|
from comfy.cli_args import args
|
||||||
from comfy_api.latest._io import ComfyNode, FolderType, Image, _UIOutput
|
from ._io import ComfyNode, FolderType, Image, _UIOutput
|
||||||
|
|
||||||
|
|
||||||
class SavedResult(dict):
|
class SavedResult(dict):
|
||||||
|
|||||||
@ -3,7 +3,7 @@ from dataclasses import dataclass
|
|||||||
from enum import Enum
|
from enum import Enum
|
||||||
from fractions import Fraction
|
from fractions import Fraction
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
from comfy_api.latest._input import ImageInput, AudioInput
|
from .._input import ImageInput, AudioInput
|
||||||
|
|
||||||
class VideoCodec(str, Enum):
|
class VideoCodec(str, Enum):
|
||||||
AUTO = "auto"
|
AUTO = "auto"
|
||||||
|
|||||||
144
comfy_api_nodes/apis/bytedance_api.py
Normal file
144
comfy_api_nodes/apis/bytedance_api.py
Normal file
@ -0,0 +1,144 @@
|
|||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
|
||||||
|
class Text2ImageTaskCreationRequest(BaseModel):
|
||||||
|
model: str = Field(...)
|
||||||
|
prompt: str = Field(...)
|
||||||
|
response_format: str | None = Field("url")
|
||||||
|
size: str | None = Field(None)
|
||||||
|
seed: int | None = Field(0, ge=0, le=2147483647)
|
||||||
|
guidance_scale: float | None = Field(..., ge=1.0, le=10.0)
|
||||||
|
watermark: bool | None = Field(True)
|
||||||
|
|
||||||
|
|
||||||
|
class Image2ImageTaskCreationRequest(BaseModel):
|
||||||
|
model: str = Field(...)
|
||||||
|
prompt: str = Field(...)
|
||||||
|
response_format: str | None = Field("url")
|
||||||
|
image: str = Field(..., description="Base64 encoded string or image URL")
|
||||||
|
size: str | None = Field("adaptive")
|
||||||
|
seed: int | None = Field(..., ge=0, le=2147483647)
|
||||||
|
guidance_scale: float | None = Field(..., ge=1.0, le=10.0)
|
||||||
|
watermark: bool | None = Field(True)
|
||||||
|
|
||||||
|
|
||||||
|
class Seedream4Options(BaseModel):
|
||||||
|
max_images: int = Field(15)
|
||||||
|
|
||||||
|
|
||||||
|
class Seedream4TaskCreationRequest(BaseModel):
|
||||||
|
model: str = Field(...)
|
||||||
|
prompt: str = Field(...)
|
||||||
|
response_format: str = Field("url")
|
||||||
|
image: list[str] | None = Field(None, description="Image URLs")
|
||||||
|
size: str = Field(...)
|
||||||
|
seed: int = Field(..., ge=0, le=2147483647)
|
||||||
|
sequential_image_generation: str = Field("disabled")
|
||||||
|
sequential_image_generation_options: Seedream4Options = Field(Seedream4Options(max_images=15))
|
||||||
|
watermark: bool = Field(True)
|
||||||
|
|
||||||
|
|
||||||
|
class ImageTaskCreationResponse(BaseModel):
|
||||||
|
model: str = Field(...)
|
||||||
|
created: int = Field(..., description="Unix timestamp (in seconds) indicating time when the request was created.")
|
||||||
|
data: list = Field([], description="Contains information about the generated image(s).")
|
||||||
|
error: dict = Field({}, description="Contains `code` and `message` fields in case of error.")
|
||||||
|
|
||||||
|
|
||||||
|
class TaskTextContent(BaseModel):
|
||||||
|
type: str = Field("text")
|
||||||
|
text: str = Field(...)
|
||||||
|
|
||||||
|
|
||||||
|
class TaskImageContentUrl(BaseModel):
|
||||||
|
url: str = Field(...)
|
||||||
|
|
||||||
|
|
||||||
|
class TaskImageContent(BaseModel):
|
||||||
|
type: str = Field("image_url")
|
||||||
|
image_url: TaskImageContentUrl = Field(...)
|
||||||
|
role: Literal["first_frame", "last_frame", "reference_image"] | None = Field(None)
|
||||||
|
|
||||||
|
|
||||||
|
class Text2VideoTaskCreationRequest(BaseModel):
|
||||||
|
model: str = Field(...)
|
||||||
|
content: list[TaskTextContent] = Field(..., min_length=1)
|
||||||
|
|
||||||
|
|
||||||
|
class Image2VideoTaskCreationRequest(BaseModel):
|
||||||
|
model: str = Field(...)
|
||||||
|
content: list[TaskTextContent | TaskImageContent] = Field(..., min_length=2)
|
||||||
|
|
||||||
|
|
||||||
|
class TaskCreationResponse(BaseModel):
|
||||||
|
id: str = Field(...)
|
||||||
|
|
||||||
|
|
||||||
|
class TaskStatusError(BaseModel):
|
||||||
|
code: str = Field(...)
|
||||||
|
message: str = Field(...)
|
||||||
|
|
||||||
|
|
||||||
|
class TaskStatusResult(BaseModel):
|
||||||
|
video_url: str = Field(...)
|
||||||
|
|
||||||
|
|
||||||
|
class TaskStatusResponse(BaseModel):
|
||||||
|
id: str = Field(...)
|
||||||
|
model: str = Field(...)
|
||||||
|
status: Literal["queued", "running", "cancelled", "succeeded", "failed"] = Field(...)
|
||||||
|
error: TaskStatusError | None = Field(None)
|
||||||
|
content: TaskStatusResult | None = Field(None)
|
||||||
|
|
||||||
|
|
||||||
|
RECOMMENDED_PRESETS = [
|
||||||
|
("1024x1024 (1:1)", 1024, 1024),
|
||||||
|
("864x1152 (3:4)", 864, 1152),
|
||||||
|
("1152x864 (4:3)", 1152, 864),
|
||||||
|
("1280x720 (16:9)", 1280, 720),
|
||||||
|
("720x1280 (9:16)", 720, 1280),
|
||||||
|
("832x1248 (2:3)", 832, 1248),
|
||||||
|
("1248x832 (3:2)", 1248, 832),
|
||||||
|
("1512x648 (21:9)", 1512, 648),
|
||||||
|
("2048x2048 (1:1)", 2048, 2048),
|
||||||
|
("Custom", None, None),
|
||||||
|
]
|
||||||
|
|
||||||
|
RECOMMENDED_PRESETS_SEEDREAM_4 = [
|
||||||
|
("2048x2048 (1:1)", 2048, 2048),
|
||||||
|
("2304x1728 (4:3)", 2304, 1728),
|
||||||
|
("1728x2304 (3:4)", 1728, 2304),
|
||||||
|
("2560x1440 (16:9)", 2560, 1440),
|
||||||
|
("1440x2560 (9:16)", 1440, 2560),
|
||||||
|
("2496x1664 (3:2)", 2496, 1664),
|
||||||
|
("1664x2496 (2:3)", 1664, 2496),
|
||||||
|
("3024x1296 (21:9)", 3024, 1296),
|
||||||
|
("4096x4096 (1:1)", 4096, 4096),
|
||||||
|
("Custom", None, None),
|
||||||
|
]
|
||||||
|
|
||||||
|
# The time in this dictionary are given for 10 seconds duration.
|
||||||
|
VIDEO_TASKS_EXECUTION_TIME = {
|
||||||
|
"seedance-1-0-lite-t2v-250428": {
|
||||||
|
"480p": 40,
|
||||||
|
"720p": 60,
|
||||||
|
"1080p": 90,
|
||||||
|
},
|
||||||
|
"seedance-1-0-lite-i2v-250428": {
|
||||||
|
"480p": 40,
|
||||||
|
"720p": 60,
|
||||||
|
"1080p": 90,
|
||||||
|
},
|
||||||
|
"seedance-1-0-pro-250528": {
|
||||||
|
"480p": 70,
|
||||||
|
"720p": 85,
|
||||||
|
"1080p": 115,
|
||||||
|
},
|
||||||
|
"seedance-1-0-pro-fast-251015": {
|
||||||
|
"480p": 50,
|
||||||
|
"720p": 65,
|
||||||
|
"1080p": 100,
|
||||||
|
},
|
||||||
|
}
|
||||||
@ -84,15 +84,7 @@ class GeminiSystemInstructionContent(BaseModel):
|
|||||||
description="A list of ordered parts that make up a single message. "
|
description="A list of ordered parts that make up a single message. "
|
||||||
"Different parts may have different IANA MIME types.",
|
"Different parts may have different IANA MIME types.",
|
||||||
)
|
)
|
||||||
role: GeminiRole = Field(
|
role: GeminiRole | None = Field(..., description="The role field of systemInstruction may be ignored.")
|
||||||
...,
|
|
||||||
description="The identity of the entity that creates the message. "
|
|
||||||
"The following values are supported: "
|
|
||||||
"user: This indicates that the message is sent by a real person, typically a user-generated message. "
|
|
||||||
"model: This indicates that the message is generated by the model. "
|
|
||||||
"The model value is used to insert messages from model into the conversation during multi-turn conversations. "
|
|
||||||
"For non-multi-turn conversations, this field can be left blank or unset.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class GeminiFunctionDeclaration(BaseModel):
|
class GeminiFunctionDeclaration(BaseModel):
|
||||||
|
|||||||
@ -85,7 +85,7 @@ class Response1(BaseModel):
|
|||||||
raiMediaFilteredReasons: Optional[list[str]] = Field(
|
raiMediaFilteredReasons: Optional[list[str]] = Field(
|
||||||
None, description='Reasons why media was filtered by responsible AI policies'
|
None, description='Reasons why media was filtered by responsible AI policies'
|
||||||
)
|
)
|
||||||
videos: Optional[list[Video]] = None
|
videos: Optional[list[Video]] = Field(None)
|
||||||
|
|
||||||
|
|
||||||
class VeoGenVidPollResponse(BaseModel):
|
class VeoGenVidPollResponse(BaseModel):
|
||||||
|
|||||||
@ -1,13 +1,27 @@
|
|||||||
import logging
|
import logging
|
||||||
import math
|
import math
|
||||||
from enum import Enum
|
|
||||||
from typing import Literal, Optional, Union
|
|
||||||
|
|
||||||
import torch
|
import torch
|
||||||
from pydantic import BaseModel, Field
|
|
||||||
from typing_extensions import override
|
from typing_extensions import override
|
||||||
|
|
||||||
from comfy_api.latest import IO, ComfyExtension
|
from comfy_api.latest import IO, ComfyExtension, Input
|
||||||
|
from comfy_api_nodes.apis.bytedance_api import (
|
||||||
|
RECOMMENDED_PRESETS,
|
||||||
|
RECOMMENDED_PRESETS_SEEDREAM_4,
|
||||||
|
VIDEO_TASKS_EXECUTION_TIME,
|
||||||
|
Image2ImageTaskCreationRequest,
|
||||||
|
Image2VideoTaskCreationRequest,
|
||||||
|
ImageTaskCreationResponse,
|
||||||
|
Seedream4Options,
|
||||||
|
Seedream4TaskCreationRequest,
|
||||||
|
TaskCreationResponse,
|
||||||
|
TaskImageContent,
|
||||||
|
TaskImageContentUrl,
|
||||||
|
TaskStatusResponse,
|
||||||
|
TaskTextContent,
|
||||||
|
Text2ImageTaskCreationRequest,
|
||||||
|
Text2VideoTaskCreationRequest,
|
||||||
|
)
|
||||||
from comfy_api_nodes.util import (
|
from comfy_api_nodes.util import (
|
||||||
ApiEndpoint,
|
ApiEndpoint,
|
||||||
download_url_to_image_tensor,
|
download_url_to_image_tensor,
|
||||||
@ -29,162 +43,6 @@ BYTEPLUS_TASK_ENDPOINT = "/proxy/byteplus/api/v3/contents/generations/tasks"
|
|||||||
BYTEPLUS_TASK_STATUS_ENDPOINT = "/proxy/byteplus/api/v3/contents/generations/tasks" # + /{task_id}
|
BYTEPLUS_TASK_STATUS_ENDPOINT = "/proxy/byteplus/api/v3/contents/generations/tasks" # + /{task_id}
|
||||||
|
|
||||||
|
|
||||||
class Text2ImageModelName(str, Enum):
|
|
||||||
seedream_3 = "seedream-3-0-t2i-250415"
|
|
||||||
|
|
||||||
|
|
||||||
class Image2ImageModelName(str, Enum):
|
|
||||||
seededit_3 = "seededit-3-0-i2i-250628"
|
|
||||||
|
|
||||||
|
|
||||||
class Text2VideoModelName(str, Enum):
|
|
||||||
seedance_1_pro = "seedance-1-0-pro-250528"
|
|
||||||
seedance_1_lite = "seedance-1-0-lite-t2v-250428"
|
|
||||||
|
|
||||||
|
|
||||||
class Image2VideoModelName(str, Enum):
|
|
||||||
"""note(August 31): Pro model only supports FirstFrame: https://docs.byteplus.com/en/docs/ModelArk/1520757"""
|
|
||||||
|
|
||||||
seedance_1_pro = "seedance-1-0-pro-250528"
|
|
||||||
seedance_1_lite = "seedance-1-0-lite-i2v-250428"
|
|
||||||
|
|
||||||
|
|
||||||
class Text2ImageTaskCreationRequest(BaseModel):
|
|
||||||
model: Text2ImageModelName = Text2ImageModelName.seedream_3
|
|
||||||
prompt: str = Field(...)
|
|
||||||
response_format: Optional[str] = Field("url")
|
|
||||||
size: Optional[str] = Field(None)
|
|
||||||
seed: Optional[int] = Field(0, ge=0, le=2147483647)
|
|
||||||
guidance_scale: Optional[float] = Field(..., ge=1.0, le=10.0)
|
|
||||||
watermark: Optional[bool] = Field(True)
|
|
||||||
|
|
||||||
|
|
||||||
class Image2ImageTaskCreationRequest(BaseModel):
|
|
||||||
model: Image2ImageModelName = Image2ImageModelName.seededit_3
|
|
||||||
prompt: str = Field(...)
|
|
||||||
response_format: Optional[str] = Field("url")
|
|
||||||
image: str = Field(..., description="Base64 encoded string or image URL")
|
|
||||||
size: Optional[str] = Field("adaptive")
|
|
||||||
seed: Optional[int] = Field(..., ge=0, le=2147483647)
|
|
||||||
guidance_scale: Optional[float] = Field(..., ge=1.0, le=10.0)
|
|
||||||
watermark: Optional[bool] = Field(True)
|
|
||||||
|
|
||||||
|
|
||||||
class Seedream4Options(BaseModel):
|
|
||||||
max_images: int = Field(15)
|
|
||||||
|
|
||||||
|
|
||||||
class Seedream4TaskCreationRequest(BaseModel):
|
|
||||||
model: str = Field("seedream-4-0-250828")
|
|
||||||
prompt: str = Field(...)
|
|
||||||
response_format: str = Field("url")
|
|
||||||
image: Optional[list[str]] = Field(None, description="Image URLs")
|
|
||||||
size: str = Field(...)
|
|
||||||
seed: int = Field(..., ge=0, le=2147483647)
|
|
||||||
sequential_image_generation: str = Field("disabled")
|
|
||||||
sequential_image_generation_options: Seedream4Options = Field(Seedream4Options(max_images=15))
|
|
||||||
watermark: bool = Field(True)
|
|
||||||
|
|
||||||
|
|
||||||
class ImageTaskCreationResponse(BaseModel):
|
|
||||||
model: str = Field(...)
|
|
||||||
created: int = Field(..., description="Unix timestamp (in seconds) indicating time when the request was created.")
|
|
||||||
data: list = Field([], description="Contains information about the generated image(s).")
|
|
||||||
error: dict = Field({}, description="Contains `code` and `message` fields in case of error.")
|
|
||||||
|
|
||||||
|
|
||||||
class TaskTextContent(BaseModel):
|
|
||||||
type: str = Field("text")
|
|
||||||
text: str = Field(...)
|
|
||||||
|
|
||||||
|
|
||||||
class TaskImageContentUrl(BaseModel):
|
|
||||||
url: str = Field(...)
|
|
||||||
|
|
||||||
|
|
||||||
class TaskImageContent(BaseModel):
|
|
||||||
type: str = Field("image_url")
|
|
||||||
image_url: TaskImageContentUrl = Field(...)
|
|
||||||
role: Optional[Literal["first_frame", "last_frame", "reference_image"]] = Field(None)
|
|
||||||
|
|
||||||
|
|
||||||
class Text2VideoTaskCreationRequest(BaseModel):
|
|
||||||
model: Text2VideoModelName = Text2VideoModelName.seedance_1_pro
|
|
||||||
content: list[TaskTextContent] = Field(..., min_length=1)
|
|
||||||
|
|
||||||
|
|
||||||
class Image2VideoTaskCreationRequest(BaseModel):
|
|
||||||
model: Image2VideoModelName = Image2VideoModelName.seedance_1_pro
|
|
||||||
content: list[Union[TaskTextContent, TaskImageContent]] = Field(..., min_length=2)
|
|
||||||
|
|
||||||
|
|
||||||
class TaskCreationResponse(BaseModel):
|
|
||||||
id: str = Field(...)
|
|
||||||
|
|
||||||
|
|
||||||
class TaskStatusError(BaseModel):
|
|
||||||
code: str = Field(...)
|
|
||||||
message: str = Field(...)
|
|
||||||
|
|
||||||
|
|
||||||
class TaskStatusResult(BaseModel):
|
|
||||||
video_url: str = Field(...)
|
|
||||||
|
|
||||||
|
|
||||||
class TaskStatusResponse(BaseModel):
|
|
||||||
id: str = Field(...)
|
|
||||||
model: str = Field(...)
|
|
||||||
status: Literal["queued", "running", "cancelled", "succeeded", "failed"] = Field(...)
|
|
||||||
error: Optional[TaskStatusError] = Field(None)
|
|
||||||
content: Optional[TaskStatusResult] = Field(None)
|
|
||||||
|
|
||||||
|
|
||||||
RECOMMENDED_PRESETS = [
|
|
||||||
("1024x1024 (1:1)", 1024, 1024),
|
|
||||||
("864x1152 (3:4)", 864, 1152),
|
|
||||||
("1152x864 (4:3)", 1152, 864),
|
|
||||||
("1280x720 (16:9)", 1280, 720),
|
|
||||||
("720x1280 (9:16)", 720, 1280),
|
|
||||||
("832x1248 (2:3)", 832, 1248),
|
|
||||||
("1248x832 (3:2)", 1248, 832),
|
|
||||||
("1512x648 (21:9)", 1512, 648),
|
|
||||||
("2048x2048 (1:1)", 2048, 2048),
|
|
||||||
("Custom", None, None),
|
|
||||||
]
|
|
||||||
|
|
||||||
RECOMMENDED_PRESETS_SEEDREAM_4 = [
|
|
||||||
("2048x2048 (1:1)", 2048, 2048),
|
|
||||||
("2304x1728 (4:3)", 2304, 1728),
|
|
||||||
("1728x2304 (3:4)", 1728, 2304),
|
|
||||||
("2560x1440 (16:9)", 2560, 1440),
|
|
||||||
("1440x2560 (9:16)", 1440, 2560),
|
|
||||||
("2496x1664 (3:2)", 2496, 1664),
|
|
||||||
("1664x2496 (2:3)", 1664, 2496),
|
|
||||||
("3024x1296 (21:9)", 3024, 1296),
|
|
||||||
("4096x4096 (1:1)", 4096, 4096),
|
|
||||||
("Custom", None, None),
|
|
||||||
]
|
|
||||||
|
|
||||||
# The time in this dictionary are given for 10 seconds duration.
|
|
||||||
VIDEO_TASKS_EXECUTION_TIME = {
|
|
||||||
"seedance-1-0-lite-t2v-250428": {
|
|
||||||
"480p": 40,
|
|
||||||
"720p": 60,
|
|
||||||
"1080p": 90,
|
|
||||||
},
|
|
||||||
"seedance-1-0-lite-i2v-250428": {
|
|
||||||
"480p": 40,
|
|
||||||
"720p": 60,
|
|
||||||
"1080p": 90,
|
|
||||||
},
|
|
||||||
"seedance-1-0-pro-250528": {
|
|
||||||
"480p": 70,
|
|
||||||
"720p": 85,
|
|
||||||
"1080p": 115,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def get_image_url_from_response(response: ImageTaskCreationResponse) -> str:
|
def get_image_url_from_response(response: ImageTaskCreationResponse) -> str:
|
||||||
if response.error:
|
if response.error:
|
||||||
error_msg = f"ByteDance request failed. Code: {response.error['code']}, message: {response.error['message']}"
|
error_msg = f"ByteDance request failed. Code: {response.error['code']}, message: {response.error['message']}"
|
||||||
@ -194,13 +52,6 @@ def get_image_url_from_response(response: ImageTaskCreationResponse) -> str:
|
|||||||
return response.data[0]["url"]
|
return response.data[0]["url"]
|
||||||
|
|
||||||
|
|
||||||
def get_video_url_from_task_status(response: TaskStatusResponse) -> Union[str, None]:
|
|
||||||
"""Returns the video URL from the task status response if it exists."""
|
|
||||||
if hasattr(response, "content") and response.content:
|
|
||||||
return response.content.video_url
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
class ByteDanceImageNode(IO.ComfyNode):
|
class ByteDanceImageNode(IO.ComfyNode):
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@ -211,12 +62,7 @@ class ByteDanceImageNode(IO.ComfyNode):
|
|||||||
category="api node/image/ByteDance",
|
category="api node/image/ByteDance",
|
||||||
description="Generate images using ByteDance models via api based on prompt",
|
description="Generate images using ByteDance models via api based on prompt",
|
||||||
inputs=[
|
inputs=[
|
||||||
IO.Combo.Input(
|
IO.Combo.Input("model", options=["seedream-3-0-t2i-250415"]),
|
||||||
"model",
|
|
||||||
options=Text2ImageModelName,
|
|
||||||
default=Text2ImageModelName.seedream_3,
|
|
||||||
tooltip="Model name",
|
|
||||||
),
|
|
||||||
IO.String.Input(
|
IO.String.Input(
|
||||||
"prompt",
|
"prompt",
|
||||||
multiline=True,
|
multiline=True,
|
||||||
@ -335,12 +181,7 @@ class ByteDanceImageEditNode(IO.ComfyNode):
|
|||||||
category="api node/image/ByteDance",
|
category="api node/image/ByteDance",
|
||||||
description="Edit images using ByteDance models via api based on prompt",
|
description="Edit images using ByteDance models via api based on prompt",
|
||||||
inputs=[
|
inputs=[
|
||||||
IO.Combo.Input(
|
IO.Combo.Input("model", options=["seededit-3-0-i2i-250628"]),
|
||||||
"model",
|
|
||||||
options=Image2ImageModelName,
|
|
||||||
default=Image2ImageModelName.seededit_3,
|
|
||||||
tooltip="Model name",
|
|
||||||
),
|
|
||||||
IO.Image.Input(
|
IO.Image.Input(
|
||||||
"image",
|
"image",
|
||||||
tooltip="The base image to edit",
|
tooltip="The base image to edit",
|
||||||
@ -394,7 +235,7 @@ class ByteDanceImageEditNode(IO.ComfyNode):
|
|||||||
async def execute(
|
async def execute(
|
||||||
cls,
|
cls,
|
||||||
model: str,
|
model: str,
|
||||||
image: torch.Tensor,
|
image: Input.Image,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
seed: int,
|
seed: int,
|
||||||
guidance_scale: float,
|
guidance_scale: float,
|
||||||
@ -434,7 +275,7 @@ class ByteDanceSeedreamNode(IO.ComfyNode):
|
|||||||
inputs=[
|
inputs=[
|
||||||
IO.Combo.Input(
|
IO.Combo.Input(
|
||||||
"model",
|
"model",
|
||||||
options=["seedream-4-0-250828"],
|
options=["seedream-4-5-251128", "seedream-4-0-250828"],
|
||||||
tooltip="Model name",
|
tooltip="Model name",
|
||||||
),
|
),
|
||||||
IO.String.Input(
|
IO.String.Input(
|
||||||
@ -459,7 +300,7 @@ class ByteDanceSeedreamNode(IO.ComfyNode):
|
|||||||
default=2048,
|
default=2048,
|
||||||
min=1024,
|
min=1024,
|
||||||
max=4096,
|
max=4096,
|
||||||
step=64,
|
step=8,
|
||||||
tooltip="Custom width for image. Value is working only if `size_preset` is set to `Custom`",
|
tooltip="Custom width for image. Value is working only if `size_preset` is set to `Custom`",
|
||||||
optional=True,
|
optional=True,
|
||||||
),
|
),
|
||||||
@ -468,7 +309,7 @@ class ByteDanceSeedreamNode(IO.ComfyNode):
|
|||||||
default=2048,
|
default=2048,
|
||||||
min=1024,
|
min=1024,
|
||||||
max=4096,
|
max=4096,
|
||||||
step=64,
|
step=8,
|
||||||
tooltip="Custom height for image. Value is working only if `size_preset` is set to `Custom`",
|
tooltip="Custom height for image. Value is working only if `size_preset` is set to `Custom`",
|
||||||
optional=True,
|
optional=True,
|
||||||
),
|
),
|
||||||
@ -532,7 +373,7 @@ class ByteDanceSeedreamNode(IO.ComfyNode):
|
|||||||
cls,
|
cls,
|
||||||
model: str,
|
model: str,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
image: torch.Tensor = None,
|
image: Input.Image | None = None,
|
||||||
size_preset: str = RECOMMENDED_PRESETS_SEEDREAM_4[0][0],
|
size_preset: str = RECOMMENDED_PRESETS_SEEDREAM_4[0][0],
|
||||||
width: int = 2048,
|
width: int = 2048,
|
||||||
height: int = 2048,
|
height: int = 2048,
|
||||||
@ -555,6 +396,18 @@ class ByteDanceSeedreamNode(IO.ComfyNode):
|
|||||||
raise ValueError(
|
raise ValueError(
|
||||||
f"Custom size out of range: {w}x{h}. " "Both width and height must be between 1024 and 4096 pixels."
|
f"Custom size out of range: {w}x{h}. " "Both width and height must be between 1024 and 4096 pixels."
|
||||||
)
|
)
|
||||||
|
out_num_pixels = w * h
|
||||||
|
mp_provided = out_num_pixels / 1_000_000.0
|
||||||
|
if "seedream-4-5" in model and out_num_pixels < 3686400:
|
||||||
|
raise ValueError(
|
||||||
|
f"Minimum image resolution that Seedream 4.5 can generate is 3.68MP, "
|
||||||
|
f"but {mp_provided:.2f}MP provided."
|
||||||
|
)
|
||||||
|
if "seedream-4-0" in model and out_num_pixels < 921600:
|
||||||
|
raise ValueError(
|
||||||
|
f"Minimum image resolution that the selected model can generate is 0.92MP, "
|
||||||
|
f"but {mp_provided:.2f}MP provided."
|
||||||
|
)
|
||||||
n_input_images = get_number_of_images(image) if image is not None else 0
|
n_input_images = get_number_of_images(image) if image is not None else 0
|
||||||
if n_input_images > 10:
|
if n_input_images > 10:
|
||||||
raise ValueError(f"Maximum of 10 reference images are supported, but {n_input_images} received.")
|
raise ValueError(f"Maximum of 10 reference images are supported, but {n_input_images} received.")
|
||||||
@ -607,9 +460,8 @@ class ByteDanceTextToVideoNode(IO.ComfyNode):
|
|||||||
inputs=[
|
inputs=[
|
||||||
IO.Combo.Input(
|
IO.Combo.Input(
|
||||||
"model",
|
"model",
|
||||||
options=Text2VideoModelName,
|
options=["seedance-1-0-pro-250528", "seedance-1-0-lite-t2v-250428", "seedance-1-0-pro-fast-251015"],
|
||||||
default=Text2VideoModelName.seedance_1_pro,
|
default="seedance-1-0-pro-fast-251015",
|
||||||
tooltip="Model name",
|
|
||||||
),
|
),
|
||||||
IO.String.Input(
|
IO.String.Input(
|
||||||
"prompt",
|
"prompt",
|
||||||
@ -714,9 +566,8 @@ class ByteDanceImageToVideoNode(IO.ComfyNode):
|
|||||||
inputs=[
|
inputs=[
|
||||||
IO.Combo.Input(
|
IO.Combo.Input(
|
||||||
"model",
|
"model",
|
||||||
options=Image2VideoModelName,
|
options=["seedance-1-0-pro-250528", "seedance-1-0-lite-t2v-250428", "seedance-1-0-pro-fast-251015"],
|
||||||
default=Image2VideoModelName.seedance_1_pro,
|
default="seedance-1-0-pro-fast-251015",
|
||||||
tooltip="Model name",
|
|
||||||
),
|
),
|
||||||
IO.String.Input(
|
IO.String.Input(
|
||||||
"prompt",
|
"prompt",
|
||||||
@ -787,7 +638,7 @@ class ByteDanceImageToVideoNode(IO.ComfyNode):
|
|||||||
cls,
|
cls,
|
||||||
model: str,
|
model: str,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
image: torch.Tensor,
|
image: Input.Image,
|
||||||
resolution: str,
|
resolution: str,
|
||||||
aspect_ratio: str,
|
aspect_ratio: str,
|
||||||
duration: int,
|
duration: int,
|
||||||
@ -833,9 +684,8 @@ class ByteDanceFirstLastFrameNode(IO.ComfyNode):
|
|||||||
inputs=[
|
inputs=[
|
||||||
IO.Combo.Input(
|
IO.Combo.Input(
|
||||||
"model",
|
"model",
|
||||||
options=[model.value for model in Image2VideoModelName],
|
options=["seedance-1-0-pro-250528", "seedance-1-0-lite-i2v-250428"],
|
||||||
default=Image2VideoModelName.seedance_1_lite.value,
|
default="seedance-1-0-lite-i2v-250428",
|
||||||
tooltip="Model name",
|
|
||||||
),
|
),
|
||||||
IO.String.Input(
|
IO.String.Input(
|
||||||
"prompt",
|
"prompt",
|
||||||
@ -910,8 +760,8 @@ class ByteDanceFirstLastFrameNode(IO.ComfyNode):
|
|||||||
cls,
|
cls,
|
||||||
model: str,
|
model: str,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
first_frame: torch.Tensor,
|
first_frame: Input.Image,
|
||||||
last_frame: torch.Tensor,
|
last_frame: Input.Image,
|
||||||
resolution: str,
|
resolution: str,
|
||||||
aspect_ratio: str,
|
aspect_ratio: str,
|
||||||
duration: int,
|
duration: int,
|
||||||
@ -968,9 +818,8 @@ class ByteDanceImageReferenceNode(IO.ComfyNode):
|
|||||||
inputs=[
|
inputs=[
|
||||||
IO.Combo.Input(
|
IO.Combo.Input(
|
||||||
"model",
|
"model",
|
||||||
options=[Image2VideoModelName.seedance_1_lite.value],
|
options=["seedance-1-0-pro-250528", "seedance-1-0-lite-i2v-250428"],
|
||||||
default=Image2VideoModelName.seedance_1_lite.value,
|
default="seedance-1-0-lite-i2v-250428",
|
||||||
tooltip="Model name",
|
|
||||||
),
|
),
|
||||||
IO.String.Input(
|
IO.String.Input(
|
||||||
"prompt",
|
"prompt",
|
||||||
@ -1034,7 +883,7 @@ class ByteDanceImageReferenceNode(IO.ComfyNode):
|
|||||||
cls,
|
cls,
|
||||||
model: str,
|
model: str,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
images: torch.Tensor,
|
images: Input.Image,
|
||||||
resolution: str,
|
resolution: str,
|
||||||
aspect_ratio: str,
|
aspect_ratio: str,
|
||||||
duration: int,
|
duration: int,
|
||||||
@ -1069,8 +918,8 @@ class ByteDanceImageReferenceNode(IO.ComfyNode):
|
|||||||
|
|
||||||
async def process_video_task(
|
async def process_video_task(
|
||||||
cls: type[IO.ComfyNode],
|
cls: type[IO.ComfyNode],
|
||||||
payload: Union[Text2VideoTaskCreationRequest, Image2VideoTaskCreationRequest],
|
payload: Text2VideoTaskCreationRequest | Image2VideoTaskCreationRequest,
|
||||||
estimated_duration: Optional[int],
|
estimated_duration: int | None,
|
||||||
) -> IO.NodeOutput:
|
) -> IO.NodeOutput:
|
||||||
initial_response = await sync_op(
|
initial_response = await sync_op(
|
||||||
cls,
|
cls,
|
||||||
@ -1085,7 +934,7 @@ async def process_video_task(
|
|||||||
estimated_duration=estimated_duration,
|
estimated_duration=estimated_duration,
|
||||||
response_model=TaskStatusResponse,
|
response_model=TaskStatusResponse,
|
||||||
)
|
)
|
||||||
return IO.NodeOutput(await download_url_to_video_output(get_video_url_from_task_status(response)))
|
return IO.NodeOutput(await download_url_to_video_output(response.content.video_url))
|
||||||
|
|
||||||
|
|
||||||
def raise_if_text_params(prompt: str, text_params: list[str]) -> None:
|
def raise_if_text_params(prompt: str, text_params: list[str]) -> None:
|
||||||
|
|||||||
@ -13,8 +13,7 @@ import torch
|
|||||||
from typing_extensions import override
|
from typing_extensions import override
|
||||||
|
|
||||||
import folder_paths
|
import folder_paths
|
||||||
from comfy_api.latest import IO, ComfyExtension, Input
|
from comfy_api.latest import IO, ComfyExtension, Input, Types
|
||||||
from comfy_api.util import VideoCodec, VideoContainer
|
|
||||||
from comfy_api_nodes.apis.gemini_api import (
|
from comfy_api_nodes.apis.gemini_api import (
|
||||||
GeminiContent,
|
GeminiContent,
|
||||||
GeminiFileData,
|
GeminiFileData,
|
||||||
@ -27,6 +26,8 @@ from comfy_api_nodes.apis.gemini_api import (
|
|||||||
GeminiMimeType,
|
GeminiMimeType,
|
||||||
GeminiPart,
|
GeminiPart,
|
||||||
GeminiRole,
|
GeminiRole,
|
||||||
|
GeminiSystemInstructionContent,
|
||||||
|
GeminiTextPart,
|
||||||
Modality,
|
Modality,
|
||||||
)
|
)
|
||||||
from comfy_api_nodes.util import (
|
from comfy_api_nodes.util import (
|
||||||
@ -43,6 +44,14 @@ from comfy_api_nodes.util import (
|
|||||||
|
|
||||||
GEMINI_BASE_ENDPOINT = "/proxy/vertexai/gemini"
|
GEMINI_BASE_ENDPOINT = "/proxy/vertexai/gemini"
|
||||||
GEMINI_MAX_INPUT_FILE_SIZE = 20 * 1024 * 1024 # 20 MB
|
GEMINI_MAX_INPUT_FILE_SIZE = 20 * 1024 * 1024 # 20 MB
|
||||||
|
GEMINI_IMAGE_SYS_PROMPT = (
|
||||||
|
"You are an expert image-generation engine. You must ALWAYS produce an image.\n"
|
||||||
|
"Interpret all user input—regardless of "
|
||||||
|
"format, intent, or abstraction—as literal visual directives for image composition.\n"
|
||||||
|
"If a prompt is conversational or lacks specific visual details, "
|
||||||
|
"you must creatively invent a concrete visual scenario that depicts the concept.\n"
|
||||||
|
"Prioritize generating the visual representation above any text, formatting, or conversational requests."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class GeminiModel(str, Enum):
|
class GeminiModel(str, Enum):
|
||||||
@ -68,7 +77,7 @@ class GeminiImageModel(str, Enum):
|
|||||||
|
|
||||||
async def create_image_parts(
|
async def create_image_parts(
|
||||||
cls: type[IO.ComfyNode],
|
cls: type[IO.ComfyNode],
|
||||||
images: torch.Tensor,
|
images: Input.Image,
|
||||||
image_limit: int = 0,
|
image_limit: int = 0,
|
||||||
) -> list[GeminiPart]:
|
) -> list[GeminiPart]:
|
||||||
image_parts: list[GeminiPart] = []
|
image_parts: list[GeminiPart] = []
|
||||||
@ -154,8 +163,8 @@ def get_text_from_response(response: GeminiGenerateContentResponse) -> str:
|
|||||||
return "\n".join([part.text for part in parts])
|
return "\n".join([part.text for part in parts])
|
||||||
|
|
||||||
|
|
||||||
def get_image_from_response(response: GeminiGenerateContentResponse) -> torch.Tensor:
|
def get_image_from_response(response: GeminiGenerateContentResponse) -> Input.Image:
|
||||||
image_tensors: list[torch.Tensor] = []
|
image_tensors: list[Input.Image] = []
|
||||||
parts = get_parts_by_type(response, "image/png")
|
parts = get_parts_by_type(response, "image/png")
|
||||||
for part in parts:
|
for part in parts:
|
||||||
image_data = base64.b64decode(part.inlineData.data)
|
image_data = base64.b64decode(part.inlineData.data)
|
||||||
@ -277,6 +286,13 @@ class GeminiNode(IO.ComfyNode):
|
|||||||
tooltip="Optional file(s) to use as context for the model. "
|
tooltip="Optional file(s) to use as context for the model. "
|
||||||
"Accepts inputs from the Gemini Generate Content Input Files node.",
|
"Accepts inputs from the Gemini Generate Content Input Files node.",
|
||||||
),
|
),
|
||||||
|
IO.String.Input(
|
||||||
|
"system_prompt",
|
||||||
|
multiline=True,
|
||||||
|
default="",
|
||||||
|
optional=True,
|
||||||
|
tooltip="Foundational instructions that dictate an AI's behavior.",
|
||||||
|
),
|
||||||
],
|
],
|
||||||
outputs=[
|
outputs=[
|
||||||
IO.String.Output(),
|
IO.String.Output(),
|
||||||
@ -293,7 +309,9 @@ class GeminiNode(IO.ComfyNode):
|
|||||||
def create_video_parts(cls, video_input: Input.Video) -> list[GeminiPart]:
|
def create_video_parts(cls, video_input: Input.Video) -> list[GeminiPart]:
|
||||||
"""Convert video input to Gemini API compatible parts."""
|
"""Convert video input to Gemini API compatible parts."""
|
||||||
|
|
||||||
base_64_string = video_to_base64_string(video_input, container_format=VideoContainer.MP4, codec=VideoCodec.H264)
|
base_64_string = video_to_base64_string(
|
||||||
|
video_input, container_format=Types.VideoContainer.MP4, codec=Types.VideoCodec.H264
|
||||||
|
)
|
||||||
return [
|
return [
|
||||||
GeminiPart(
|
GeminiPart(
|
||||||
inlineData=GeminiInlineData(
|
inlineData=GeminiInlineData(
|
||||||
@ -343,10 +361,11 @@ class GeminiNode(IO.ComfyNode):
|
|||||||
prompt: str,
|
prompt: str,
|
||||||
model: str,
|
model: str,
|
||||||
seed: int,
|
seed: int,
|
||||||
images: torch.Tensor | None = None,
|
images: Input.Image | None = None,
|
||||||
audio: Input.Audio | None = None,
|
audio: Input.Audio | None = None,
|
||||||
video: Input.Video | None = None,
|
video: Input.Video | None = None,
|
||||||
files: list[GeminiPart] | None = None,
|
files: list[GeminiPart] | None = None,
|
||||||
|
system_prompt: str = "",
|
||||||
) -> IO.NodeOutput:
|
) -> IO.NodeOutput:
|
||||||
validate_string(prompt, strip_whitespace=False)
|
validate_string(prompt, strip_whitespace=False)
|
||||||
|
|
||||||
@ -363,7 +382,10 @@ class GeminiNode(IO.ComfyNode):
|
|||||||
if files is not None:
|
if files is not None:
|
||||||
parts.extend(files)
|
parts.extend(files)
|
||||||
|
|
||||||
# Create response
|
gemini_system_prompt = None
|
||||||
|
if system_prompt:
|
||||||
|
gemini_system_prompt = GeminiSystemInstructionContent(parts=[GeminiTextPart(text=system_prompt)], role=None)
|
||||||
|
|
||||||
response = await sync_op(
|
response = await sync_op(
|
||||||
cls,
|
cls,
|
||||||
endpoint=ApiEndpoint(path=f"{GEMINI_BASE_ENDPOINT}/{model}", method="POST"),
|
endpoint=ApiEndpoint(path=f"{GEMINI_BASE_ENDPOINT}/{model}", method="POST"),
|
||||||
@ -373,7 +395,8 @@ class GeminiNode(IO.ComfyNode):
|
|||||||
role=GeminiRole.user,
|
role=GeminiRole.user,
|
||||||
parts=parts,
|
parts=parts,
|
||||||
)
|
)
|
||||||
]
|
],
|
||||||
|
systemInstruction=gemini_system_prompt,
|
||||||
),
|
),
|
||||||
response_model=GeminiGenerateContentResponse,
|
response_model=GeminiGenerateContentResponse,
|
||||||
price_extractor=calculate_tokens_price,
|
price_extractor=calculate_tokens_price,
|
||||||
@ -523,6 +546,13 @@ class GeminiImage(IO.ComfyNode):
|
|||||||
"'IMAGE+TEXT' to return both the generated image and a text response.",
|
"'IMAGE+TEXT' to return both the generated image and a text response.",
|
||||||
optional=True,
|
optional=True,
|
||||||
),
|
),
|
||||||
|
IO.String.Input(
|
||||||
|
"system_prompt",
|
||||||
|
multiline=True,
|
||||||
|
default=GEMINI_IMAGE_SYS_PROMPT,
|
||||||
|
optional=True,
|
||||||
|
tooltip="Foundational instructions that dictate an AI's behavior.",
|
||||||
|
),
|
||||||
],
|
],
|
||||||
outputs=[
|
outputs=[
|
||||||
IO.Image.Output(),
|
IO.Image.Output(),
|
||||||
@ -542,10 +572,11 @@ class GeminiImage(IO.ComfyNode):
|
|||||||
prompt: str,
|
prompt: str,
|
||||||
model: str,
|
model: str,
|
||||||
seed: int,
|
seed: int,
|
||||||
images: torch.Tensor | None = None,
|
images: Input.Image | None = None,
|
||||||
files: list[GeminiPart] | None = None,
|
files: list[GeminiPart] | None = None,
|
||||||
aspect_ratio: str = "auto",
|
aspect_ratio: str = "auto",
|
||||||
response_modalities: str = "IMAGE+TEXT",
|
response_modalities: str = "IMAGE+TEXT",
|
||||||
|
system_prompt: str = "",
|
||||||
) -> IO.NodeOutput:
|
) -> IO.NodeOutput:
|
||||||
validate_string(prompt, strip_whitespace=True, min_length=1)
|
validate_string(prompt, strip_whitespace=True, min_length=1)
|
||||||
parts: list[GeminiPart] = [GeminiPart(text=prompt)]
|
parts: list[GeminiPart] = [GeminiPart(text=prompt)]
|
||||||
@ -559,6 +590,10 @@ class GeminiImage(IO.ComfyNode):
|
|||||||
if files is not None:
|
if files is not None:
|
||||||
parts.extend(files)
|
parts.extend(files)
|
||||||
|
|
||||||
|
gemini_system_prompt = None
|
||||||
|
if system_prompt:
|
||||||
|
gemini_system_prompt = GeminiSystemInstructionContent(parts=[GeminiTextPart(text=system_prompt)], role=None)
|
||||||
|
|
||||||
response = await sync_op(
|
response = await sync_op(
|
||||||
cls,
|
cls,
|
||||||
endpoint=ApiEndpoint(path=f"{GEMINI_BASE_ENDPOINT}/{model}", method="POST"),
|
endpoint=ApiEndpoint(path=f"{GEMINI_BASE_ENDPOINT}/{model}", method="POST"),
|
||||||
@ -570,6 +605,7 @@ class GeminiImage(IO.ComfyNode):
|
|||||||
responseModalities=(["IMAGE"] if response_modalities == "IMAGE" else ["TEXT", "IMAGE"]),
|
responseModalities=(["IMAGE"] if response_modalities == "IMAGE" else ["TEXT", "IMAGE"]),
|
||||||
imageConfig=None if aspect_ratio == "auto" else image_config,
|
imageConfig=None if aspect_ratio == "auto" else image_config,
|
||||||
),
|
),
|
||||||
|
systemInstruction=gemini_system_prompt,
|
||||||
),
|
),
|
||||||
response_model=GeminiGenerateContentResponse,
|
response_model=GeminiGenerateContentResponse,
|
||||||
price_extractor=calculate_tokens_price,
|
price_extractor=calculate_tokens_price,
|
||||||
@ -640,6 +676,13 @@ class GeminiImage2(IO.ComfyNode):
|
|||||||
tooltip="Optional file(s) to use as context for the model. "
|
tooltip="Optional file(s) to use as context for the model. "
|
||||||
"Accepts inputs from the Gemini Generate Content Input Files node.",
|
"Accepts inputs from the Gemini Generate Content Input Files node.",
|
||||||
),
|
),
|
||||||
|
IO.String.Input(
|
||||||
|
"system_prompt",
|
||||||
|
multiline=True,
|
||||||
|
default=GEMINI_IMAGE_SYS_PROMPT,
|
||||||
|
optional=True,
|
||||||
|
tooltip="Foundational instructions that dictate an AI's behavior.",
|
||||||
|
),
|
||||||
],
|
],
|
||||||
outputs=[
|
outputs=[
|
||||||
IO.Image.Output(),
|
IO.Image.Output(),
|
||||||
@ -662,8 +705,9 @@ class GeminiImage2(IO.ComfyNode):
|
|||||||
aspect_ratio: str,
|
aspect_ratio: str,
|
||||||
resolution: str,
|
resolution: str,
|
||||||
response_modalities: str,
|
response_modalities: str,
|
||||||
images: torch.Tensor | None = None,
|
images: Input.Image | None = None,
|
||||||
files: list[GeminiPart] | None = None,
|
files: list[GeminiPart] | None = None,
|
||||||
|
system_prompt: str = "",
|
||||||
) -> IO.NodeOutput:
|
) -> IO.NodeOutput:
|
||||||
validate_string(prompt, strip_whitespace=True, min_length=1)
|
validate_string(prompt, strip_whitespace=True, min_length=1)
|
||||||
|
|
||||||
@ -679,6 +723,10 @@ class GeminiImage2(IO.ComfyNode):
|
|||||||
if aspect_ratio != "auto":
|
if aspect_ratio != "auto":
|
||||||
image_config.aspectRatio = aspect_ratio
|
image_config.aspectRatio = aspect_ratio
|
||||||
|
|
||||||
|
gemini_system_prompt = None
|
||||||
|
if system_prompt:
|
||||||
|
gemini_system_prompt = GeminiSystemInstructionContent(parts=[GeminiTextPart(text=system_prompt)], role=None)
|
||||||
|
|
||||||
response = await sync_op(
|
response = await sync_op(
|
||||||
cls,
|
cls,
|
||||||
ApiEndpoint(path=f"{GEMINI_BASE_ENDPOINT}/{model}", method="POST"),
|
ApiEndpoint(path=f"{GEMINI_BASE_ENDPOINT}/{model}", method="POST"),
|
||||||
@ -690,6 +738,7 @@ class GeminiImage2(IO.ComfyNode):
|
|||||||
responseModalities=(["IMAGE"] if response_modalities == "IMAGE" else ["TEXT", "IMAGE"]),
|
responseModalities=(["IMAGE"] if response_modalities == "IMAGE" else ["TEXT", "IMAGE"]),
|
||||||
imageConfig=image_config,
|
imageConfig=image_config,
|
||||||
),
|
),
|
||||||
|
systemInstruction=gemini_system_prompt,
|
||||||
),
|
),
|
||||||
response_model=GeminiGenerateContentResponse,
|
response_model=GeminiGenerateContentResponse,
|
||||||
price_extractor=calculate_tokens_price,
|
price_extractor=calculate_tokens_price,
|
||||||
|
|||||||
@ -1,12 +1,9 @@
|
|||||||
from io import BytesIO
|
from io import BytesIO
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
import torch
|
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
from typing_extensions import override
|
from typing_extensions import override
|
||||||
|
|
||||||
from comfy_api.input_impl import VideoFromFile
|
from comfy_api.latest import IO, ComfyExtension, Input, InputImpl
|
||||||
from comfy_api.latest import IO, ComfyExtension
|
|
||||||
from comfy_api_nodes.util import (
|
from comfy_api_nodes.util import (
|
||||||
ApiEndpoint,
|
ApiEndpoint,
|
||||||
get_number_of_images,
|
get_number_of_images,
|
||||||
@ -26,9 +23,9 @@ class ExecuteTaskRequest(BaseModel):
|
|||||||
model: str = Field(...)
|
model: str = Field(...)
|
||||||
duration: int = Field(...)
|
duration: int = Field(...)
|
||||||
resolution: str = Field(...)
|
resolution: str = Field(...)
|
||||||
fps: Optional[int] = Field(25)
|
fps: int | None = Field(25)
|
||||||
generate_audio: Optional[bool] = Field(True)
|
generate_audio: bool | None = Field(True)
|
||||||
image_uri: Optional[str] = Field(None)
|
image_uri: str | None = Field(None)
|
||||||
|
|
||||||
|
|
||||||
class TextToVideoNode(IO.ComfyNode):
|
class TextToVideoNode(IO.ComfyNode):
|
||||||
@ -103,7 +100,7 @@ class TextToVideoNode(IO.ComfyNode):
|
|||||||
as_binary=True,
|
as_binary=True,
|
||||||
max_retries=1,
|
max_retries=1,
|
||||||
)
|
)
|
||||||
return IO.NodeOutput(VideoFromFile(BytesIO(response)))
|
return IO.NodeOutput(InputImpl.VideoFromFile(BytesIO(response)))
|
||||||
|
|
||||||
|
|
||||||
class ImageToVideoNode(IO.ComfyNode):
|
class ImageToVideoNode(IO.ComfyNode):
|
||||||
@ -153,7 +150,7 @@ class ImageToVideoNode(IO.ComfyNode):
|
|||||||
@classmethod
|
@classmethod
|
||||||
async def execute(
|
async def execute(
|
||||||
cls,
|
cls,
|
||||||
image: torch.Tensor,
|
image: Input.Image,
|
||||||
model: str,
|
model: str,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
duration: int,
|
duration: int,
|
||||||
@ -183,7 +180,7 @@ class ImageToVideoNode(IO.ComfyNode):
|
|||||||
as_binary=True,
|
as_binary=True,
|
||||||
max_retries=1,
|
max_retries=1,
|
||||||
)
|
)
|
||||||
return IO.NodeOutput(VideoFromFile(BytesIO(response)))
|
return IO.NodeOutput(InputImpl.VideoFromFile(BytesIO(response)))
|
||||||
|
|
||||||
|
|
||||||
class LtxvApiExtension(ComfyExtension):
|
class LtxvApiExtension(ComfyExtension):
|
||||||
|
|||||||
@ -1,11 +1,8 @@
|
|||||||
import logging
|
import logging
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
import torch
|
|
||||||
from typing_extensions import override
|
from typing_extensions import override
|
||||||
|
|
||||||
from comfy_api.input import VideoInput
|
from comfy_api.latest import IO, ComfyExtension, Input
|
||||||
from comfy_api.latest import IO, ComfyExtension
|
|
||||||
from comfy_api_nodes.apis import (
|
from comfy_api_nodes.apis import (
|
||||||
MoonvalleyPromptResponse,
|
MoonvalleyPromptResponse,
|
||||||
MoonvalleyTextToVideoInferenceParams,
|
MoonvalleyTextToVideoInferenceParams,
|
||||||
@ -61,7 +58,7 @@ def validate_task_creation_response(response) -> None:
|
|||||||
raise RuntimeError(error_msg)
|
raise RuntimeError(error_msg)
|
||||||
|
|
||||||
|
|
||||||
def validate_video_to_video_input(video: VideoInput) -> VideoInput:
|
def validate_video_to_video_input(video: Input.Video) -> Input.Video:
|
||||||
"""
|
"""
|
||||||
Validates and processes video input for Moonvalley Video-to-Video generation.
|
Validates and processes video input for Moonvalley Video-to-Video generation.
|
||||||
|
|
||||||
@ -82,7 +79,7 @@ def validate_video_to_video_input(video: VideoInput) -> VideoInput:
|
|||||||
return _validate_and_trim_duration(video)
|
return _validate_and_trim_duration(video)
|
||||||
|
|
||||||
|
|
||||||
def _get_video_dimensions(video: VideoInput) -> tuple[int, int]:
|
def _get_video_dimensions(video: Input.Video) -> tuple[int, int]:
|
||||||
"""Extracts video dimensions with error handling."""
|
"""Extracts video dimensions with error handling."""
|
||||||
try:
|
try:
|
||||||
return video.get_dimensions()
|
return video.get_dimensions()
|
||||||
@ -106,7 +103,7 @@ def _validate_video_dimensions(width: int, height: int) -> None:
|
|||||||
raise ValueError(f"Resolution {width}x{height} not supported. Supported: {supported_list}")
|
raise ValueError(f"Resolution {width}x{height} not supported. Supported: {supported_list}")
|
||||||
|
|
||||||
|
|
||||||
def _validate_and_trim_duration(video: VideoInput) -> VideoInput:
|
def _validate_and_trim_duration(video: Input.Video) -> Input.Video:
|
||||||
"""Validates video duration and trims to 5 seconds if needed."""
|
"""Validates video duration and trims to 5 seconds if needed."""
|
||||||
duration = video.get_duration()
|
duration = video.get_duration()
|
||||||
_validate_minimum_duration(duration)
|
_validate_minimum_duration(duration)
|
||||||
@ -119,7 +116,7 @@ def _validate_minimum_duration(duration: float) -> None:
|
|||||||
raise ValueError("Input video must be at least 5 seconds long.")
|
raise ValueError("Input video must be at least 5 seconds long.")
|
||||||
|
|
||||||
|
|
||||||
def _trim_if_too_long(video: VideoInput, duration: float) -> VideoInput:
|
def _trim_if_too_long(video: Input.Video, duration: float) -> Input.Video:
|
||||||
"""Trims video to 5 seconds if longer."""
|
"""Trims video to 5 seconds if longer."""
|
||||||
if duration > 5:
|
if duration > 5:
|
||||||
return trim_video(video, 5)
|
return trim_video(video, 5)
|
||||||
@ -241,7 +238,7 @@ class MoonvalleyImg2VideoNode(IO.ComfyNode):
|
|||||||
@classmethod
|
@classmethod
|
||||||
async def execute(
|
async def execute(
|
||||||
cls,
|
cls,
|
||||||
image: torch.Tensor,
|
image: Input.Image,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
negative_prompt: str,
|
negative_prompt: str,
|
||||||
resolution: str,
|
resolution: str,
|
||||||
@ -362,9 +359,9 @@ class MoonvalleyVideo2VideoNode(IO.ComfyNode):
|
|||||||
prompt: str,
|
prompt: str,
|
||||||
negative_prompt: str,
|
negative_prompt: str,
|
||||||
seed: int,
|
seed: int,
|
||||||
video: Optional[VideoInput] = None,
|
video: Input.Video | None = None,
|
||||||
control_type: str = "Motion Transfer",
|
control_type: str = "Motion Transfer",
|
||||||
motion_intensity: Optional[int] = 100,
|
motion_intensity: int | None = 100,
|
||||||
steps=33,
|
steps=33,
|
||||||
prompt_adherence=4.5,
|
prompt_adherence=4.5,
|
||||||
) -> IO.NodeOutput:
|
) -> IO.NodeOutput:
|
||||||
|
|||||||
@ -11,12 +11,11 @@ User Guides:
|
|||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from typing import Union, Optional
|
|
||||||
from typing_extensions import override
|
|
||||||
from enum import Enum
|
from enum import Enum
|
||||||
|
|
||||||
import torch
|
from typing_extensions import override
|
||||||
|
|
||||||
|
from comfy_api.latest import IO, ComfyExtension, Input, InputImpl
|
||||||
from comfy_api_nodes.apis import (
|
from comfy_api_nodes.apis import (
|
||||||
RunwayImageToVideoRequest,
|
RunwayImageToVideoRequest,
|
||||||
RunwayImageToVideoResponse,
|
RunwayImageToVideoResponse,
|
||||||
@ -44,8 +43,6 @@ from comfy_api_nodes.util import (
|
|||||||
sync_op,
|
sync_op,
|
||||||
poll_op,
|
poll_op,
|
||||||
)
|
)
|
||||||
from comfy_api.input_impl import VideoFromFile
|
|
||||||
from comfy_api.latest import ComfyExtension, IO
|
|
||||||
|
|
||||||
PATH_IMAGE_TO_VIDEO = "/proxy/runway/image_to_video"
|
PATH_IMAGE_TO_VIDEO = "/proxy/runway/image_to_video"
|
||||||
PATH_TEXT_TO_IMAGE = "/proxy/runway/text_to_image"
|
PATH_TEXT_TO_IMAGE = "/proxy/runway/text_to_image"
|
||||||
@ -80,7 +77,7 @@ class RunwayGen3aAspectRatio(str, Enum):
|
|||||||
field_1280_768 = "1280:768"
|
field_1280_768 = "1280:768"
|
||||||
|
|
||||||
|
|
||||||
def get_video_url_from_task_status(response: TaskStatusResponse) -> Union[str, None]:
|
def get_video_url_from_task_status(response: TaskStatusResponse) -> str | None:
|
||||||
"""Returns the video URL from the task status response if it exists."""
|
"""Returns the video URL from the task status response if it exists."""
|
||||||
if hasattr(response, "output") and len(response.output) > 0:
|
if hasattr(response, "output") and len(response.output) > 0:
|
||||||
return response.output[0]
|
return response.output[0]
|
||||||
@ -89,13 +86,13 @@ def get_video_url_from_task_status(response: TaskStatusResponse) -> Union[str, N
|
|||||||
|
|
||||||
def extract_progress_from_task_status(
|
def extract_progress_from_task_status(
|
||||||
response: TaskStatusResponse,
|
response: TaskStatusResponse,
|
||||||
) -> Union[float, None]:
|
) -> float | None:
|
||||||
if hasattr(response, "progress") and response.progress is not None:
|
if hasattr(response, "progress") and response.progress is not None:
|
||||||
return response.progress * 100
|
return response.progress * 100
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def get_image_url_from_task_status(response: TaskStatusResponse) -> Union[str, None]:
|
def get_image_url_from_task_status(response: TaskStatusResponse) -> str | None:
|
||||||
"""Returns the image URL from the task status response if it exists."""
|
"""Returns the image URL from the task status response if it exists."""
|
||||||
if hasattr(response, "output") and len(response.output) > 0:
|
if hasattr(response, "output") and len(response.output) > 0:
|
||||||
return response.output[0]
|
return response.output[0]
|
||||||
@ -103,7 +100,7 @@ def get_image_url_from_task_status(response: TaskStatusResponse) -> Union[str, N
|
|||||||
|
|
||||||
|
|
||||||
async def get_response(
|
async def get_response(
|
||||||
cls: type[IO.ComfyNode], task_id: str, estimated_duration: Optional[int] = None
|
cls: type[IO.ComfyNode], task_id: str, estimated_duration: int | None = None
|
||||||
) -> TaskStatusResponse:
|
) -> TaskStatusResponse:
|
||||||
"""Poll the task status until it is finished then get the response."""
|
"""Poll the task status until it is finished then get the response."""
|
||||||
return await poll_op(
|
return await poll_op(
|
||||||
@ -119,8 +116,8 @@ async def get_response(
|
|||||||
async def generate_video(
|
async def generate_video(
|
||||||
cls: type[IO.ComfyNode],
|
cls: type[IO.ComfyNode],
|
||||||
request: RunwayImageToVideoRequest,
|
request: RunwayImageToVideoRequest,
|
||||||
estimated_duration: Optional[int] = None,
|
estimated_duration: int | None = None,
|
||||||
) -> VideoFromFile:
|
) -> InputImpl.VideoFromFile:
|
||||||
initial_response = await sync_op(
|
initial_response = await sync_op(
|
||||||
cls,
|
cls,
|
||||||
endpoint=ApiEndpoint(path=PATH_IMAGE_TO_VIDEO, method="POST"),
|
endpoint=ApiEndpoint(path=PATH_IMAGE_TO_VIDEO, method="POST"),
|
||||||
@ -193,7 +190,7 @@ class RunwayImageToVideoNodeGen3a(IO.ComfyNode):
|
|||||||
async def execute(
|
async def execute(
|
||||||
cls,
|
cls,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
start_frame: torch.Tensor,
|
start_frame: Input.Image,
|
||||||
duration: str,
|
duration: str,
|
||||||
ratio: str,
|
ratio: str,
|
||||||
seed: int,
|
seed: int,
|
||||||
@ -283,7 +280,7 @@ class RunwayImageToVideoNodeGen4(IO.ComfyNode):
|
|||||||
async def execute(
|
async def execute(
|
||||||
cls,
|
cls,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
start_frame: torch.Tensor,
|
start_frame: Input.Image,
|
||||||
duration: str,
|
duration: str,
|
||||||
ratio: str,
|
ratio: str,
|
||||||
seed: int,
|
seed: int,
|
||||||
@ -381,8 +378,8 @@ class RunwayFirstLastFrameNode(IO.ComfyNode):
|
|||||||
async def execute(
|
async def execute(
|
||||||
cls,
|
cls,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
start_frame: torch.Tensor,
|
start_frame: Input.Image,
|
||||||
end_frame: torch.Tensor,
|
end_frame: Input.Image,
|
||||||
duration: str,
|
duration: str,
|
||||||
ratio: str,
|
ratio: str,
|
||||||
seed: int,
|
seed: int,
|
||||||
@ -467,7 +464,7 @@ class RunwayTextToImageNode(IO.ComfyNode):
|
|||||||
cls,
|
cls,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
ratio: str,
|
ratio: str,
|
||||||
reference_image: Optional[torch.Tensor] = None,
|
reference_image: Input.Image | None = None,
|
||||||
) -> IO.NodeOutput:
|
) -> IO.NodeOutput:
|
||||||
validate_string(prompt, min_length=1)
|
validate_string(prompt, min_length=1)
|
||||||
|
|
||||||
|
|||||||
@ -1,11 +1,9 @@
|
|||||||
import base64
|
import base64
|
||||||
from io import BytesIO
|
from io import BytesIO
|
||||||
|
|
||||||
import torch
|
|
||||||
from typing_extensions import override
|
from typing_extensions import override
|
||||||
|
|
||||||
from comfy_api.input_impl.video_types import VideoFromFile
|
from comfy_api.latest import IO, ComfyExtension, Input, InputImpl
|
||||||
from comfy_api.latest import IO, ComfyExtension
|
|
||||||
from comfy_api_nodes.apis.veo_api import (
|
from comfy_api_nodes.apis.veo_api import (
|
||||||
VeoGenVidPollRequest,
|
VeoGenVidPollRequest,
|
||||||
VeoGenVidPollResponse,
|
VeoGenVidPollResponse,
|
||||||
@ -232,7 +230,7 @@ class VeoVideoGenerationNode(IO.ComfyNode):
|
|||||||
|
|
||||||
# Check if video is provided as base64 or URL
|
# Check if video is provided as base64 or URL
|
||||||
if hasattr(video, "bytesBase64Encoded") and video.bytesBase64Encoded:
|
if hasattr(video, "bytesBase64Encoded") and video.bytesBase64Encoded:
|
||||||
return IO.NodeOutput(VideoFromFile(BytesIO(base64.b64decode(video.bytesBase64Encoded))))
|
return IO.NodeOutput(InputImpl.VideoFromFile(BytesIO(base64.b64decode(video.bytesBase64Encoded))))
|
||||||
|
|
||||||
if hasattr(video, "gcsUri") and video.gcsUri:
|
if hasattr(video, "gcsUri") and video.gcsUri:
|
||||||
return IO.NodeOutput(await download_url_to_video_output(video.gcsUri))
|
return IO.NodeOutput(await download_url_to_video_output(video.gcsUri))
|
||||||
@ -431,8 +429,8 @@ class Veo3FirstLastFrameNode(IO.ComfyNode):
|
|||||||
aspect_ratio: str,
|
aspect_ratio: str,
|
||||||
duration: int,
|
duration: int,
|
||||||
seed: int,
|
seed: int,
|
||||||
first_frame: torch.Tensor,
|
first_frame: Input.Image,
|
||||||
last_frame: torch.Tensor,
|
last_frame: Input.Image,
|
||||||
model: str,
|
model: str,
|
||||||
generate_audio: bool,
|
generate_audio: bool,
|
||||||
):
|
):
|
||||||
@ -493,7 +491,7 @@ class Veo3FirstLastFrameNode(IO.ComfyNode):
|
|||||||
if response.videos:
|
if response.videos:
|
||||||
video = response.videos[0]
|
video = response.videos[0]
|
||||||
if video.bytesBase64Encoded:
|
if video.bytesBase64Encoded:
|
||||||
return IO.NodeOutput(VideoFromFile(BytesIO(base64.b64decode(video.bytesBase64Encoded))))
|
return IO.NodeOutput(InputImpl.VideoFromFile(BytesIO(base64.b64decode(video.bytesBase64Encoded))))
|
||||||
if video.gcsUri:
|
if video.gcsUri:
|
||||||
return IO.NodeOutput(await download_url_to_video_output(video.gcsUri))
|
return IO.NodeOutput(await download_url_to_video_output(video.gcsUri))
|
||||||
raise Exception("Video returned but no data or URL was provided")
|
raise Exception("Video returned but no data or URL was provided")
|
||||||
|
|||||||
@ -8,10 +8,7 @@ import json
|
|||||||
from typing import Optional
|
from typing import Optional
|
||||||
from typing_extensions import override
|
from typing_extensions import override
|
||||||
from fractions import Fraction
|
from fractions import Fraction
|
||||||
from comfy_api.input import AudioInput, ImageInput, VideoInput
|
from comfy_api.latest import ComfyExtension, io, ui, Input, InputImpl, Types
|
||||||
from comfy_api.input_impl import VideoFromComponents, VideoFromFile
|
|
||||||
from comfy_api.util import VideoCodec, VideoComponents, VideoContainer
|
|
||||||
from comfy_api.latest import ComfyExtension, io, ui
|
|
||||||
from comfy.cli_args import args
|
from comfy.cli_args import args
|
||||||
|
|
||||||
class SaveWEBM(io.ComfyNode):
|
class SaveWEBM(io.ComfyNode):
|
||||||
@ -28,7 +25,6 @@ class SaveWEBM(io.ComfyNode):
|
|||||||
io.Float.Input("fps", default=24.0, min=0.01, max=1000.0, step=0.01),
|
io.Float.Input("fps", default=24.0, min=0.01, max=1000.0, step=0.01),
|
||||||
io.Float.Input("crf", default=32.0, min=0, max=63.0, step=1, tooltip="Higher crf means lower quality with a smaller file size, lower crf means higher quality higher filesize."),
|
io.Float.Input("crf", default=32.0, min=0, max=63.0, step=1, tooltip="Higher crf means lower quality with a smaller file size, lower crf means higher quality higher filesize."),
|
||||||
],
|
],
|
||||||
outputs=[],
|
|
||||||
hidden=[io.Hidden.prompt, io.Hidden.extra_pnginfo],
|
hidden=[io.Hidden.prompt, io.Hidden.extra_pnginfo],
|
||||||
is_output_node=True,
|
is_output_node=True,
|
||||||
)
|
)
|
||||||
@ -79,16 +75,15 @@ class SaveVideo(io.ComfyNode):
|
|||||||
inputs=[
|
inputs=[
|
||||||
io.Video.Input("video", tooltip="The video to save."),
|
io.Video.Input("video", tooltip="The video to save."),
|
||||||
io.String.Input("filename_prefix", default="video/ComfyUI", tooltip="The prefix for the file to save. This may include formatting information such as %date:yyyy-MM-dd% or %Empty Latent Image.width% to include values from nodes."),
|
io.String.Input("filename_prefix", default="video/ComfyUI", tooltip="The prefix for the file to save. This may include formatting information such as %date:yyyy-MM-dd% or %Empty Latent Image.width% to include values from nodes."),
|
||||||
io.Combo.Input("format", options=VideoContainer.as_input(), default="auto", tooltip="The format to save the video as."),
|
io.Combo.Input("format", options=Types.VideoContainer.as_input(), default="auto", tooltip="The format to save the video as."),
|
||||||
io.Combo.Input("codec", options=VideoCodec.as_input(), default="auto", tooltip="The codec to use for the video."),
|
io.Combo.Input("codec", options=Types.VideoCodec.as_input(), default="auto", tooltip="The codec to use for the video."),
|
||||||
],
|
],
|
||||||
outputs=[],
|
|
||||||
hidden=[io.Hidden.prompt, io.Hidden.extra_pnginfo],
|
hidden=[io.Hidden.prompt, io.Hidden.extra_pnginfo],
|
||||||
is_output_node=True,
|
is_output_node=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def execute(cls, video: VideoInput, filename_prefix, format: str, codec) -> io.NodeOutput:
|
def execute(cls, video: Input.Video, filename_prefix, format: str, codec) -> io.NodeOutput:
|
||||||
width, height = video.get_dimensions()
|
width, height = video.get_dimensions()
|
||||||
full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(
|
full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(
|
||||||
filename_prefix,
|
filename_prefix,
|
||||||
@ -105,10 +100,10 @@ class SaveVideo(io.ComfyNode):
|
|||||||
metadata["prompt"] = cls.hidden.prompt
|
metadata["prompt"] = cls.hidden.prompt
|
||||||
if len(metadata) > 0:
|
if len(metadata) > 0:
|
||||||
saved_metadata = metadata
|
saved_metadata = metadata
|
||||||
file = f"{filename}_{counter:05}_.{VideoContainer.get_extension(format)}"
|
file = f"{filename}_{counter:05}_.{Types.VideoContainer.get_extension(format)}"
|
||||||
video.save_to(
|
video.save_to(
|
||||||
os.path.join(full_output_folder, file),
|
os.path.join(full_output_folder, file),
|
||||||
format=VideoContainer(format),
|
format=Types.VideoContainer(format),
|
||||||
codec=codec,
|
codec=codec,
|
||||||
metadata=saved_metadata
|
metadata=saved_metadata
|
||||||
)
|
)
|
||||||
@ -135,9 +130,9 @@ class CreateVideo(io.ComfyNode):
|
|||||||
)
|
)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def execute(cls, images: ImageInput, fps: float, audio: Optional[AudioInput] = None) -> io.NodeOutput:
|
def execute(cls, images: Input.Image, fps: float, audio: Optional[Input.Audio] = None) -> io.NodeOutput:
|
||||||
return io.NodeOutput(
|
return io.NodeOutput(
|
||||||
VideoFromComponents(VideoComponents(images=images, audio=audio, frame_rate=Fraction(fps)))
|
InputImpl.VideoFromComponents(Types.VideoComponents(images=images, audio=audio, frame_rate=Fraction(fps)))
|
||||||
)
|
)
|
||||||
|
|
||||||
class GetVideoComponents(io.ComfyNode):
|
class GetVideoComponents(io.ComfyNode):
|
||||||
@ -159,11 +154,11 @@ class GetVideoComponents(io.ComfyNode):
|
|||||||
)
|
)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def execute(cls, video: VideoInput) -> io.NodeOutput:
|
def execute(cls, video: Input.Video) -> io.NodeOutput:
|
||||||
components = video.get_components()
|
components = video.get_components()
|
||||||
|
|
||||||
return io.NodeOutput(components.images, components.audio, float(components.frame_rate))
|
return io.NodeOutput(components.images, components.audio, float(components.frame_rate))
|
||||||
|
|
||||||
|
|
||||||
class LoadVideo(io.ComfyNode):
|
class LoadVideo(io.ComfyNode):
|
||||||
@classmethod
|
@classmethod
|
||||||
def define_schema(cls):
|
def define_schema(cls):
|
||||||
@ -185,7 +180,7 @@ class LoadVideo(io.ComfyNode):
|
|||||||
@classmethod
|
@classmethod
|
||||||
def execute(cls, file) -> io.NodeOutput:
|
def execute(cls, file) -> io.NodeOutput:
|
||||||
video_path = folder_paths.get_annotated_filepath(file)
|
video_path = folder_paths.get_annotated_filepath(file)
|
||||||
return io.NodeOutput(VideoFromFile(video_path))
|
return io.NodeOutput(InputImpl.VideoFromFile(video_path))
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def fingerprint_inputs(s, file):
|
def fingerprint_inputs(s, file):
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
comfyui-frontend-package==1.33.10
|
comfyui-frontend-package==1.33.10
|
||||||
comfyui-workflow-templates==0.7.25
|
comfyui-workflow-templates==0.7.51
|
||||||
comfyui-embedded-docs==0.3.1
|
comfyui-embedded-docs==0.3.1
|
||||||
torch
|
torch
|
||||||
torchsde
|
torchsde
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user