arXiv:2503.17671cs.MAcs.AI2025-03被引 13

用多智能体自动生成ComfyUI图像工作流,更准更快。

ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation

  • 四个专用智能体协作,分步构建工作流节点连接。
  • 在13,571对数据上训练,生成准确率显著提升。
  • 适合需要高效设计AI绘图流程的用户和开发者。

ComfyUI 是一个流行的基于工作流的界面,通过直观的节点系统让用户自定义图像生成任务。然而,管理节点连接和多样模块的复杂性对用户构成挑战。本文提出 ComfyGPT,一个自优化的多智能体系统,可基于任务描述自动生成 ComfyUI 工作流。其核心创新包括:(1) 由四个专用智能体组成的多智能体系统——ReformatAgent、FlowAgent、RefineAgent 与 ExecuteAgent;(2) 聚焦生成精确的节点连接而非完整工作流,提升生成准确性;(3) 通过强化学习优化工作流生成。我们还构建了 FlowDataset(包含13,571个工作流-描述对)和 FlowBench 基准测试集,并提出四项新评估指标:格式验证(FV)、通过率(PA)、指令对齐度(PIA)和节点多样性(PND)。实验表明,ComfyGPT 显著优于现有基于大模型的方法,在该领域迈出重要一步。代码已开源。

原文摘要 · Abstract (English)

ComfyUI is a popular workflow-based interface that allows users to customize image generation tasks through an intuitive node-based system. However, the complexity of managing node connections and diverse modules can be challenging for users. In this paper, we introduce ComfyGPT, a self-optimizing multi-agent system designed to generate ComfyUI workflows based on task descriptions automatically. The key innovations of ComfyGPT include: (1) consisting of four specialized agents to build a multi-agent workflow generation system: ReformatAgent, FlowAgent, RefineAgent, and ExecuteAgent; (2) focusing on generating precise node connections instead of entire workflows, improving generation accuracy; and (3) enhancing workflow generation through reinforcement learning. Moreover, we introduce FlowDataset, a large-scale dataset containing 13,571 workflow-description pairs, and FlowBench, a comprehensive benchmark for evaluating workflow generation systems. Additionally, we propose four novel evaluation metrics: Format Validation (FV), Pass Accuracy (PA), Pass Instruct Alignment (PIA), and Pass Node Diversity (PND). Experimental results demonstrate that ComfyGPT significantly outperforms existing LLM-based methods in workflow generation, making it a significant step forward in this field. Code is avaliable at https://github.com/comfygpt/comfygpt.

多智能体图像生成自动化ComfyUI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。