用树状规划与反馈机制提升生成系统稳定性,让开源工具更强大
ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback
- 将节点图抽象为自然语言模块,支持高层组合与错误减少
- 通过分层决策与局部反馈,实现生成过程的动态修正
- 在多个评测中表现接近闭源模型,适合复杂任务开发
随着生成模型快速发展,统一多模态任务的通用生成成为焦点。然而现有开源框架因缺乏结构化工作流规划与执行反馈,仍易失效,难以支撑复杂现实应用。为此,我们提出ComfyMind,基于ComfyUI平台构建的协作式AI系统,具备两大创新:语义工作流接口(SWI)将底层节点图抽象为自然语言描述的功能模块,支持高层组合并减少结构错误;基于搜索树的规划机制结合局部反馈执行,将生成建模为分层决策过程,可在各阶段自适应修正。在ComfyBench、GenEval和Reason-Edit三个公开基准上评估,ComfyMind持续优于现有开源基线,性能接近GPT-Image-1。该系统为开源通用生成AI的发展提供了可行路径。
原文摘要 · Abstract (English)
With the rapid advancement of generative models, general-purpose generation has gained increasing attention as a promising approach to unify diverse tasks across modalities within a single system. Despite this progress, existing open-source frameworks often remain fragile and struggle to support complex real-world applications due to the lack of structured workflow planning and execution-level feedback. To address these limitations, we present ComfyMind, a collaborative AI system designed to enable robust and scalable general-purpose generation, built on the ComfyUI platform. ComfyMind introduces two core innovations: Semantic Workflow Interface (SWI) that abstracts low-level node graphs into callable functional modules described in natural language, enabling high-level composition and reducing structural errors; Search Tree Planning mechanism with localized feedback execution, which models generation as a hierarchical decision process and allows adaptive correction at each stage. Together, these components improve the stability and flexibility of complex generative workflows. We evaluate ComfyMind on three public benchmarks: ComfyBench, GenEval, and Reason-Edit, which span generation, editing, and reasoning tasks. Results show that ComfyMind consistently outperforms existing open-source baselines and achieves performance comparable to GPT-Image-1. ComfyMind paves a promising path for the development of open-source general-purpose generative AI systems. Project page: https://github.com/LitaoGuo/ComfyMind
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。