arXiv:2607.01709cs.AIcs.LG2026-07被引 2

让图像生成工作流自动进化技能,提升稳定性和效率

COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflows

论文配图:COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflows
图 1 · 摘自论文原文
  • 将工作流构建视为带类型的图编辑,支持分阶段工具调用
  • 在六组配置中均优于基线,平均得分提升显著
  • 适合需要重复图像生成的开发者和设计师

随着代理在构建重复性任务工作流中的应用日益广泛,其记忆与可复用技能变得愈发关键:代理应能从过往运行中回忆工作流模式、执行约束和用户偏好。本文聚焦基于工作流的图像生成,提出COMFYCLAW,一种用于控制ComfyUI工作流的智能体技能演化框架。COMFYCLAW将工作流构建建模为带类型的图编辑,按构建阶段组织工具,自动回滚无效操作,并利用区域级视觉语言模型(VLM)验证器,将视觉失败转化为可操作的修复建议。该框架进一步演化一个逐步公开的技能库,将历史轨迹、执行错误及验证反馈提炼为可复用的代理技能。在四个基准划分、三种代理模型和两种图像骨干网络下,COMFYCLAW在全部六种配置中均取得最优平均图像生成评估得分,显著优于仅使用验证器而无技能演化的基线。人工标注结果也显示,标注者更偏好COMFYCLAW而非无技能演化的变体。结果表明,技能演化是提升代理在重复性视觉工作流构建中可靠性与性能的有效机制。

原文摘要 · Abstract (English)

Agents are increasingly used to construct workflows and assist humans in completing recurring tasks more efficiently. As these workflows become repeated and domain-specific, agent memory and reusable skills become increasingly important: agents should be able to recall workflow patterns, execution constraints, and user preferences from previous runs. We study this problem in workflow-based image generation and introduce COMFYCLAW, an agentic skill evolution harness for controlling ComfyUI workflows. COMFYCLAW formulates workflow construction as typed graph editing, exposes tools organized by construction stage, automatically reverts invalid edits, and uses a region-level vision-language model (VLM) verifier to translate visual failures into actionable repair suggestions. The framework further evolves a progressively disclosed skill library, where trajectories, execution errors, and verifier feedback from previous runs are distilled into reusable Agent Skills. Across four benchmark splits, three agent models, and two image backbones, COMFYCLAW achieves the best average image-generation evaluation score across all six agent configurations, outperforming a verifier-only baseline without skill evolution. Human annotations further show that annotators prefer COMFYCLAW over variants without skill evolution. Our results suggest that skill evolution is an effective mechanism for improving agent reliability and performance in recurring visual workflow construction.

图像生成智能体工作流优化技能演化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。