用自然语言生成可执行的可视化工作流,推动工业自动化落地。
Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language

- 基于真实业务流程构建评测集,支持直接部署到Dify、Coze平台。
- 现有大模型生成正确工作流的成功率不足,复杂需求下稳定性差。
- 提出智能体基线,提升6.05%解决率,仍存工业级应用差距。
目前可执行的可视化工作流已成为工业实际部署的主流范式,具备强可靠性与可控性。然而当前工作流几乎全靠人工设计:开发者需逐步编写提示词并反复修改逻辑,过程成本高、耗时长且易出错。为评估大语言模型是否能自动化这一多轮交互过程,我们提出Chat2Workflow基准,支持从自然语言直接生成可执行视觉工作流,并设计稳健的智能体基线以提升性能。该基准源自大量真实业务流程,每个实例均可转化为Dify、Coze等平台的直接部署方案。实验表明,尽管先进语言模型常能捕捉高层意图,但在复杂演化需求下仍难以生成正确、稳定且可执行的工作流。尽管智能体基线带来最高6.05%的解决率提升,现实差距依然显著,凸显该基准在推进工业级自动化中的基础价值。代码已开源:https://github.com/zjunlp/Chat2Workflow。
原文摘要 · Abstract (English)
At present, executable visual workflows have emerged as a mainstream paradigm in real-world industrial deployments, offering strong reliability and controllability. However, in current practice, such workflows are almost entirely constructed through manual engineering: developers must carefully design workflows, write prompts for each step, and repeatedly revise the logic as requirements evolve -- making development costly, time-consuming, and error-prone. To study whether large language models can automate this multi-round interaction process, we introduce Chat2Workflow, a benchmark for generating executable visual workflows directly from natural language, and propose a robust agentic baseline to improve performance. The benchmark is built from a large collection of real-world business workflows, with each instance designed so that the generated workflow can be transformed and directly deployed to practical workflow platforms such as Dify and Coze. Experimental results show that while state-of-the-art language models can often capture high-level intent, they struggle to generate correct, stable, and executable workflows, especially given complex and evolving requirements. Although our agentic baseline yields up to 6.05% resolve rate gains, the remaining real-world gap positions Chat2Workflow as a foundation for advancing industrial-grade automation. Code is available at https://github.com/zjunlp/Chat2Workflow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。