通过块级诊断优化大模型智能体流程，提升效率与可解释性。
JudgeFlow: Agentic Workflow Optimization via Block Judge
- 将流程拆分为可复用逻辑块，实现细粒度诊断
- 失败运行中定位问题块，得分最高者优先优化
- 适合需要高效迭代的复杂智能体系统开发
优化基于大模型的智能体工作流对拓展AI能力至关重要。现有方法依赖粗粒度端到端评估信号，缺乏精确定位优化方向的能力，常导致低效或无效修改。为此，我们提出JudgeFlow：一个评估-判断-优化-更新的闭环流程。将可复用、可配置的逻辑块嵌入智能体流程以捕捉基础逻辑形式，并设计专用判断模块，分析执行轨迹（特别是失败案例），为有问题的块分配基于排名的责任分数。这些细粒度诊断信号由大模型优化器使用，聚焦于责任分最高的块进行修改。该方法提升了样本效率，增强了块级可解释性，并为自动化复杂智能体工作流提供了可扩展基础。我们在数学推理和代码生成基准上验证了JudgeFlow，结果表明其性能与效率均优于现有方法。
原文摘要 · Abstract (English)
Optimizing LLM-based agentic workflows is challenging for scaling AI capabilities. Current methods rely on coarse, end-to-end evaluation signals and lack fine-grained signals on where to refine, often resulting in inefficient or low-impact modifications. To address these limitations, we propose JudgeFlow, an Evaluation-Judge-Optimization-Update pipeline. We incorporate reusable, configurable logic blocks into agentic workflows to capture fundamental forms of logic. On top of this abstraction, we design a dedicated Judge module that inspects execution traces particularly failed runs and assigns rank-based responsibility scores to problematic blocks. These fine-grained diagnostic signals are then leveraged by an LLM-based optimizer, which focuses modifications on the most problematic block in the workflow. Our approach improves sample efficiency, enhances interpretability through block-level diagnostics, and provides a scalable foundation for automating increasingly complex agentic workflows. We evaluate JudgeFlow on mathematical reasoning and code generation benchmarks, where JudgeFlow achieves superior performance and efficiency compared to existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。