用连续空间梯度优化，让小模型也能高效完成复杂任务
ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization
- 在连续空间中用梯度优化替代离散搜索，提升灵活性与可扩展性
- 六项基准测试平均性能比现有方法高8.2%，小模型可超越大模型
- 适合追求低成本高效推理的AI系统开发者
近期研究利用大语言模型多智能体系统解决复杂问题，同时试图减少构建所需的手动工作量，推动了自动化智能体工作流优化方法的发展。然而，现有方法因表示能力有限、适应性差和可扩展性不足，仍显僵化，尤其依赖离散优化技术。我们提出ScoreFlow，一种简单但高性能的框架，通过在连续空间中进行高效的梯度优化来解决上述挑战。ScoreFlow引入Score-DPO，一种新型直接偏好优化变体，能处理量化反馈。在涵盖问答、编程和数学推理的六个基准上，ScoreFlow相比现有基线平均提升8.2%。此外，它使较小模型在更低推理成本下表现优于更大模型。
原文摘要 · Abstract (English)
Recent research has leveraged large language model multi-agent systems for complex problem-solving while trying to reduce the manual effort required to build them, driving the development of automated agent workflow optimization methods. However, existing methods remain inflexible due to representational limitations, a lack of adaptability, and poor scalability when relying on discrete optimization techniques. We address these challenges with ScoreFlow, a simple yet high-performance framework that leverages efficient gradient-based optimization in a continuous space. ScoreFlow incorporates Score-DPO, a novel variant of the direct preference optimization method that accounts for quantitative feedback. Across six benchmarks spanning question answering, coding, and mathematical reasoning, ScoreFlow achieves an 8.2% improvement over existing baselines. Moreover, it empowers smaller models to outperform larger ones with lower inference costs. Project: https://github.com/Gen-Verse/ScoreFlow
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。