arXiv:2412.03624cs.AIcs.CL2024-12被引 18

让语言智能体系统自动优化,只需一次反馈就能精准调整各模块。

How to Correctly do Semantic Backpropagation on Language-based Agentic Systems

  • 用语义梯度统一多种优化方法,精确计算每个模块的改进方向。
  • 在BIG-Bench Hard和GSM8K上超越现有最优方法,提升显著。
  • 代码开源,适合研究智能体系统自动优化的开发者使用。

基于语言的智能体系统近年来展现出巨大潜力,从解决小型研究问题发展到部署于复杂现实任务。然而,优化这些系统通常需要大量人工干预。近期研究证明,此类系统可建模为计算图,从而实现自动优化。尽管如此,当前大多数基于图的智能体系统优化(GASO)方法在输出反馈时无法正确分配梯度给系统组件。为此,我们形式化了语义反向传播的概念,引入语义梯度——一种推广性框架,统一了反向模式自动微分与TextGrad等关键技术,通过共享后继节点的关系来传递方向性信息。该方法能有效指导每个组件如何调整以改善系统输出。我们提出语义梯度下降法,显著提升了GASO求解效果。在BIG-Bench Hard和GSM8K上的实验表明,本方法优于现有最先进方法。针对LIAR数据集的消融研究验证了方法的简洁高效。完整实现已公开于https://github.com/HishamAlyahya/semantic_backprop。

原文摘要 · Abstract (English)

Language-based agentic systems have shown great promise in recent years, transitioning from solving small-scale research problems to being deployed in challenging real-world tasks. However, optimizing these systems often requires substantial manual labor. Recent studies have demonstrated that these systems can be represented as computational graphs, enabling automatic optimization. Despite these advancements, most current efforts in Graph-based Agentic System Optimization (GASO) fail to properly assign feedback to the system's components given feedback on the system's output. To address this challenge, we formalize the concept of semantic backpropagation with semantic gradients -- a generalization that aligns several key optimization techniques, including reverse-mode automatic differentiation and the more recent TextGrad by exploiting the relationship among nodes with a common successor. This serves as a method for computing directional information about how changes to each component of an agentic system might improve the system's output. To use these gradients, we propose a method called semantic gradient descent which enables us to solve GASO effectively. Our results on both BIG-Bench Hard and GSM8K show that our approach outperforms existing state-of-the-art methods for solving GASO problems. A detailed ablation study on the LIAR dataset demonstrates the parsimonious nature of our method. A full copy of our implementation is publicly available at https://github.com/HishamAlyahya/semantic_backprop

智能体系统语义梯度自动优化反向传播

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。