arXiv:2601.21064cs.LGcs.AI2026-01中稿 · ICLR被引 1

解决深度AI系统中文本反馈失效问题,实现稳定高效优化。

Textual Equilibrium Propagation for Deep Compound AI Systems

  • 提出局部优化机制,通过平衡循环修正提示词直至收敛
  • 在长流程问答和多智能体任务中准确率显著提升,深度越大效果越明显
  • 无需反向传播,适合黑箱大模型,保持实用性和可扩展性

大型语言模型(LLMs)正被用于协调多个模块(如检索器、工具、验证器)的复合型AI系统,处理长时程工作流。现有全局文本反馈方法(如TextGrad)虽可行,但随系统深度增加性能下降。主要表现为:1)文本梯度爆炸,反馈随深度指数增长,导致消息过长并放大评估偏差;2)文本梯度消失,受限于长上下文能力,模型过度依赖部分反馈,且长反馈压缩导致下游信息逐渐失真。为此,我们提出文本均衡传播(TEP),受能量模型中均衡传播启发。TEP包含两个阶段:1)自由阶段,本地LLM评论者迭代优化提示词直至达到平衡(无进一步改进建议);2)微调阶段,采用有限强度的近端提示修改,通过前向信号而非反向反馈链传播任务目标。该设计支持局部提示优化与可控全局适应,避免全局文本反向传播带来的计算负担和信号衰减。在多个长时程问答基准和多智能体工具使用数据集上,TEP持续优于TextGrad等全局方法,增益随深度增大,同时保留深复合系统中黑箱模型的实用性。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly deployed as part of compound AI systems that coordinate multiple modules (e.g., retrievers, tools, verifiers) over long-horizon workflows. Recent approaches that propagate textual feedback globally (e.g., TextGrad) make it feasible to optimize such pipelines, but we find that performance degrades as system depth grows. In particular, long-horizon agentic workflows exhibit two depth-scaling failure modes: 1) exploding textual gradient, where textual feedback grows exponentially with depth, leading to prohibitively long message and amplifies evaluation biases; and 2) vanishing textual gradient, where limited long-context ability causes models overemphasize partial feedback and compression of lengthy feedback causes downstream messages to lose specificity gradually as they propagate many hops upstream. To mitigate these issues, we introduce Textual Equilibrium Propagation (TEP), a local learning principle inspired by Equilibrium Propagation in energy-based models. TEP includes two phases: 1) a free phase where a local LLM critics iteratively refine prompts until reaching equilibrium (no further improvements are suggested); and 2) a nudged phase which applies proximal prompt edits with bounded modification intensity, using task-level objectives that propagate via forward signaling rather than backward feedback chains. This design supports local prompt optimization followed by controlled adaptation toward global goals without the computational burden and signal degradation of global textual backpropagation. Across long-horizon QA benchmarks and multi-agent tool-use dataset, TEP consistently improves accuracy and efficiency over global propagation methods such as TextGrad. The gains grows with depth, while preserving the practicality of black-box LLM components in deep compound AI system.

AI系统提示优化长流程推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。