解决复杂AI系统中反馈信号混乱问题,提升深层链条的优化稳定性。
TextResNet: Decoupling and Routing Optimization Signals in Compound AI Systems via Deep Residual Tuning
- 通过语义增量保持梯度通路,实现信号精准传递。
- 分解反馈为独立子空间,解决上下文混淆问题。
- 动态分配资源至关键瓶颈,适合复杂智能体任务。
文本梯度优化器(TextGrad)可在复合AI系统中实现类梯度反馈传播,但在深层链路中表现不佳。根本原因在于扩展工作流中的语义纠缠问题:标准文本反向传播使局部批评与上游上下文混合,导致归属模糊。为此,我们提出TextResNet框架,通过四项创新重构优化过程,实现精确信号路由。首先,在前向传播中引入加性语义增量,保持梯度流动的恒等路径;其次,在反向传播中采用语义投影器进行语义梯度分解,将反馈解耦为因果独立子空间;第三,实施因果路由,将分解后的信号定向传递至对应组件;第四,执行密度感知优化调度,利用解耦信号动态分配资源至系统瓶颈。实验表明,TextResNet不仅显著优于TextGrad,且在基线崩溃的代理任务中展现出卓越稳定性。代码已开源:https://github.com/JeanDiable/TextResNet。
原文摘要 · Abstract (English)
Textual Gradient-style optimizers (TextGrad) enable gradient-like feedback propagation through compound AI systems. However, they do not work well for deep chains. The root cause of this limitation stems from the Semantic Entanglement problem in these extended workflows. In standard textual backpropagation, feedback signals mix local critiques with upstream contexts, leading to Attribution Ambiguity. To address this challenge, we propose TextResNet, a framework that reformulates the optimization process to achieve precise signal routing via four key innovations. Firstly, in the forward pass, it enforces Additive Semantic Deltas to preserve an Identity Highway for gradient flow. Secondly, in the backward pass, it introduces Semantic Gradient Decomposition via a Semantic Projector to disentangle feedback into causally independent subspaces. Thirdly, it implements Causal Routing, which routes projected signals to their specific components. Finally, it performs Density-Aware Optimization Scheduling to leverage the disentangled signals to dynamically allocate resources to key system bottlenecks. Our results show that TextResNet not only achieves superior performance compared to TextGrad, but also exhibits remarkable stability for agentic tasks in compound AI systems where baselines collapse. Code is available at https://github.com/JeanDiable/TextResNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。