解决大模型对话中因注意力机制导致的决策僵化问题,提升自主推理稳定性。
Beyond the Attention Stability Boundary: Agentic Self-Synthesizing Reasoning Protocols

- 将规划与执行分离,构建元认知推理框架防止历史信息干扰
- 在多轮对话中实现715倍抗干扰能力提升,成功率从0.1%增至71.5%
- 适用于需要高可靠性推理的自动化任务,如客服、决策支持系统
随着大语言模型代理向自主数字同事演进,非线性多轮对话中的确定性目标导向成为架构瓶颈。本文识别并形式化了编码器-解码器自回归变换器中的系统性失效模式——注意力锁(Attention Latch),其表现为信息过压缩导致的历史上下文权重压制中途更新,使代理固守过时约束而无视明确反向指令。为此提出自合成推理协议(SSRP),一种元认知框架,实现高层架构规划(Architect)与逐轮执行(Executive)的离散分离。在MultiWOZ 2.2数据集上,通过9000条轨迹评估,引入新指标聚合枢轴准确率(APA),并验证其与'中间丢失'的U型曲线映射关系。实验设三类层级:基于近期性的简单检索、高熵标准操作流程,以及语义劫持的三跳多事实合成任务。结果实证定位注意力稳定边界:无状态的Vanilla ReAct基线在GPT 5.4上成功率跌至0.1%,而SSRP达到71.5%的成功率,实现715倍韧性提升。在Gemini 3.1 Pro、Claude Sonnet 4.6和DeepSeek V3.2上均获显著性能增益。审计证实:递归反思基线成功率达100%;等距压力测试证明可剥离位置偏倚达90%准确率;通过信息瓶颈原理与粒度消融正式化框架。过程完整性审计显示98.8%遵从性,揭示‘锚定悖论’——高稳定性模型因拒绝在检索-推理污染下幻觉生成而失败。
原文摘要 · Abstract (English)
As LLM agents transition to autonomous digital coworkers, maintaining deterministic goal-directedness in non-linear multi-turn conversations emerged as an architectural bottleneck. We identify and formalize a systemic failure mode termed the Attention Latch in decoder-only autoregressive Transformers. This phenomenon, a behavioral manifestation of Information Over-squashing, occurs when the cumulative probabilistic weight of historical context overrides mid-task updates, causing agents to remain anchored to obsolete constraints despite explicit contradictory instructions. We propose Self-Synthesizing Reasoning Protocols (SSRP), a metacognitive framework that implements a discrete separation between high-level architectural planning (Architect) and turn-by-turn procedural execution (Executive). We evaluate SSRP across 9K trajectories using the MultiWOZ 2.2 dataset and the Aggregate Pivot Accuracy (APA), a novel metric we validate by mapping its scores to the U-shaped 'Lost in the Middle' curve. We present 3 experimental tiers: a shallow recency-based retrieval pilot, a high-entropy SOP, and a semantic hijacked 3-hop Multi-Fact Synthesis task. Our results empirically locate the Attention Stability Boundary, where stateless Vanilla ReAct baselines for GPT 5.4 collapse to 0.1% success while SSRP achieves a 715X Resilience Lift. We demonstrate statistically significant gains across Gemini 3.1 Pro, Claude Sonnet 4.6 and DeepSeek V3.2. Audits confirm SSRP necessity by proving attentional lapse via a recursive reflexion baseline (100% success); decoupling the latch from positional bias through equidistant stress testing (90% accuracy); and formalizing SSRP via the Information Bottleneck principle and granularity ablations. Procedural Integrity audit (98.8% adherence) reveals a Grounding Paradox where high-stability models fail by refusing to hallucinate under retrieval-reasoning contamination.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。