揭示大模型反馈循环为何失效,并提出可验证的突破方法。
Closed-Loop Knowledge Dynamics: An Operational Framework for Saturation and Escape

- 构建三层动态框架,用结构参数控制知识演化路径。
- 发现反馈系统会收敛至稳定区域,噪声导致残留误差。
- 通过干预检测和概率下界,实现跨领域质量跃迁诊断。
反馈回路在大语言模型、强化学习和自主探索中推动迭代优化,但其增益常因持续内部反馈而趋于饱和。本文研究闭环知识系统饱和的原因及外部信息如何使其突破当前吸引子。提出一个三层次操作框架,知识状态 $x_t$ 通过依赖结构参数 $θ$ 的转移核 $K_θ$ 演化,其主控结构由核诱导的观测等价类定义,吸引子与吸引盆为固定 $θ$ 动态的属性。结构性干预改变 $θ$ 并在预设探测状态上引发可观测的核差异,使结构变化可被证伪。基于李雅普诺夫漂移条件,证明稳定内生动态趋近有界稳定区,瞬态呈指数衰减,残差受噪声控制。通过干预引起的吸引子位移度量与基准相对的KL下界,刻画逃逸概率提升机制。分析还解释了仅靠条件互信息无法确证逃逸的原因:它衡量的是干预后更新的变化,而非偏离无干预规律。在LLM代码修复、稀疏奖励强化学习与贝叶斯优化中的案例研究,采用匹配延续控制,说明反馈强度与对齐程度如何影响质量提升型逃逸。本工作建立稳定性工具、可观测干预效应与跨领域诊断之间的操作性关联。
原文摘要 · Abstract (English)
Feedback-driven loops support iterative improvement in large language models, reinforcement learning, and autonomous discovery, yet their gains often diminish under repeated internal feedback. We study why closed-loop knowledge systems saturate and what external information can move them beyond their current attractors. We introduce a three-level operational framework in which knowledge states $x_t$ evolve through transition kernels $K_θ$ indexed by a structural parameter $θ$. The governing structure is defined as the observational equivalence class of $θ$ induced by these kernels, while attractors and basins are properties of the fixed-$θ$ dynamics. A structural intervention changes $θ$ and produces a detectable kernel discrepancy on pre-specified probe states, making structural change falsifiable. Using a Lyapunov drift condition, we show that stable internal dynamics approach bounded stability regions with exponentially attenuated transients and a noise-controlled residual floor. We characterize escape through a metric condition on intervention-induced attractor displacement and a baseline-relative KL lower bound for increasing escape probability. This analysis also explains why conditional mutual information alone cannot certify escape: it measures variation among intervention-conditioned updates rather than departure from the no-intervention law. Case studies in LLM code repair, sparse-reward reinforcement learning, and Bayesian optimization use matched continuation controls to illustrate how feedback strength and alignment affect quality-improving escape. Our contribution is an operational connection among stability tools, measurable intervention effects, and cross-domain diagnostics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。