揭示大模型迭代过程中的几何动态规律,可预测并控制行为演化。
Geometric Dynamics of Agentic Loops in Large Language Models
- 将迭代大模型视为语义空间中的动力系统,定义轨迹与吸引子。
- 发现三种可分类的动态:收敛、振荡、探索,且结果可量化。
- 提示设计直接影响动态类型,为系统可控性提供新思路。
迭代式大模型系统(如自修正、思维链、自主代理)日益广泛应用,但其时间动态尚未被刻画。以往研究仅关注任务在收敛时的表现,忽略了过程:语义内容如何随迭代演变?是否趋于稳定、漂移或振荡?若不回答这些问题,就无法预测系统行为、保证稳定性或系统化设计迭代架构。本文将代理循环形式化为语义空间中的离散动力系统,借鉴动力系统理论,定义轨迹、吸引子与动力学范式,提出适用于该场景的几何框架。实验验证表明,迭代改写产生收缩动力学,出现可测量的吸引子且分散度下降;而迭代否定则呈现探索型动力学,无稳定结构。关键发现是:提示设计直接决定动力学范式——相同模型因变换方式不同,表现出根本不同的几何行为。本工作确立了迭代大模型动态具有可预测性和可控制性,为稳定性分析、轨迹预测及复合循环的合理设计开辟新方向。
原文摘要 · Abstract (English)
Iterative LLM systems(self-refinement, chain-of-thought, autonomous agents) are increasingly deployed, yet their temporal dynamics remain uncharacterized. Prior work evaluates task performance at convergence but ignores the trajectory: how does semantic content evolve across iterations? Does it stabilize, drift, or oscillate? Without answering these questions, we cannot predict system behavior, guarantee stability, or systematically design iterative architectures. We formalize agentic loops as discrete dynamical systems in semantic space. Borrowing from dynamical systems theory, we define trajectories, attractors and dynamical regimes for recursive LLM transformations, providing rigorous geometric definitions adapted to this setting. Our framework reveals that agentic loops exhibit classifiable dynamics: contractive (convergence toward stable semantic attractors), oscillatory (cycling among attractors), or exploratory (unbounded divergence). Experiments on singular loops validate the framework. Iterative paraphrasing produces contractive dynamics with measurable attractor formation and decreasing dispersion. Iterative negation produces exploratory dynamics with no stable structure. Crucially, prompt design directly controls the dynamical regime - the same model exhibits fundamentally different geometric behaviors depending solely on the transformation applied. This work establishes that iterative LLM dynamics are predictable and controllable, opening new directions for stability analysis, trajectory forecasting, and principled design of composite loops that balance convergence and exploration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。