将隐式推理过程看作动态系统，揭示其内在演化规律
Interpreting Latent CoT Reasoning as Dynamical Systems
- 把隐藏层推理轨迹建模为表示空间中的动态路径
- 发现CODI稳定收敛，COCONUT持续发散，二者具不同稳定性特征
- 适合关注大模型推理可解释性的研究者与开发者
近期的隐式推理方法（如CODI和COCONUT）存在根本性可解释性难题：在每一步中，隐藏空间同时保留多个叠加的候选推理路径，与显式链式思维（explicit-CoT）的单一透明路径不同。现有机制分析仅揭示压缩、捷径和叠加现象，却未能说明推理如何随隐层步骤演变。为此，本文将隐式标记序列视为表示空间中的轨迹，采用动力系统分析刻画推理演化过程。通过量化指标（如步间变化、方向一致性、Lyapunov敏感度）与定性投影（如UMAP、DMD/PHATE），我们发现隐式CoT具有结构化、非随机的动力学特征，并呈现两种不同的稳定性类别：CODI表现为稳定吸引子，而COCONUT则表现为不稳定的发散系统；引入SIM-CoT监督可收紧两者行为，但不改变其底层动态。该框架提升了对隐式推理动态的理解，并为改进性能提供可操作洞察。代码与项目页面已公开。
原文摘要 · Abstract (English)
Recent latent reasoning methods, such as CODI and COCONUT, face a fundamental interpretability problem: they maintain multiple superimposed candidate traces in the hidden space at each step, unlike explicit- CoT, which follows a single transparent reasoning trace. Existing mechanistic methods show compression, shortcuts, and superposition without explaining how reasoning evolves across latent steps. To address this gap, we model latent token sequences as trajectories in representation space and apply dynamical systems analysis to characterize the evolution of reasoning. Using quantitative measures, such as step-to-step change, direction consistency, and Lyapunov sensitivity, alongside qualitative projections, such as UMAP and DMD/PHATE, we show that latent CoT exhibits structured, non-random dynamics with two distinct stability classes. CODI behaves as a stable attractor, while COCONUT behaves as an unstable expanding system, and SIM-CoT supervision tightens both behaviors without changing the underlying dynamics. This framework advances the interpretability of latent CoT reasoning dynamics and provides actionable insights for improving latent reasoning performance. Code1 and Project page2 available online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。