让状态空间模型学会预判停止时机，提升推理效率与自我感知能力。
Architectural Proprioception in State Space Models: Thermodynamic Training Induces Anticipatory Halt Detection
- 用热力学损失训练模型，让其计算过程更节能高效。
- 发现状态熵与停止信心高度负相关，提前两步预判停止（相关系数-0.836）。
- 该能力仅存在于状态空间模型，适合需动态预算和自适应推理的系统。
我们提出概率导航架构（PNA），将神经计算视为在概率流形上的导航，受热力学原理支配。通过一种新型热力学损失函数训练状态空间模型（SSMs）和变压器（Transformers），该损失同时惩罚计算浪费与标准交叉熵。在19次实验中，我们发现热力学训练的SSMs发展出架构本体感知：递归状态熵与停止置信度之间存在强前瞻性耦合（r = -0.836, p < 0.001），其中停止信号领先状态熵坍塌恰好两个词元（tau = -2.0）。这一通用停止签名（USS）在不同随机种子下重现至小数点后四位，并可泛化至结构不同的排序任务。关键的是，同条件下训练的变压器无此耦合（r = -0.07），表明该现象具有架构依赖性。跨任务迁移实验确认，SSM的停止检测反映真实元认知（零样本转移F1：SSM 64.2% vs. Transformer 69.3%；后适应：SSM 94.5% vs. Transformer 86.4%），而变压器依赖句法模式匹配。在能量惩罚（alpha）与停止监督（beta）的二维超参数扫描中，发现该前瞻性耦合可通过训练连续调控，热力学压强是主要诱导机制，显式停止监督为放大器。结果表明，SSMs是热力学原生架构，其固定大小的递归状态天然支持马尔可夫压缩，从而实现计算自我意识，对成本感知推理、动态词元预算及基于置信度路由具有重要启示。
原文摘要 · Abstract (English)
We introduce the Probability Navigation Architecture (PNA) framework, which treats neural computation as navigation through a probability manifold governed by thermodynamic principles. We train State Space Models (SSMs) and Transformers with a novel thermodynamic loss function that penalizes computational waste alongside standard cross-entropy. Across 19 experimental phases, we discover that thermodynamically-trained SSMs develop architectural proprioception: a strong anticipatory coupling between recurrent state entropy and halt confidence (r = -0.836, p < 0.001) in which the halt signal leads state entropy collapse by exactly two tokens (tau = -2.0). This Universal Stopping Signature (USS) reproduces to four decimal places across random seeds and generalizes to a structurally distinct sorting task. Critically, Transformers trained identically show no such coupling (r = -0.07), demonstrating that the phenomenon is architecture-dependent. Cross-task transfer experiments confirm that SSM halt detection reflects genuine meta-cognition (zero-shot transfer F1: SSMs 64.2% vs. Transformers 69.3%; post-adaptation: SSMs 94.5% vs. Transformers 86.4%), while Transformer halt detection relies on syntactic pattern matching. A 2D hyperparameter sweep over energy penalty (alpha) and halt supervision (beta) reveals that the anticipatory coupling is continuously controllable through training, with thermodynamic pressure serving as the primary induction mechanism and explicit halt supervision as an amplifier. Our results establish that SSMs are thermodynamically native architectures whose fixed-size recurrent states naturally support the Markovian compression that enables computational self-awareness, with implications for cost-aware inference, dynamic token budgets, and confidence-based routing in production systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。