解释大模型为何会丢失语义，揭示连续计算如何自动形成离散语义结构。
How to Tame Your LLM: Semantic Collapse in Continuous Systems
- 将大模型视为连续状态机，用数学工具分析其语义演化过程。
- 证明语义会塌缩成有限个可解释的逻辑区域，对应真实语义概念。
- 适用于动态变化的模型，对理解模型稳定性有重要意义。
我们通过形式化大语言模型为连续状态机（CSMs）——一种在概率转移算子下演化的平滑动力系统——建立了大语言模型的语义动力学通用理论。相关转移算子 $P: L^2(M,μ) o L^2(M,μ)$ 编码了语义质量的传播。在适度正则性假设（紧性、遍历性、有界雅可比）下，$P$ 是紧算子且具有离散谱。在此框架下，我们证明了语义表征定理（SCT）：$P$ 的主特征函数诱导出有限个不变语义的谱盆地，每个均可在 $bR$ 上的极小结构中定义。因此，谱可约性与逻辑简洁性一致。这解释了离散符号语义如何从连续计算中涌现——连续激活流形会塌缩为有限、可解释的本体。我们进一步将SCT扩展至随机与绝热（时变）情形，证明缓慢漂移的核仍保持紧性、谱一致性与盆地结构。
原文摘要 · Abstract (English)
We develop a general theory of semantic dynamics for large language models by formalizing them as Continuous State Machines (CSMs): smooth dynamical systems whose latent manifolds evolve under probabilistic transition operators. The associated transfer operator $P: L^2(M,μ) \to L^2(M,μ)$ encodes the propagation of semantic mass. Under mild regularity assumptions (compactness, ergodicity, bounded Jacobian), $P$ is compact with discrete spectrum. Within this setting, we prove the Semantic Characterization Theorem (SCT): the leading eigenfunctions of $P$ induce finitely many spectral basins of invariant meaning, each definable in an o-minimal structure over $\mathbb{R}$. Thus spectral lumpability and logical tameness coincide. This explains how discrete symbolic semantics can emerge from continuous computation: the continuous activation manifold collapses into a finite, logically interpretable ontology. We further extend the SCT to stochastic and adiabatic (time-inhomogeneous) settings, showing that slowly drifting kernels preserve compactness, spectral coherence, and basin structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。