发现短距依赖语言模型仍存相变，说明语言本质促发相变而非长程关联。
Phase transition on a context-sensitive random language model with short range interactions
- 构建上下文敏感的短距交互语言模型，符合乔姆斯基层级中的上下文有关文法。
- 即使上下文长度固定，模型仍出现有限温相变，表明相变源于语言内在特性。
- 为语言模型相变机制提供新解释，适合对语言与统计物理交叉研究者阅读。
自E. DeGiuli提出随机语言模型以来，语言模型已从统计力学视角被广泛研究。近期，具有符号间长程相互作用的模型中数值证明了贝雷津-科斯特里茨-索利斯（Berezinskii--Kosterlitz--Thouless）相变的存在。在统计力学中，长程相互作用可引发相变，因此语言模型中的相变是否源于真正的语言特性尚不明确。本研究构建了一个具有短程相互作用的随机语言模型，并数值分析其统计性质。该模型属于乔姆斯基层级中的上下文敏感文法，支持显式上下文引用。研究发现，即使上下文长度随句子长度保持不变，仍存在相变现象。这表明语言模型中的有限温度相变确实由语言固有性质驱动，而非长程相互作用所致。
原文摘要 · Abstract (English)
Since the random language model was proposed by E. DeGiuli [Phys. Rev. Lett. 122, 128301], language models have been investigated intensively from the viewpoint of statistical mechanics. Recently, the existence of a Berezinskii--Kosterlitz--Thouless transition was numerically demonstrated in models with long-range interactions between symbols. In statistical mechanics, it has long been known that long-range interactions can induce phase transitions. Therefore, it has remained unclear whether phase transitions observed in language models originate from genuinely linguistic properties that are absent in conventional spin models. In this study, we construct a random language model with short-range interactions and numerically investigate its statistical properties. Our model belongs to the class of context-sensitive grammars in the Chomsky hierarchy and allows explicit reference to contexts. We find that a phase transition occurs even when the model refers only to contexts whose length remains constant with respect to the sentence length. This result indicates that finite-temperature phase transitions in language models are genuinely induced by the intrinsic nature of language, rather than by long-range interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。