用统计场论分析LLM生成文本，发现温度调控下存在类似相变的临界现象。
Generative Criticality in Large Language Model Temperature Scaling
- 将令牌嵌入视为一维链上的连续自旋变量，通过温度调节研究其集体行为。
- 在临界温度附近出现敏感度峰值和语义方向坍缩，且内在维度达最小值。
- 适用于不同模型规模与提示类型，为解码策略提供新分析视角。
我们提出一种针对大语言模型（LLM）生成文本的统计场框架，将令牌嵌入视为一维链上的连续自旋变量。通过定义由两点关联函数导出的易感度和由系综平均嵌入场导出的序参量，调节softmax温度T，观察到在特征温度T_c附近出现尖锐的易感度峰值，具有类幂律标度；同时序参量发生急剧变化，并在T_c以下坍缩至单一语义方向。独立使用两近邻（TwoNN）方法估计的内在维度在T_c附近达到最小值，结果在不同模型规模（Qwen3: 0.6B–32B）和提示类别间均保持稳健。虽然该现象与连续相变高度相似，但自回归生成的非平衡特性仍需进一步研究。本框架为探测LLM输出的集体统计结构提供了量化工具，并暗示解码策略与临界现象间的潜在联系。
原文摘要 · Abstract (English)
We propose a statistical-field framework for text generated by large language models (LLMs), treating token embeddings as continuous spin variables on a one-dimensional chain. Defining a susceptibility from the connected two-point correlator and an order parameter from the ensemble-averaged embedding field, we vary the \texttt{softmax} temperature $T$ and observe a sharp susceptibility peak near a characteristic $T_c$ with power-law-like scaling, a concurrent rapid change in the order parameter, and a collapse onto a single semantic direction below $T_c$. The intrinsic dimension estimated by the two nearest neighbor (TwoNN) method independently corroborates these findings, reaching a minimum near $T_c$. Results are robust across model scales (Qwen3: 0.6B--32B) and prompt categories. While the phenomenology closely resembles a continuous phase transition, the non-equilibrium nature of autoregressive generation warrants further investigation. Our framework provides quantitative tools for probing the collective statistical structure of LLM outputs and suggests connections between decoding strategies and critical phenomena.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。