温度越高，AI输出越有序，揭示其非线性动力学本质
Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs
- 通过调整解码温度，发现模型输出呈现熵先升后降的非线性演化
- 12,000次生成实验显示长期输出出现种群反转与周期性行为
- 可测量内部动态，为控制和干预生成过程提供新视角
普通多状态系统的温度升高会增加状态访问范围，从而提升熵。然而我们发现，在ChatGPT类AI中情况相反：尽管提高解码温度也扩大了下一个词的选择范围，但长期生成输出却经历熵最大值后进入种群反转。基于11个AI模型的12,000次续写实验，自回归反馈导致输出分布经过冻结态、周期、间歇性和噪声诱导有序等现象。我们发现一个隐藏坐标，可作为有效非线性映射的状态变量，其轨迹平均能强预测独立测试路径中的输出重复。因此，ChatGPT类AI并非简单的“随机鹦鹉”，而是一类可调控的新型非线性物理系统，其内部动态可被测量与扰动。
原文摘要 · Abstract (English)
Increasing the temperature of an ordinary many-state system increases access to a wider range of states and hence increases its entropy. We find the opposite in ChatGPT-like AIs, even though raising the decoder temperature likewise increases access to a wider range of states (next-token choices). Across 12,000 continuations from 11 AIs, autoregressive feedback drives the long-time output population through an entropy maximum and into population inversion. The transition features frozen states, cycles, intermittency and noise-induced ordering. We present evidence of a hidden coordinate that acts as the state variable of an effective nonlinear map. Its trajectory average strongly predicts output repetition in separate test trajectories. ChatGPT-like AIs therefore behave not as `stochastic parrots', but as a new class of controllable nonlinear physical systems whose internal dynamics can be measured and perturbed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。