用统计力学类比模型复杂度与损失，揭示过拟合临界点
Loss-Complexity Landscape and Model Structure Functions
- 将结构函数与自由能通过勒让德-芬赫尔对偶关联
- 复杂度方差峰值出现在损失-复杂度权衡处，对应相变点
- 适用于理解线性与树模型的泛化能力边界
我们构建了柯尔莫哥洛夫结构函数 $h_x(α)$ 的对偶框架，引入可计算的复杂度代理量。建立了信息论构造与统计力学之间的数学类比，提出合适的配分函数和自由能泛函。明确证明了结构函数与自由能之间的勒让德-芬赫尔对偶关系，展示了马尔可夫核的细致平衡，并将接受概率解释为信息论意义上的散射振幅。模型复杂度的类似敏感度的方差被证明在损失-复杂度权衡点精确达到峰值，可被解读为相变。在线性回归与基于树的回归模型上的实际实验验证了这些理论预测，清晰展现了模型复杂度、泛化性能与过拟合阈值之间的相互作用。
原文摘要 · Abstract (English)
We develop a framework for dualizing the Kolmogorov structure function $h_x(α)$, which then allows using computable complexity proxies. We establish a mathematical analogy between information-theoretic constructs and statistical mechanics, introducing a suitable partition function and free energy functional. We explicitly prove the Legendre-Fenchel duality between the structure function and free energy, showing detailed balance of the Metropolis kernel, and interpret acceptance probabilities as information-theoretic scattering amplitudes. A susceptibility-like variance of model complexity is shown to peak precisely at loss-complexity trade-offs interpreted as phase transitions. Practical experiments with linear and tree-based regression models verify these theoretical predictions, explicitly demonstrating the interplay between the model complexity, generalization, and overfitting threshold.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。