用热力学原理重新理解机器学习中的分布偏移问题
Laws of thermodynamics for exponential families
- 将学习问题转化为最大熵与统计力学框架
- 揭示了功、热等概念在统计学习中的对应关系
- 适合研究分布偏移与模型泛化的从业者
我们以一般指数族为框架，构建了热力学定律的表述。通过将学习（对数损失最小化）问题置于最大熵和统计力学的语境中，可将热力学结果映射到学习场景。扩展了指数族在热力学与学习平衡中的经典表征方式。功与热的基本概念，以及热力学循环与能量均分等高级概念，在人工智能与统计学中找到了精确且有用的对应。这些思想对量化和应对分布偏移具有广泛意义。
原文摘要 · Abstract (English)
We develop the laws of thermodynamics in terms of general exponential families. By casting learning (log-loss minimization) problems in max-entropy and statistical mechanics terms, we translate thermodynamics results to learning scenarios. We extend the well-known way in which exponential families characterize thermodynamic and learning equilibria. Basic ideas of work and heat, and advanced concepts of thermodynamic cycles and equipartition of energy, find exact and useful counterparts in AI / statistics terms. These ideas have broad implications for quantifying and addressing distribution shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。