用统计物理方法分析高维极小极大问题的平衡特性。
Statistical Mechanics of Min-Max Problems
- 构建高维下极小极大问题的统计力学框架，正确定义极小与极大的顺序。
- 推导出训练数据量与泛化误差的关系，发现真实与虚假数据的最佳比例。
- 为生成对抗网络等模型提供理论基础，适合研究机器学习理论的研究者。
极小极大优化问题（又称鞍点问题）因其在公平波束成形、生成对抗网络（GANs）和对抗学习中的广泛应用而受到广泛关注。然而，理解这类问题的性质仍是一个重大挑战。本文引入一种统计力学形式化方法，用于分析高维极限下极小极大问题的平衡值，同时正确处理极小与极大操作的顺序。作为初步应用，该方法被用于双线性极小极大博弈和简单GANs，推导出训练数据量与泛化误差之间的关系，并指出有效学习时真实数据与伪造数据的最佳比例。该形式化为基于极小极大问题的各类机器学习方法的深层理论分析奠定了基础，推动了新算法与架构的发展。
原文摘要 · Abstract (English)
Min-max optimization problems, also known as saddle point problems, have attracted significant attention due to their applications in various fields, such as fair beamforming, generative adversarial networks (GANs), and adversarial learning. However, understanding the properties of these min-max problems has remained a substantial challenge. This study introduces a statistical mechanical formalism for analyzing the equilibrium values of min-max problems in the high-dimensional limit, while appropriately addressing the order of operations for min and max. As a first step, we apply this formalism to bilinear min-max games and simple GANs, deriving the relationship between the amount of training data and generalization error and indicating the optimal ratio of fake to real data for effective learning. This formalism provides a groundwork for a deeper theoretical analysis of the equilibrium properties in various machine learning methods based on min-max problems and encourages the development of new algorithms and architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。