用变分法分析高斯混合数据下的感知机学习极限性能
Variational Bounds for Perceptron Learning from Structured Data
- 引入变分框架结合插值法与对数凹性，推导压力下界和上界
- 上下界仅在优化顺序上差异，条件满足时完全一致
- 统一求解基态能量、训练损失与泛化误差，适用于多种模型
我们提出一种针对高斯混合数据上训练的连续自旋感知机的有限温度变分方法。该模型支持广泛的凹效用函数及可分离的对数凹先验分布。通过结合插值法、对数凹性与集中估计，推导出极限冻压的上下界。令人惊讶的是，两个边界仅在两个变分参数的优化顺序上不同，其余极值均由变分势的凸-凹结构控制。当两种优化可交换时，上下界重合，精确确定了模型解。同一变分势同时给出不动点方程作为驻定条件，为基态能量、训练损失和泛化误差的计算提供了统一路径。
原文摘要 · Abstract (English)
We introduce a variational approach to a finite-temperature continuous-spin perceptron trained on a Gaussian mixture. The model allows for a broad class of concave utilities and log-concave separable prior measures on the spins. By combining the interpolation method with log-concavity and concentration estimates, we derive lower and upper minimax variational bounds for the limiting quenched pressure. Remarkably, the two bounds differ only in the order of optimization of two variational parameters, while all remaining extrema are controlled by the concave--convex structure of the variational potential. Whenever the two optimizations commute, the two bounds match and identify the solution of the model. The same potential yields the fixed-point equations as stationarity conditions and provides a unified route to the computation of the ground-state energy, training loss, and generalization error.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。