将对称感知机转为教师-学生模型,实现任意样本密度下的可解推理。
The Symmetric Perceptron: a Teacher-Student Scenario
- 引入带标签的对称感知机新框架,构建可解的教师-学生问题
- 发现二阶不稳定性引发次优解,随后一阶相变实现完全对齐
- 揭示不同势函数下解的稳定性与优化算法的关系,适合研究学习动力学者
我们提出并求解了对称二元感知机的教师-学生模型,将传统以存储为主的模型转化为具有保证解的植入学问题,适用于任意样本密度。通过在两个区域中引入标签,改进了传统的u形或矩形势函数形式。分析无噪声情况下的贝叶斯最优及热噪声影响下的两种势函数/分类规则。利用高维极限下的退火与淬火自由熵计算,绘制出由样本密度α、原点到对称超平面距离κ及温度T三个控制参数决定的相图。识别出一种鲁棒学习机制:先由二阶不稳定性产生与教师相关的次优状态,再经一阶相变过渡至完全对齐。该结构依赖于势函数选择,并揭示了次优解的亚稳态与向植根配置熔化的相互作用,对基于蒙特卡洛的优化算法具有重要意义。
原文摘要 · Abstract (English)
We introduce and solve a teacher-student formulation of the symmetric binary Perceptron, turning a traditionally storage-oriented model into a planted inference problem with a guaranteed solution at any sample density. We adapt the formulation of the symmetric Perceptron which traditionally considers either the u-shaped potential or the rectangular one, by including labels in both regions. With this formulation, we analyze both the Bayes-optimal regime at for noise-less examples and the effect of thermal noise under two different potential/classification rules. Using annealed and quenched free-entropy calculations in the high-dimensional limit, we map the phase diagram in the three control parameters, namely the sample density $α$, the distance between the origin and one of the symmetric hyperplanes $κ$ and temperature $T$, and identify a robust scenario where learning is organized by a second-order instability that creates teacher-correlated suboptimal states, followed by a first-order transition to full alignment. We show how this structure depends on the choice of potential, the interplay between metastability of the suboptimal solution and its melting towards the planted configuration, which is relevant for Monte Carlo-based optimization algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。