研究矩阵版随机镜像下降，揭示其在多输出问题中的收敛与偏好机制。
Implicit Bias and Convergence of Matrix Stochastic Mirror Descent
- 用矩阵镜像函数设计更新规则，适配多分类与矩阵补全任务。
- 在参数超样本情况下，算法指数级收敛到数据插值解。
- 首次证明矩阵SMD会趋向最小化特定散度的唯一解，适合高维多输出场景。
我们研究了具有矩阵参数和向量预测的随机镜像下降（SMD），该框架适用于多分类与矩阵补全问题。聚焦于参数量超过训练样本数的过参数化情形，我们证明使用矩阵镜像函数ψ(·)的SMD能指数级收敛至全局插值解。进一步地，我们推广了经典向量SMD的隐式偏差结果，表明矩阵SMD收敛于在满足数据插值条件下，使由ψ(·)诱导的Bregman散度最小化的唯一解。这些发现揭示了矩阵镜像映射如何在高维多输出问题中决定归纳偏置。
原文摘要 · Abstract (English)
We investigate Stochastic Mirror Descent (SMD) with matrix parameters and vector-valued predictions, a framework relevant to multi-class classification and matrix completion problems. Focusing on the overparameterized regime, where the total number of parameters exceeds the number of training samples, we prove that SMD with matrix mirror functions $ψ(\cdot)$ converges exponentially to a global interpolator. Furthermore, we generalize classical implicit bias results of vector SMD by demonstrating that the matrix SMD algorithm converges to the unique solution minimizing the Bregman divergence induced by $ψ(\cdot)$ from initialization subject to interpolating the data. These findings reveal how matrix mirror maps dictate inductive bias in high-dimensional, multi-output problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。