将两层ReLU网络训练转为可解的凸优化问题，提升模型可解释性。
Convex Formulations for Training Two-Layer ReLU Neural Networks
- 将无限宽两层ReLU网络训练转化为有限维凸完全正问题
- 引入半定松弛，多项式时间求解且测试准确率表现优异
- 适合关注模型可解释性与理论保障的研究者
训练机器学习模型中的非凸、NP难优化问题是关键挑战，但常导致黑箱模型。尽管凸方法已用于验证神经网络鲁棒性，其在训练中的应用仍不充分。本文将无限宽两层ReLU网络的训练问题重构成有限维（升维）空间中的凸完全正规划。虽凸性存在，但完全正约束使问题仍为NP难。为此，提出一种半定松弛，可在多项式时间内求解。实验评估显示该松弛紧致性良好，在多种分类任务中测试准确率表现具有竞争力。
原文摘要 · Abstract (English)
Solving non-convex, NP-hard optimization problems is crucial for training machine learning models, including neural networks. However, non-convexity often leads to black-box machine learning models with unclear inner workings. While convex formulations have been used for verifying neural network robustness, their application to training neural networks remains less explored. In response to this challenge, we reformulate the problem of training infinite-width two-layer ReLU networks as a convex completely positive program in a finite-dimensional (lifted) space. Despite the convexity, solving this problem remains NP-hard due to the complete positivity constraint. To overcome this challenge, we introduce a semidefinite relaxation that can be solved in polynomial time. We then experimentally evaluate the tightness of this relaxation, demonstrating its competitive performance in test accuracy across a range of classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。