arXiv:2410.02145cs.LGmath.OC2024-10ICML被引 1

用无梯度切平面法训练深层神经网络,实现首个有收敛保证的深度主动学习。

Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes

  • 提出无梯度切平面法训练任意深度ReLU网络
  • 首次证明该方法在非凸模型上可实现可行集几何收缩
  • 适用于需理论保证的主动学习场景

主动学习旨在提升机器学习的样本效率。本文研究了一种基于新型无梯度切平面训练方法的主动学习方案,适用于任意深度的ReLU网络,并建立了收敛性理论。首次证明了传统用于线性模型的切平面算法可扩展至深层神经网络,尽管其存在非凸性和非线性决策边界。该训练方法首次实现了已知具有收敛保证的深度主动学习,揭示了可行集的几何收缩速率。通过合成数据实验和真实数据集上的情感分类任务,验证了所提方法相较于主流深度主动学习基线的有效性。

原文摘要 · Abstract (English)

Active learning methods aim to improve sample complexity in machine learning. In this work, we investigate an active learning scheme via a novel gradient-free cutting-plane training method for ReLU networks of arbitrary depth and develop a convergence theory. We demonstrate, for the first time, that cutting-plane algorithms, traditionally used in linear models, can be extended to deep neural networks despite their nonconvexity and nonlinear decision boundaries. Moreover, this training method induces the first deep active learning scheme known to achieve convergence guarantees, revealing a geometric contraction rate of the feasible set. We exemplify the effectiveness of our proposed active learning method against popular deep active learning baselines via both synthetic data experiments and sentimental classification task on real datasets.

主动学习深度学习优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。