提出高效算法计算点集度量大小,提升机器学习训练与聚类效果。
Approximating Metric Magnitude of Point Sets
- 将度量大小计算转化为凸优化问题,避免次模优化陷阱。
- 设计快速收敛的迭代算法和子集选择方法,显著降低计算成本。
- 适用于神经网络正则化与新型聚类分析,尤其适合大规模数据。
度量大小是衡量点云“规模”的几何特性,具备诸多优良性质,近年来被用于增强机器学习与优化算法。然而,当数据集庞大或需反复计算(如模型训练)时,其高计算成本限制了实际应用。本文研究度量大小的计算问题,证明其可转化为凸优化但非次模优化。提出两种新算法:一种快速收敛的迭代逼近法,另一种子集选择加速法。此前研究表明,随机梯度下降中模型序列的度量大小与泛化差距相关;利用本文更高效的算法扩展后发现,更长序列的相关性更高。此外,还揭示了度量大小在机器学习中的新应用:作为神经网络训练的有效正则项,以及一种新颖的聚类判据。
原文摘要 · Abstract (English)
Metric magnitude is a measure of the "size" of point clouds with many desirable geometric properties. It has been adapted to various mathematical contexts and recent work suggests that it can enhance machine learning and optimization algorithms. But its usability is limited due to the computational cost when the dataset is large or when the computation must be carried out repeatedly (e.g. in model training). In this paper, we study the magnitude computation problem, and show efficient ways of approximating it. We show that it can be cast as a convex optimization problem, but not as a submodular optimization. The paper describes two new algorithms - an iterative approximation algorithm that converges fast and is accurate, and a subset selection method that makes the computation even faster. It has been previously proposed that magnitude of model sequences generated during stochastic gradient descent is correlated to generalization gap. Extension of this result using our more scalable algorithms shows that longer sequences in fact bear higher correlations. We also describe new applications of magnitude in machine learning - as an effective regularizer for neural network training, and as a novel clustering criterion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。