用秩1格点压缩数据，加速机器学习损失计算
Data Compression using Rank-1 Lattices for Parameter Estimation in Machine Learning
- 用秩1格点对大数据集降维，保留关键信息
- 压缩后损失函数计算速度显著提升，收敛率可任意高
- 适合处理大规模数据的迭代优化任务
均方误差及其正则化版本是监督学习中的标准损失函数。然而，在大规模数据集上计算这些损失函数非常耗时。受J. Dick和M. Feischl（Journal of Complexity 67, 2021）方法启发，本文提出基于秩1格点的数据压缩算法，将大规模数据集缩减为更小规模。秩1格点是准蒙特卡洛（QMC）点集，若精心选择，可在多维单位立方体中均匀分布。预处理阶段通过为每个格点分配一对权重（基于原始数据与响应），体现其相对重要性。压缩后的数据使优化过程中的迭代损失计算大幅提速。我们分析了该QMC数据压缩算法的误差及预处理成本，针对傅里叶系数衰减足够快、属于特定威纳代数或柯罗博夫空间的函数进行研究。特别地，证明只要函数足够光滑，该方法即可实现任意高的收敛率。
原文摘要 · Abstract (English)
The mean squared error and regularized versions of it are standard loss functions in supervised machine learning. However, calculating these losses for large data sets can be computationally demanding. Modifying an approach of J. Dick and M. Feischl [Journal of Complexity 67 (2021)], we present algorithms to reduce extensive data sets to a smaller size using rank-1 lattices. Rank-1 lattices are quasi-Monte Carlo (QMC) point sets that are, if carefully chosen, well-distributed in a multidimensional unit cube. The compression strategy in the preprocessing step assigns every lattice point a pair of weights depending on the original data and responses, representing its relative importance. As a result, the compressed data makes iterative loss calculations in optimization steps much faster. We analyze the errors of our QMC data compression algorithms and the cost of the preprocessing step for functions whose Fourier coefficients decay sufficiently fast so that they lie in certain Wiener algebras or Korobov spaces. In particular, we prove that our approach can lead to arbitrary high convergence rates as long as the functions are sufficiently smooth.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。