提出两种无需调参的快速混合数据增强算法,适用于小样本分类。
Linearly Convergent Mixup Learning
- 设计不依赖学习率的双优化算法,直接处理混合标签
- 收敛迭代次数与计算成本均随数据量线性增长
- 在多种损失函数下提升预测性能,适合小数据场景
在再生核希尔伯特空间(RKHS)中的学习,如支持向量机,被公认为一种有前景的技术,在训练数据有限或计算资源受限的场景中仍具高度有效性与竞争力。这类方法尤其因其在小数据集上的表现和可解释性而受青睐。为应对数据不足问题,深度学习中广泛使用的mixup数据增强技术难以应用于RKHS学习,因其中间类标签的生成导致问题。尽管梯度下降方法能有效处理这些标签,但对偶优化方法通常无法直接适用。本文提出两种新算法,可扩展至更广泛的二分类模型。与基于梯度的方法不同,本算法无需学习率等超参数,简化实现与优化。算法的收敛迭代次数及每轮计算成本均与数据规模呈线性关系。数值实验表明,所提算法比梯度下降更快收敛至最优解,且在多种损失函数下,mixup增强均持续提升预测性能。
原文摘要 · Abstract (English)
Learning in the reproducing kernel Hilbert space (RKHS) such as the support vector machine has been recognized as a promising technique. It continues to be highly effective and competitive in numerous prediction tasks, particularly in settings where there is a shortage of training data or computational limitations exist. These methods are especially valued for their ability to work with small datasets and their interpretability. To address the issue of limited training data, mixup data augmentation, widely used in deep learning, has remained challenging to apply to learning in RKHS due to the generation of intermediate class labels. Although gradient descent methods handle these labels effectively, dual optimization approaches are typically not directly applicable. In this study, we present two novel algorithms that extend to a broader range of binary classification models. Unlike gradient-based approaches, our algorithms do not require hyperparameters like learning rates, simplifying their implementation and optimization. Both the number of iterations to converge and the computational cost per iteration scale linearly with respect to the dataset size. The numerical experiments demonstrate that our algorithms achieve faster convergence to the optimal solution compared to gradient descent approaches, and that mixup data augmentation consistently improves the predictive performance across various loss functions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。