arXiv:2511.17399cs.LG2025-11NeurIPS

用后验采样构建稳定核心集,提升小样本训练的泛化能力。

Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes

  • 基于后验采样构建平滑损失函数,增强核心集稳定性。
  • 在高数据污染下仍保持良好代表性,训练速度更快。
  • 适合数据稀缺或噪声大的场景,尤其对小规模训练有效。

随着深度学习模型规模扩大,计算需求激增,高效核心集选择技术愈发重要。核心集旨在通过选取少量代表性数据加速训练,同时保持与全数据集相当的性能。目前主流的基于梯度的方法虽有良好理论基础和实际优势,但在数据预算有限时面临挑战:随机梯度下降(SGD)常成为强基线,且随时间推移因损失曲率不匹配导致代表性下降。本文提出新框架,首次建立后验采样与损失曲面之间的联系,使核心集在高数据污染下依然稳健。引入基于权重后验采样的平滑损失函数,提升稳定性与泛化性,同时保持高效计算。我们还提供了采样式核心集选择方法的新型收敛性分析。大量实验表明,该方法在多种数据集上均实现更快训练速度与更强泛化能力,优于当前最优方案。

原文摘要 · Abstract (English)

As deep learning models continue to scale, the growing computational demands have amplified the need for effective coreset selection techniques. Coreset selection aims to accelerate training by identifying small, representative subsets of data that approximate the performance of the full dataset. Among various approaches, gradient based methods stand out due to their strong theoretical underpinnings and practical benefits, particularly under limited data budgets. However, these methods face challenges such as naive stochastic gradient descent (SGD) acting as a surprisingly strong baseline and the breakdown of representativeness due to loss curvature mismatches over time. In this work, we propose a novel framework that addresses these limitations. First, we establish a connection between posterior sampling and loss landscapes, enabling robust coreset selection even in high data corruption scenarios. Second, we introduce a smoothed loss function based on posterior sampling onto the model weights, enhancing stability and generalization while maintaining computational efficiency. We also present a novel convergence analysis for our sampling-based coreset selection method. Finally, through extensive experiments, we demonstrate how our approach achieves faster training and enhanced generalization across diverse datasets than the current state of the art.

核心集后验采样训练加速泛化提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。