动态数据流中高效筛选关键训练样本，提升模型泛化能力。
PEAKS: Selecting Key Training Examples Incrementally via Prediction Error Anchored by Kernel Similarity
- 基于特征空间相似性与预测误差双重锚定选择样本
- 在真实数据集上随数据量增大，性能超越随机选择
- 适用于持续学习、在线训练等动态场景
随着深度学习依赖的数据集不断增大，理解哪些样本对泛化最重要已成为关键问题。现有数据选择方法虽有进展，但新兴应用亟需在动态环境中研究该问题。为此，我们提出增量数据选择（IDS）问题：样本以连续流形式到达，且无法访问完整数据源。在此设定下，学习者需在不完全数据的前提下，逐步构建固定大小的训练集并同时学习任务。我们发现，在IDS中，新样本对模型状态的影响取决于其在特征空间中的几何关系及其预测误差。基于此洞察，我们提出PEAKS（Prediction Error Anchored by Kernel Similarity），一种专为IDS设计的高效数据选择方法。全面评估表明，PEAKS始终优于现有策略；且在真实数据集上，随着训练数据规模扩大，其相对于随机选择的性能优势愈发显著。代码已公开于https://github.com/BurakGurbuz97/PEAKS。
原文摘要 · Abstract (English)
As deep learning continues to be driven by ever-larger datasets, understanding which examples are most important for generalization has become a critical question. While progress in data selection continues, emerging applications require studying this problem in dynamic contexts. To bridge this gap, we pose the Incremental Data Selection (IDS) problem, where examples arrive as a continuous stream, and need to be selected without access to the full data source. In this setting, the learner must incrementally build a training dataset of predefined size while simultaneously learning the underlying task. We find that in IDS, the impact of a new sample on the model state depends fundamentally on both its geometric relationship in the feature space and its prediction error. Leveraging this insight, we propose PEAKS (Prediction Error Anchored by Kernel Similarity), an efficient data selection method tailored for IDS. Our comprehensive evaluations demonstrate that PEAKS consistently outperforms existing selection strategies. Furthermore, PEAKS yields increasingly better performance returns than random selection as training data size grows on real-world datasets. The code is available at https://github.com/BurakGurbuz97/PEAKS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。