通过损失轨迹相关性筛选关键训练样本，提升模型效率与泛化能力。
Finding the Muses: Identifying Coresets through Loss Trajectories
- 基于训练与验证损失轨迹的对齐度识别核心样本
- 在CIFAR-100和ImageNet-1k上精度媲美或超越现有方法（差异<1%）
- 适用于多种架构且计算成本极低，适合资源受限场景
深度学习模型在多个领域取得顶尖性能，但在实时或资源受限场景面临可扩展性挑战。为解决此问题，本文提出损失轨迹相关性（LTC）这一新型核心集选择指标，通过量化训练样本损失轨迹与验证集损失轨迹之间的对齐程度，识别出驱动泛化能力的关键训练样本，从而构建紧凑且具有代表性的子集。与传统方法相比，LTC无需额外计算和存储开销，可作为训练过程的副产品高效计算。在CIFAR-100和ImageNet-1k上的实验表明，LTC在精度上达到或超过当前最优核心集选择方法，差异始终低于1%。该方法还具备跨架构泛化能力，适用于ResNet、VGG、DenseNet及Swin Transformer等多种模型，性能下降小于2%。此外，LTC能以极低计算成本揭示训练动态，如识别样本间的协同与冲突行为。该框架为大规模核心集选择与高效数据优化提供了新路径。
原文摘要 · Abstract (English)
Deep learning models achieve state-of-the-art performance across domains but face scalability challenges in real-time or resource-constrained scenarios. To address this, we propose Loss Trajectory Correlation (LTC), a novel metric for coreset selection that identifies critical training samples driving generalization. $LTC$ quantifies the alignment between training sample loss trajectories and validation set loss trajectories, enabling the construction of compact, representative subsets. Unlike traditional methods with computational and storage overheads that are infeasible to scale to large datasets, $LTC$ achieves superior efficiency as it can be computed as a byproduct of training. Our results on CIFAR-100 and ImageNet-1k show that $LTC$ consistently achieves accuracy on par with or surpassing state-of-the-art coreset selection methods, with any differences remaining under 1%. LTC also effectively transfers across various architectures, including ResNet, VGG, DenseNet, and Swin Transformer, with minimal performance degradation (<2%). Additionally, LTC offers insights into training dynamics, such as identifying aligned and conflicting sample behaviors, at a fraction of the computational cost of traditional methods. This framework paves the way for scalable coreset selection and efficient dataset optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。