通过损失轨迹对齐提升动态剪枝在噪声标签下的鲁棒性
Beyond Loss Values: Robust Dynamic Pruning via Loss Trajectory Alignment
- 基于样本损失变化轨迹计算动态对齐得分，识别噪声样本
- 在多种噪声场景下提升准确率，最高比现有方法高6.3%
- 无需改动模型或训练流程，可直接嵌入主流剪枝框架
现有动态数据剪枝方法在标签噪声环境下表现不佳，因其通常以单样本损失值作为排序依据，可能因噪声样本损失值高而被误保留，导致性能显著下降。为此，我们提出AlignPrune，一种针对噪声的鲁棒剪枝模块，引入基于损失轨迹的动态对齐得分（DAS），更准确识别噪声样本，提升剪枝有效性。该模块为简单高效的即插即用组件，可无缝集成至先进动态剪枝框架中，无需修改模型结构或训练流程，始终优于基线方法。在五个广泛使用的基准数据集上，针对多种噪声类型和剪枝比例的大量实验表明，AlignPrune能将准确率最高提升6.3%。结果为噪声数据下的剪枝提供了可推广的解决方案，推动真实场景学习研究。代码已公开于：https://github.com/leonqin430/AlignPrune。
原文摘要 · Abstract (English)
Existing dynamic data pruning methods often fail under noisy-label settings, as they typically rely on per-sample loss as the ranking criterion. This could mistakenly lead to preserving noisy samples due to their high loss values, resulting in significant performance drop. To address this, we propose AlignPrune, a noise-robust module designed to enhance the reliability of dynamic pruning under label noise. Specifically, AlignPrune introduces the Dynamic Alignment Score (DAS), which is a loss-trajectory-based criterion that enables more accurate identification of noisy samples, thereby improving pruning effectiveness. As a simple yet effective plug-and-play module, AlignPrune can be seamlessly integrated into state-of-the-art dynamic pruning frameworks, consistently outperforming them without modifying either the model architecture or the training pipeline. Extensive experiments on five widely-used benchmarks across various noise types and pruning ratios demonstrate the effectiveness of AlignPrune, boosting accuracy by up to 6.3\% over state-of-the-art baselines. Our results offer a generalizable solution for pruning under noisy data, encouraging further exploration of learning in real-world scenarios. Code is available at: https://github.com/leonqin430/AlignPrune.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。