提出一种无需模型依赖的单步去噪方法，提升训练鲁棒性。
Debiased Orthogonal Boundary-Driven Efficient Noise Mitigation
- 利用高维正交性在锥空间构建清洁与噪声样本边界
- 单步推理即可评估样本噪声水平，降低计算开销
- 适用于多种模型和任务，部署简单且迁移能力强
在大规模预训练中，获取完全干净的人工标注样本往往不现实，因此缓解噪声标签对训练的负面影响变得至关重要。然而，现有去噪方法常因任务特定设计、模型依赖性及显著计算开销而难以实用。本文利用高维正交性，在锥空间中识别出一个稳健有效的边界，用于分离清洁与噪声样本。基于此，提出单步抗噪（OSA）方法，采用估计模型与评分函数，仅通过一次推理即可评估输入样本对的噪声程度。实验验证了OSA的优势：在多种基准、模型与任务上均展现出更强的训练鲁棒性、更好的任务迁移能力、更简化的部署流程以及更低的计算开销。代码已开源：https://github.com/leolee99/OSA。
原文摘要 · Abstract (English)
Mitigating the detrimental effects of noisy labels on the training process has become increasingly critical, as obtaining entirely clean or human-annotated samples for large-scale pre-training tasks is often impractical. Nonetheless, existing noise mitigation methods often encounter limitations in practical applications due to their task-specific design, model dependency, and significant computational overhead. In this work, we exploit the properties of high-dimensional orthogonality to identify a robust and effective boundary in cone space for separating clean and noisy samples. Building on this, we propose One-Step Anti-noise (OSA), a model-agnostic noisy label mitigation paradigm that employs an estimator model and a scoring function to assess the noise level of input pairs through just one-step inference. We empirically validate the superiority of OSA, demonstrating its enhanced training robustness, improved task transferability, streamlined deployment, and reduced computational overhead across diverse benchmarks, models, and tasks. Our code is released at https://github.com/leolee99/OSA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。