动态调整样本权重，让模型自动关注难样本，提升噪声数据下的训练效果。
LiLAW: Lightweight Learnable Adaptive Weighting to Learn Sample Difficulty & Improve Noisy Training
- 用三个可学习参数动态划分样本难易，自适应调整损失权重。
- 在高噪声下准确率与AUROC显著提升，尤其在医学图像等复杂场景。
- 无需干净验证集，计算开销小，适合资源受限环境使用。
在噪声数据和数据异构性挑战下训练深度神经网络仍具难度。本文提出轻量级可学习自适应加权方法（LiLAW），基于样本动态难度（易、中、难）调整其损失权重，仅需三个全局可学习标量参数。每轮训练后，通过单次梯度下降更新参数，利用验证小批量数据完成自适应优化，无需依赖无偏验证集。在通用及医学影像数据集上，涵盖多种噪声类型与强度、损失函数、模型架构（含线性探测与全微调），均显示一致性能提升，尤其在高噪声条件下表现突出。结合合成与增强数据（SynPAIN、GAITGen、ECG5000）取得当前最优结果，并在Adult数据集上提升公平性。该方法轻量、实用且计算高效，适用于各类深度学习训练场景，尤其适合资源受限环境。
原文摘要 · Abstract (English)
Training deep neural networks with noise and data heterogeneity is a major challenge. We introduce Lightweight Learnable Adaptive Weighting (LiLAW), a method that dynamically adjusts the loss weight of each training sample based on its evolving difficulty, categorized as easy, moderate, and hard, using only three global learnable scalar parameters. LiLAW learns to adaptively prioritize samples by updating these parameters with a single gradient descent step on a validation mini-batch after each training mini-batch, without requiring a clean, unbiased validation set. Experiments across general and medical imaging datasets, several noise types and levels, loss functions, and architectures with and without pretraining, including linear probing and full fine-tuning, show that LiLAW consistently improves accuracy and AUROC, especially in higher-noise settings, without requiring excessive tuning. We also obtain state-of-the-art results incorporating synthetic and augmented data from SynPAIN, GAITGen, ECG5000, and improved fairness on the Adult dataset. LiLAW is lightweight, practical, and computationally efficient, making it an effective, scalable approach to boost generalization and robustness across diverse deep learning training setups, especially in resource-constrained settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。