用语言模型引导特征提取,提升噪声环境下的模型泛化能力
A Language Anchor-Guided Method for Robust Noisy Domain Generalization
- 用自然语言锚点指导样本重加权,挖掘更鲁棒的不变特征
- 在多个数据集上显著优于现有方法,噪声下准确率提升10%以上
- 适合处理带标签噪声的真实场景,尤其适用于跨域学习
现实世界机器学习常面临分布偏移和标签噪声两大挑战。模型易过度依赖训练数据中的冗余和无信息特征,导致难以泛化至目标域。噪声数据进一步加剧过拟合,使现有方法难以区分真实不变特征与虚假相关特征。为此,我们提出锚点对齐与自适应加权(A3W)算法。该方法利用自然语言处理(NLP)锚点引导样本重加权,以提取更具代表性的特征。具体而言,A3W借助自然语言模型的语义表示作为领域不变先验知识,并采用加权损失函数,根据样本与对应NLP锚点的相似度动态调整其贡献。该机制增强了模型对噪声标签的鲁棒性。大量实验表明,A3W在标准基准数据集上持续优于现有最先进域泛化方法,在不同数据集和噪声水平下均实现显著的准确率与鲁棒性提升。
原文摘要 · Abstract (English)
Real-world machine learning applications often struggle with two major challenges: distribution shift and label noise. Models tend to overfit by focusing on redundant and uninformative features in the training data, which makes it hard for them to generalize to the target domain. Noisy data worsens this problem by causing further overfitting to the noise, meaning that existing methods often fail to tell the difference between true, invariant features and misleading, spurious ones. To tackle these issues, we introduce Anchor Alignment and Adaptive Weighting (A3W). This new algorithm uses sample reweighting guided by natural language processing (NLP) anchors to extract more representative features. In simple terms, A3W leverages semantic representations from natural language models as a source of domain-invariant prior knowledge. Additionally, it employs a weighted loss function that adjusts each sample's contribution based on its similarity to the corresponding NLP anchor. This adjustment makes the model more robust to noisy labels. Extensive experiments on standard benchmark datasets show that A3W consistently outperforms state-of-the-art domain generalization methods, offering significant improvements in both accuracy and robustness across different datasets and noise levels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。