用文本信息纠正图像标签错配，提升噪声数据下的长尾识别效果
Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Label
- 引入弱教师监督信号，利用文本与图像的跨模态对齐修正标签错误
- 在高噪声条件下显著提升长尾分布数据的识别准确率
- 适合处理真实世界中标签混乱、类别不均衡的数据场景
现实数据常呈现长尾分布且存在大量噪声标签，严重降低深度模型性能。现有方法虽有所进展，但忽略了高噪声环境下标签与图像严重错配的问题。尽管观测标签与图像不匹配，仍保留类别信息，我们提出利用标签中的辅助文本信息来修复这种不一致。具体地，借助预训练视觉-语言模型的内在跨模态对齐能力，生成一种不受标签噪声和数据分布偏差影响的监督信号，称为弱教师监督（WTS），其准确率有限但稳定。通过比较文本预测标签与原始标签的差异来激活WTS。大量实验表明，WTS在合成与真实数据集上均表现优异，尤其在高噪声条件下优势明显。代码已公开。
原文摘要 · Abstract (English)
Real-world data often exhibit long-tailed distributions with numerous noisy labels, substantially degrading the performance of deep models. While prior research has made progress in addressing this combined challenge, it overlooks the severe label-image mismatch inherent to high-noise settings, thereby limiting their effectiveness. Given that observed labels, though mismatched with images, still retain category information, we propose employing auxiliary text information from labels to address label-image inconsistencies in long-tailed noisy data. Specifically, we leverage the intrinsic cross-modal alignment in pre-trained visual-language models to correct the label-image inconsistencies. This supervisory signal, referred to as Weak Teacher Supervision (WTS), is unaffected by label noise and data distribution biases, albeit exhibits limited accuracy. Therefore, the activation of WTS is determined by evaluating the discrepancy between text-predicted labels and observed labels. Extensive experiments demonstrate the superior performance of WTS across synthetic and real-world datasets, particularly under high-noise conditions. The source code is available at https://anonymous.4open.science/r/WTS-0F3C.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。