变分学习自动产生自适应标签平滑，提升模型鲁棒性。
Variational Learning Induces Adaptive Label Smoothing
- 通过变分优化自然诱导出每样本自适应的标签平滑
- IVON算法在多个数据集上优于传统标签平滑方法
- 适合处理标注错误和分布偏移场景
我们证明变分学习会自然诱导出一种自适应标签平滑，其中标签噪声针对每个样本进行专门化。这种平滑有助于应对标注错误和分布偏移问题，但设计有效的自适应策略通常较难。本文提出跳过手动设计步骤，直接利用变分目标优化过程中自然产生的自适应性。实验表明，一种名为IVON的变分算法性能优于传统标签平滑，并生成与已有方法相似的自适应策略。本工作将贝叶斯方法与标签平滑相连接，为缓解模型过自信预测提供了新思路。
原文摘要 · Abstract (English)
We show that variational learning naturally induces an adaptive label smoothing where label noise is specialized for each example. Such label-smoothing is useful to handle examples with labeling errors and distribution shifts, but designing a good adaptivity strategy is not always easy. We propose to skip this step and simply use the natural adaptivity induced during the optimization of a variational objective. We show empirical results where a variational algorithm called IVON outperforms traditional label smoothing and yields adaptivity strategies similar to those of an existing approach. By connecting Bayesian methods to label smoothing, our work provides a new way to handle overconfident predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。