标签平滑本质是信息瓶颈的实用实现，能过滤无关干扰。
Label Smoothing is a Pragmatic Information Bottleneck
- 从信息瓶颈视角重新解释标签平滑的原理
- 实验证明其能忽略与目标无关的冗余信息
- 适合关注模型鲁棒性与泛化能力的研究者
本研究通过信息瓶颈理论重新审视标签平滑。在模型足够灵活且输入无冲突标签的前提下，理论与实验均表明，标签平滑所得到的模型输出恰好对应信息瓶颈的最优解。由此可将标签平滑视为一种实用的信息瓶颈方法，实现简单。作为信息瓶颈机制，实验进一步显示，标签平滑对不包含目标信息的因子，或在给定另一变量条件下不再提供额外信息的因子，均表现出鲁棒性。
原文摘要 · Abstract (English)
This study revisits label smoothing via a form of information bottleneck. Under the assumption of sufficient model flexibility and no conflicting labels for the same input, we theoretically and experimentally demonstrate that the model output obtained through label smoothing explores the optimal solution of the information bottleneck. Based on this, label smoothing can be interpreted as a practical approach to the information bottleneck, enabling simple implementation. As an information bottleneck method, we experimentally show that label smoothing also exhibits the property of being insensitive to factors that do not contain information about the target, or to factors that provide no additional information about it when conditioned on another variable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。