互蒸馏隐含正则化，让强化学习更抗干扰
Representation Convergence: Mutual Distillation is Secretly a Form of Regularization
- 用理论证明鲁棒性提升可增强泛化能力
- 实验证明互蒸馏能自发产生像素不变特征
- 适合关注泛化机制的算法研究者
本文提出，强化学习策略间的互蒸馏本质上是一种隐式正则化，可防止模型过拟合无关特征。理论层面首次证明：提升策略对无关特征的鲁棒性，能改善泛化性能。实验表明，策略间互蒸馏有助于增强这种鲁棒性，使像素输入下涌现出不变表示。本文不追求达到最先进性能，而是致力于揭示泛化的底层原理，深化对泛化机制的理解。
原文摘要 · Abstract (English)
In this paper, we argue that mutual distillation between reinforcement learning policies serves as an implicit regularization, preventing them from overfitting to irrelevant features. We highlight two separate contributions: (i) Theoretically, for the first time, we prove that enhancing the policy robustness to irrelevant features leads to improved generalization performance. (ii) Empirically, we demonstrate that mutual distillation between policies contributes to such robustness, enabling the spontaneous emergence of invariant representations over pixel inputs. Ultimately, we do not claim to achieve state-of-the-art performance but rather focus on uncovering the underlying principles of generalization and deepening our understanding of its mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。