通过平滑损失过渡提升模型在分布偏移下的适应稳定性。
Loss Smoothing for Stable Adaptation Under Distribution Shift
- 用源目标与目标目标的加权平均实现渐进式优化
- 在多种任务中均显著提升适应性能
- 适合需要稳定迁移的微调与强化学习场景
在微调和强化学习等场景中，神经网络常面临分布偏移下的适应问题。传统方法直接优化目标函数，导致源训练目标到目标目标的突变，可能破坏仍对新任务有用的特征表示。本文提出损失平滑方法，在适应初期对源目标与目标目标进行插值，实现更平缓的过渡。该方法有助于保留源分布中有用特征，同时使模型逐步适配目标分布。在受控监督迁移、预训练视觉适应、离线到在线强化学习及语言模型微调等多种场景中，损失平滑均持续提升性能，表明更平滑的目标过渡是一种广泛适用的模型适应工具。
原文摘要 · Abstract (English)
In settings such as fine-tuning and reinforcement learning, neural networks are often adapted under distribution shift. Standard adaptation methods typically optimize the target objective directly, inducing an abrupt change from the source training objective. This abrupt transition can distort learned representations, including features that may still be useful for the new task. We investigate whether a more gradual transition can improve adaptation. We propose loss smoothing, a simple approach that interpolates between the source and target training objectives at the start of adaptation. This smooth transition helps to preserve useful features from the source distribution while still enabling the model to specialize to the target distribution. Across controlled supervised shifts, pretrained vision adaptation, offline-to-online and online reinforcement learning, and language model fine-tuning, we find that loss smoothing consistently improves performance, suggesting that smoother objective transitions are a broadly useful tool for model adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。