通过定期裁剪权重奇异值，防止神经网络在持续学习中失去适应新任务的能力。
SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning
- 定期裁剪权重矩阵的奇异值，抑制其各向异性增长
- 在持续监督学习和强化学习任务中显著提升模型适应能力
- 适合需要长期学习新任务的智能系统研究者
在非平稳任务上训练的神经网络常会丧失拟合新目标的能力，称为可塑性丧失。我们发现权重矩阵奇异值的各向异性增长是导致该现象的新原因，并从实验与理论上进行了分析。为缓解此问题，提出SingularClip方法，周期性地对所有权重矩阵的奇异值进行裁剪。实验表明，SingularClip在持续监督学习和深度强化学习的多种任务中均显著优于基线方法。
原文摘要 · Abstract (English)
Neural networks trained on nonstationary tasks frequently lose the ability to fit new targets, a phenomenon referred to as loss of plasticity. We identify a novel source of plasticity loss due to the growing anisotropy of weight matrices' singular values during training, and analyze this phenomenon both empirically and theoretically. To mitigate this issue, we introduce SingularClip, a procedure that periodically clips the singular values of all weight matrices. We show that SingularClip performs strongly against baselines across a range of tasks in both continual supervised learning and deep reinforcement learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。