提出VAND方法，让RNN更稳定地学习序列与周期性行为。
Variational Adaptive Noise and Dropout towards Stable Recurrent Neural Networks
- 将RNN优化重释为变分推断，统一生成噪声与丢弃机制。
- 在移动机械臂模仿任务中，唯独VAND能准确复现指令的时序与周期动作。
- 适合需要稳定序列建模的机器人控制、时序预测场景。
本文提出一种新型稳定的递归神经网络学习理论——变分自适应噪声与丢弃（VAND）。以往研究已分别证实内部状态添加噪声和丢弃是稳定RNN的有效手段。本文将RNN优化问题重新诠释为变分推断，证明可通过将显式正则项转化为隐式正则，同时导出噪声与丢弃机制。其强度与比例可分别调整以优化主目标。在移动机械臂的模仿学习任务中，仅VAND能成功复现指定的序列与周期性行为。相关演示视频见：https://youtu.be/UOho3Xr6A2w。
原文摘要 · Abstract (English)
This paper proposes a novel stable learning theory for recurrent neural networks (RNNs), so-called variational adaptive noise and dropout (VAND). As stabilizing factors for RNNs, noise and dropout on the internal state of RNNs have been separately confirmed in previous studies. We reinterpret the optimization problem of RNNs as variational inference, showing that noise and dropout can be derived simultaneously by transforming the explicit regularization term arising in the optimization problem into implicit regularization. Their scale and ratio can also be adjusted appropriately to optimize the main objective of RNNs, respectively. In an imitation learning scenario with a mobile manipulator, only VAND is able to imitate sequential and periodic behaviors as instructed. https://youtu.be/UOho3Xr6A2w
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。