提出可扩展的稳定递归网络，训练推理快10倍且保持性能。
R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networks
- 直接参数化权重实现稳定与鲁棒性，无需每步迭代求解。
- 在非线性系统辨识等任务中，测试性能相当但速度提升近10倍。
- 适合需要高效大规模序列建模的控制与机器学习场景。
本文提出稳健递归深度网络（R2DN），一种适用于机器学习与数据驱动控制的可扩展稳健递归神经网络参数化方法。R2DN由线性时不变系统与1-Lipschitz深度前馈网络构成反馈连接，通过直接参数化权重，使模型在设计上具备稳定性（收缩性）和对小输入扰动的鲁棒性（Lipschitz性）。该结构借鉴先前的递归平衡网络（REN），但无需在每个时间步迭代求解平衡层，显著加速了模型推理与反向传播，使更大规模网络、批量和序列长度成为可能。我们在非线性系统辨识、观测器设计和基于学习的反馈控制三个典型问题上对比R2DN与REN，发现训练与推理速度最高提升一个数量级，且在模型表达能力扩展方面表现更优。
原文摘要 · Abstract (English)
This paper presents the Robust Recurrent Deep Network (R2DN), a scalable parameterization of robust recurrent neural networks for machine learning and data-driven control. We construct R2DNs as the feedback interconnection of a linear time-invariant system and a 1-Lipschitz deep feedforward network, and directly parameterize the weights so that our models are stable (contracting) and robust to small input perturbations (Lipschitz) by design. Our parameterization uses a structure similar to the previously-proposed recurrent equilibrium network (REN), but without the requirement to iteratively solve an equilibrium layer at each time-step. This speeds up both model inference and backpropagation on GPUs, and makes it computationally feasible to scale up the network size, batch size, and input sequence length in comparison to RENs. We compare R2DNs to RENs on three representative problems in nonlinear system identification, observer design, and learning-based feedback control. We find that training and inference are both up to an order of magnitude faster with similar test set performance, and that they scale more favorably with respect to model expressivity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。