用低维流形降低神经网络系统辨识的复杂度,小数据也能精准建模。
Manifold meta-learning for reduced-complexity neural system identification
- 从相关系统数据中学习参数空间的低维流形,压缩模型复杂度。
- 在少量数据下仍能准确建模布克-温振子系统,精度接近全参数模型。
- 无需二阶梯度计算,适合资源受限的大规模神经系统建模。
系统辨识已从深度学习中获益良多,尤其适用于物理机制部分未知的复杂非线性动力系统,传统方法在此类场景下可能失效。然而,深度学习模型通常需要大量数据和高算力,因参数量大而难以部署。为此,本文提出一种元学习框架,从一类相关动力系统的输入输出序列构成的元数据集中,发现过参数化神经网络参数空间中的低维流形。该流形使模型在保持表达能力的同时实现高效训练。与双层元学习不同,本方法使用辅助神经网络直接将数据映射至学习到的流形,避免了元训练中的高成本二阶梯度计算,并大幅减少推理阶段所需的一阶更新次数,对大型模型尤为有利。我们在布克-温振子(Bouc-Wen oscillators)这一经典非线性系统辨识基准上验证了该方法,在小样本条件下仍能学习到高精度模型。
原文摘要 · Abstract (English)
System identification has greatly benefited from deep learning techniques, particularly for modeling complex, nonlinear dynamical systems with partially unknown physics where traditional approaches may not be feasible. However, deep learning models often require large datasets and significant computational resources at training and inference due to their high-dimensional parameterizations. To address this challenge, we propose a meta-learning framework that discovers a low-dimensional manifold within the parameter space of an over-parameterized neural network architecture. This manifold is learned from a meta-dataset of input-output sequences generated by a class of related dynamical systems, enabling efficient model training while preserving the network's expressive power for the considered system class. Unlike bilevel meta-learning approaches, our method employs an auxiliary neural network to map datasets directly onto the learned manifold, eliminating the need for costly second-order gradient computations during meta-training and reducing the number of first-order updates required in inference, which could be expensive for large models. We validate our approach on a family of Bouc-Wen oscillators, which is a well-studied nonlinear system identification benchmark. We demonstrate that we are able to learn accurate models even in small-data scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。