用蒙特卡洛采样提升持续学习的预测精度与效率
Monte Carlo Functional Regularisation for Continual Learning
- 通过蒙特卡洛采样近似模型预测分布，降低计算误差
- 在MNIST和CIFAR上实现更高准确率与更快训练速度
- 适合需要高效持续学习的场景，如在线推理系统
持续学习（CL）对神经网络适应新环境至关重要。尽管基于功能正则化的CL方法优于权重空间正则化，但仍面临高计算成本和较大线性近似误差问题。本文提出一种新框架MCFRCL，利用蒙特卡洛（MC）采样近似模型预测分布，并通过三种连续分布结合矩估计方法捕捉采样样本的统计特性。同时，采用瓦斯赫斯坦距离和KL散度构建正则化项。在MNIST和CIFAR数据集上的实验表明，MCFRCL在预测准确性和训练效率方面均优于多个基准方法。
原文摘要 · Abstract (English)
Continual learning (CL) is crucial for the adaptation of neural network models to new environments. Although outperforming weight-space regularisation approaches, the functional regularisation-based CL methods suffer from high computational costs and large linear approximation errors. In this work, we present a new functional regularisation CL framework, called MCFRCL, which approximates model prediction distributions by Monte Carlo (MC) sampling. Moreover, three continuous distributions are leveraged to capture the statistical characteristics of the MC samples via moment-based methods. Additionally, both the Wasserstein distance and the Kullback-Leibler (KL) distance are employed to construct the regularisation function. The proposed MCFRCL is evaluated against multiple benchmark methods on the MNIST and CIFAR datasets, with simulation results highlighting its effectiveness in both prediction accuracy and training efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。