对比了EWC中费雪信息矩阵的多种计算方式，发现当前结果可因方法不同而提升。
On the Computation of the Fisher Information in Continual Learning
- 比较了EWC中费雪信息矩阵的多种实现方法
- 实验表明不同计算方式影响模型持续学习性能
- 适合关注持续学习优化与实验可复现性的研究者
深度神经网络持续学习中最常用的方法之一是弹性权重巩固（EWC），其核心在于计算费雪信息矩阵。然而，费雪信息的具体计算方式在文献中很少被详细描述，网上存在多种不同的实现方式。本文通过分析和实证比较了几种常用实现，指出当前许多EWC相关研究的结果可能因计算方式不当而未能达到最优，若采用更合适的计算方法，性能有望提升。
原文摘要 · Abstract (English)
One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information. The exact way in which the Fisher Information is computed is however rarely described, and multiple different implementations for it can be found online. This blog post discusses and empirically compares several often-used implementations, which highlights that many currently reported results for EWC could likely be improved by changing the way the Fisher Information is computed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。