用主成分初始化网络首层，让模型解释更简单且训练效果更好
Principal Components for Neural Network Initialization
- 将主成分直接用于神经网络首层权重初始化
- 相比先降维再训练，解释性更强且训练性能更优
- 适合关注模型可解释性与训练效率的研究者
主成分分析（PCA）常用于神经网络训练前的数据降维和去噪，但会增加可解释人工智能（XAI）方法的复杂性。本文分析了该做法的潜在问题，提出基于主成分的初始化方法（PCsInit），通过将主成分作为网络首层权重初始值，及其两种变体PCsInit-Act与PCsInit-Sub。实验表明，该策略使模型解释更简洁直接。同时，所提方法具备良好的理论性质，且在反向传播训练中表现优于直接在主成分上训练的模型。
原文摘要 · Abstract (English)
Principal Component Analysis (PCA) is a commonly used tool for dimension reduction and denoising. Therefore, it is also widely used on the data prior to training a neural network. However, this approach can complicate the explanation of eXplainable Artificial Intelligence (XAI) methods for the decision of the model. In this work, we analyze the potential issues with this approach and propose Principal Components-based Initialization (PCsInit), a strategy to incorporate PCA into the first layer of a neural network via initialization of the first layer in the network with the principal components, and its two variants PCsInit-Act and PCsInit-Sub. We will show that explanations using these strategies are more simple, direct and straightforward than using PCA prior to training a neural network on the principal components. We also show that the proposed techniques possess desirable theoretical properties. Moreover, as will be illustrated in the experiments, such training strategies can also allow further improvement of training via backpropagation compared to training neural networks on principal components.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。