深度集成结合迁移学习可显著提升精神分裂症与双相情感障碍分类稳定性。
How and why does deep ensemble coupled with transfer learning increase performance in bipolar disorder and schizophrenia classification?
- 采用迁移学习与深度集成,降低单个模型训练的不确定性。
- 10个模型组成的集成可达到性能上限,进一步增加无明显提升。
- 预训练模型使模型收敛到相似损失区域,增强泛化能力。
迁移学习(TL)和深度集成学习(DE)在精神疾病分类中表现优于传统机器学习,但其原理尚不明确。本文探究了在双相情感障碍(BD)和精神分裂症(SCZ)分类任务中,为何联合使用TL与DE能降低个体模型的分类变异性。通过多次训练同一骨干网络但不同初始化,评估模型参数估计的表征不确定性。结果表明,使用TL与DE可显著提升分类性能。研究进一步发现:① 当集成模型数量达到10个时,性能趋于饱和;② 使用预训练模型的TL模型会收敛至损失函数的相同区域,而随机初始化的DL模型则不会。这说明预训练有助于约束模型优化路径,提升泛化性。
原文摘要 · Abstract (English)
Transfer learning (TL) and deep ensemble learning (DE) have recently been shown to outperform simple machine learning in classifying psychiatric disorders. However, there is still a lack of understanding as to why that is. This paper aims to understand how and why DE and TL reduce the variability of single-subject classification models in bipolar disorder (BD) and schizophrenia (SCZ). To this end, we investigated the training stability of TL and DE models. For the two classification tasks under consideration, we compared the results of multiple trainings with the same backbone but with different initializations. In this way, we take into account the epistemic uncertainty associated with the uncertainty in the estimation of the model parameters. It has been shown that the performance of classifiers can be significantly improved by using TL with DE. Based on these results, we investigate i) how many models are needed to benefit from the performance improvement of DE when classifying BD and SCZ from healthy controls, and ii) how TL induces better generalization, with and without DE. In the first case, we show that DE reaches a plateau when 10 models are included in the ensemble. In the second case, we find that using a pre-trained model constrains TL models with the same pre-training to stay in the same basin of the loss function. This is not the case for DL models with randomly initialized weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。