arXiv:2501.02411stat.MLcs.LG2025-01

用正则化判别分析迁移学习,提升小样本高维数据分类效果

Transfer learning via Regularized Linear Discriminant Analysis

  • 通过加权融合目标与源模型的岭估计构建判别方向
  • 在高维下理论推导出最优权重与分类误差率
  • 适用于小样本高维场景,如生物医学风险预测

线性判别分析广泛用于分类,但高维预测变量与小样本常导致分类误差大。为解决此问题,需借助相关源模型数据提升目标模型性能。本文提出基于正则化随机效应线性判别分析的迁移学习新方法,将判别方向估计为来自目标与源模型的岭估计的加权组合。引入多种权重确定策略,包括最小化判别向量估计风险和最小化分类误差的方法。利用随机矩阵理论,在 $p/n \rightarrow γ$ 的高维设定下,显式推导出权重及对应分类误差率的渐近值。提供不同权重的几何解释,并给出选择建议。大量数值实验(含模拟与基于蛋白质组学的10年心血管疾病风险分类)验证了该方法的有效性。

原文摘要 · Abstract (English)

Linear discriminant analysis is a widely used method for classification. However, the high dimensionality of predictors combined with small sample sizes often results in large classification errors. To address this challenge, it is crucial to leverage data from related source models to enhance the classification performance of a target model. We propose to address this problem in the framework of transfer learning. In this paper, we present novel transfer learning methods via regularized random-effects linear discriminant analysis, where the discriminant direction is estimated as a weighted combination of ridge estimates obtained from both the target and source models. Multiple strategies for determining these weights are introduced and evaluated, including one that minimizes the estimation risk of the discriminant vector and another that minimizes the classification error. Utilizing results from random matrix theory, we explicitly derive the asymptotic values of these weights and the associated classification error rates in the high-dimensional setting, where $p/n \rightarrow γ$, with $p$ representing the predictor dimension and $n$ the sample size. We also provide geometric interpretations of various weights and a guidance on which weights to choose. Extensive numerical studies, including simulations and analysis of proteomics-based 10-year cardiovascular disease risk classification, demonstrate the effectiveness of the proposed approach.

迁移学习判别分析高维数据小样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。