提出一种新型凸优化LDA方法,解决高维数据分类难题
A Convex formulation for linear discriminant analysis
- 用凸优化框架统一类内聚集与类间分离目标
- 在高维生物数据上优于主流LDA方法,准确率提升显著
- 无需矩阵求逆和成对距离计算,适合大规模数据
我们提出一种名为凸线性判别分析(ConvexLDA)的监督降维技术。该模型通过平衡两个互补项的多目标损失函数进行优化:第一项最小化样本到类别中心的距离,使同类别样本在低维空间中聚集;第二项最大化类间超椭球体散度体积,通过低维类别中心外积矩阵的行列式对数(log det)实现。采用负log det形式将目标转化为最小化问题,并引入超参数λ调节两项权衡。我们证明该损失函数为凸函数。与Fisher LDA不同,本方法无需计算矩阵逆,可避免高维数据(如RNA-seq)中的病态问题。ConvexLDA不依赖成对距离计算,因此更高效且易于扩展。由于损失函数的凸性,保证了全局最优解,提升了嵌入表示的可靠性。实验表明,ConvexLDA在多种高维生物数据和图像数据集上均优于多个主流LDA方法。
原文摘要 · Abstract (English)
We present a supervised dimensionality reduction technique called Convex Linear Discriminant Analysis (ConvexLDA). The proposed model optimizes a multi-objective cost function by balancing two complementary terms. The first term pulls the samples of a class towards its centroid by minimizing a sample's distance from its class-centroid in low dimensional space. The second term pushes the classes far apart by maximizing their hyperellipsoid scattering volume via the logarithm of the determinant (\textit{log det}) of the outer product matrix formed by the low-dimensional class-centroids. Using the negative of the \textit{log det}, we pose the final cost as a minimization problem, which balances the two terms using a hyper-parameter $λ$. We demonstrate that the cost function is convex. Unlike Fisher LDA, the proposed method doesn't require to compute the inverse of a matrix, hence avoiding any ill-conditioned problem where data dimension is very high, e.g. RNA-seq data. ConvexLDA doesn't require pair-wise distance calculation, making it faster and more easily scalable. Moreover, the convex nature of the cost function ensures global optimality, enhancing the reliability of the learned embedding. Our experimental evaluation demonstrates that ConvexLDA outperforms several popular linear discriminant analysis (LDA)-based methods on a range of high-dimensional biological data, image data sets, etc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。