用信息几何方法提升分类特征的区分度,效果媲美顶尖降维技术。
Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reduction
- 基于费雪信息度量的几何距离优化特征,实现监督降维
- 在真实数据集上分类准确率媲美主流方法,最优达SQFA-H变体
- 适合追求理论严谨性与高区分度特征的机器学习研究者
监督降维通过将带标签数据映射到低维特征空间来保持类别分离。常用策略是学习最大化类别条件概率分布间统计差异性的特征。信息几何基于黎曼几何,将概率分布视为统计流形上的点,使用费雪信息度量定义测地距离(费雪-罗距离)以衡量分布间差异。该距离因具局部可区分性且有几何解释而成为理想选择。本文提出监督二次特征分析(SQFA),一种在高斯假设下学习线性特征以最大化类别条件分布间费雪-罗距离的方法。在多个真实世界数据集上,SQFA特征支持的分类准确率与最大化其他流行差异度量或由先进降维方法学习的特征相当。值得注意的是,其变体SQFA-H(最大化赫林格距离)取得最佳分类性能,后者在降维中罕见使用。结果表明信息几何在监督降维中具有潜力。代码开源于https://github.com/dherrera1911/sqfa。
原文摘要 · Abstract (English)
Supervised dimensionality reduction maps labeled data into a low-dimensional feature space while preserving class separation. A common strategy is to learn features that maximize a measure of statistical dissimilarity between the class-conditional probability distributions. Information geometry, which is rooted in Riemannian geometry, provides an alternative framework for measuring class dissimilarity. It treats probability distributions as points in a statistical manifold and uses the Fisher information metric to define a geodesic distance--the Fisher-Rao distance--between distributions The Fisher-Rao distance is an appealing candidate for measuring class separation because the Fisher information metric is a local measure of discriminability, and because it allows a geometric interpretation. Here, we present Supervised Quadratic Feature Analysis (SQFA), a supervised dimensionality reduction method which learns linear features that maximize Fisher-Rao distances between class-conditional distributions, under Gaussian assumptions. In multiple real world datasets, we find that SQFA features support classification accuracy that is competitive with features that maximize more popular measures of dissimilarity, or that are learned by other state-of-the-art dimensionality reduction methods. Notably, the best classification accuracy is achieved by SQFA-H features, a variant of SQFA that maximizes the Hellinger distance, a rarely used objective for dimensionality reduction. These results demonstrate the potential of information geometry as a tool for supervised dimensionality reduction. We provide a Python implementation of SQFA at https://github.com/dherrera1911/sqfa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。