提出基于Bures-Wasserstein几何的可学习归一化方法,提升病态协方差矩阵建模能力。
Learning to Normalize on the SPD Manifold under Bures-Wasserstein Geometry
- 基于广义Bures-Wasserstein度量设计可学习的Riemannian归一化
- 在多个数据集上显著提升SPD神经网络性能,尤其对病态矩阵效果更优
- 适合研究协方差表示学习、几何深度学习的科研人员
协方差矩阵在众多科学领域中表现出色。由于其位于对称正定(SPD)流形这一具有内在非欧几里得几何特性的黎曼空间中,表征学习的核心挑战在于尊重其几何结构。受欧式深度学习成功启发,研究者已开发出适用于SPD流形的神经网络,以实现更忠实的协方差嵌入学习。其中,黎曼批量归一化(RBN)显著提升了SPD网络模型性能。然而,现有RBN所依赖的黎曼度量在处理病态协方差矩阵(ICSM)时表现不佳,削弱了其有效性。相比之下,Bures-Wasserstein度量(BWM)在处理病态性方面表现更优。此外,近期提出的广义BWM(GBWM)通过一个SPD矩阵参数化原始BWM,能够更精细地刻画SPD流形的复杂几何特征。因此,本文提出一种基于GBW几何的新型RBN算法,引入可学习的度量参数,并通过矩阵幂对GBWM进行变形,进一步增强基于GBWM的RBN的表征能力。在多个数据集上的实验结果验证了该方法的有效性。
原文摘要 · Abstract (English)
Covariance matrices have proven highly effective across many scientific fields. Since these matrices lie within the Symmetric Positive Definite (SPD) manifold - a Riemannian space with intrinsic non-Euclidean geometry, the primary challenge in representation learning is to respect this underlying geometric structure. Drawing inspiration from the success of Euclidean deep learning, researchers have developed neural networks on the SPD manifolds for more faithful covariance embedding learning. A notable advancement in this area is the implementation of Riemannian batch normalization (RBN), which has been shown to improve the performance of SPD network models. Nonetheless, the Riemannian metric beneath the existing RBN might fail to effectively deal with the ill-conditioned SPD matrices (ICSM), undermining the effectiveness of RBN. In contrast, the Bures-Wasserstein metric (BWM) demonstrates superior performance for ill-conditioning. In addition, the recently introduced Generalized BWM (GBWM) parameterizes the vanilla BWM via an SPD matrix, allowing for a more nuanced representation of vibrant geometries of the SPD manifold. Therefore, we propose a novel RBN algorithm based on the GBW geometry, incorporating a learnable metric parameter. Moreover, the deformation of GBWM by matrix power is also introduced to further enhance the representational capacity of GBWM-based RBN. Experimental results on different datasets validate the effectiveness of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。