提出无需训练的协方差散射变换,稳定捕捉低样本下的数据模式。
Covariance Scattering Transforms
- 通过协方差小波滤波器逐层提取谱特征,构建层次化表示
- 在4个神经退行性疾病数据集上实现与复杂模型相当的年龄预测性能
- 无需训练、对近似特征值不敏感,适合小样本场景
依赖协方差信息的机器学习方法广泛用于无监督和未标记数据中识别有意义模式。主成分分析(PCA)将数据投影到协方差矩阵的特征向量上,捕捉最大方差方向,但存在两个缺陷:忽略低方差方向信息(如高方差噪声中);在低样本条件下不稳定,尤其当协方差特征值接近时。协方差神经网络(VNNs)利用协方差矩阵作为图结构,比PCA更稳定且表达能力更强,但需训练且依赖标签。为此,我们提出协方差散射变换(CSTs),一种深度非训练网络,通过在协方差谱上应用局部化滤波器,结合非线性生成层次化表征。滤波器定义为协方差小波,可捕获特定谱模式。通过剪枝机制提升计算与内存效率,并证明其有限样本估计误差对相近特征值的敏感度低于PCA,从而增强稳定性。在4个包含神经退行性疾病患者的皮层厚度数据集上,CSTs在低样本下生成稳定表征,媲美有训练的VNN,且预测性能与更复杂学习模型相当或更优。
原文摘要 · Abstract (English)
Machine learning and data processing techniques relying on covariance information are widespread as they identify meaningful patterns in unsupervised and unlabeled settings. As a prominent example, Principal Component Analysis (PCA) projects data points onto the eigenvectors of their covariance matrix, capturing the directions of maximum variance. This mapping, however, falls short in two directions: it fails to capture information in low-variance directions, relevant when, e.g., the data contains high-variance noise; and it provides unstable results in low-sample regimes, especially when covariance eigenvalues are close. CoVariance Neural Networks (VNNs), i.e., graph neural networks using the covariance matrix as a graph, show improved stability to estimation errors and learn more expressive functions in the covariance spectrum than PCA, but require training and operate in a labeled setup. To get the benefits of both worlds, we propose Covariance Scattering Transforms (CSTs), deep untrained networks that sequentially apply filters localized in the covariance spectrum to the input data and produce expressive hierarchical representations via nonlinearities. We define the filters as covariance wavelets that capture specific and detailed covariance spectral patterns. We improve CSTs' computational and memory efficiency via a pruning mechanism, and we prove that their error due to finite-sample covariance estimations is less sensitive to close covariance eigenvalues compared to PCA, improving their stability. Our experiments on age prediction from cortical thickness measurements on 4 datasets collecting patients with neurodegenerative diseases show that CSTs produce stable representations in low-data settings, as VNNs but without any training, and lead to comparable or better predictions w.r.t. more complex learning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。