用精度矩阵替代协方差，实现更精准的图神经网络学习
Precision Neural Networks: Joint Graph And Relational Learning
- 在精度矩阵上做卷积，利用其稀疏性和独立性建模优势
- 联合优化网络参数与精度矩阵，相比两步法提升性能
- 理论保证估计误差，适合需要高精度建模的任务
协方差神经网络（VNNs）在数据协方差矩阵定义的图上进行卷积，实现表达性强且稳定的协方差学习。然而协方差矩阵通常稠密，无法编码条件独立性，且常以任务无关方式预计算，可能影响性能。为此，我们研究精度神经网络（PNNs），即在精度矩阵（协方差的逆）上构建的VNNs。精度矩阵天然编码统计独立性，常具稀疏性，并保留协方差的谱结构。为使精度估计任务感知，我们提出联合优化问题，通过交替优化依次更新网络权重和精度估计。理论上界定了每轮迭代中估计精度与真实精度之间的距离，并在合成与真实数据上证明了联合估计优于两步法。
原文摘要 · Abstract (English)
CoVariance Neural Networks (VNNs) perform convolutions on the graph determined by the covariance matrix of the data, which enables expressive and stable covariance-based learning. However, covariance matrices are typically dense, fail to encode conditional independence, and are often precomputed in a task-agnostic way, which may hinder performance. To overcome these limitations, we study Precision Neural Networks (PNNs), i.e., VNNs on the precision matrix - the inverse covariance. The precision matrix naturally encodes statistical independence, often exhibits sparsity, and preserves the covariance spectral structure. To make precision estimation task-aware, we formulate an optimization problem that jointly learns the network parameters and the precision matrix, and solve it via alternating optimization, by sequentially updating the network weights and the precision estimate. We theoretically bound the distance between the estimated and true precision matrices at each iteration, and demonstrate the effectiveness of joint estimation compared to two-step approaches on synthetic and real-world data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。