arXiv:2506.12352cs.AIcs.LG2025-06ICML被引 1

提出NARD方法,高效筛选输入特征并建模输出相关性。

Efficient Network Automatic Relevance Determination

  • 用矩阵正态先验和稀疏参数自动识别无关特征。
  • 三类算法将计算复杂度从O(m³+d³)降至O(m³+p²),p为最终特征数。
  • 适合高维数据中需降维且保留输出关联性的场景。

我们提出网络自动相关性确定(NARD),这是对线性概率模型中自动相关性确定(ARD)的扩展,用于同时建模输入X∈ℝ^{d×N}与输出Y∈ℝ^{m×N}之间的稀疏关系,并捕捉Y内部的相关结构。NARD采用包含稀疏诱导参数的矩阵正态先验,以识别并剔除无关特征,从而促进模型稀疏性。算法上,它迭代更新精度矩阵及输出与精炼输入间的关系。为缓解每轮迭代中O(m³ + d³)的计算开销,我们引入顺序NARD,以及基于代理函数的方法,通过高效近似边缘似然,简化中间矩阵行列式与逆的计算。结合顺序更新与代理函数方法进一步降低计算成本。三种方法的每轮复杂度分别降至O(m³+p³)、O(m³ + d²)、O(m³+p²),其中p≪d为模型最终特征数。在合成与真实数据集上,方法显著提升效率,性能相当。

原文摘要 · Abstract (English)

We propose Network Automatic Relevance Determination (NARD), an extension of ARD for linearly probabilistic models, to simultaneously model sparse relationships between inputs $X \in \mathbb R^{d \times N}$ and outputs $Y \in \mathbb R^{m \times N}$, while capturing the correlation structure among the $Y$. NARD employs a matrix normal prior which contains a sparsity-inducing parameter to identify and discard irrelevant features, thereby promoting sparsity in the model. Algorithmically, it iteratively updates both the precision matrix and the relationship between $Y$ and the refined inputs. To mitigate the computational inefficiencies of the $\mathcal O(m^3 + d^3)$ cost per iteration, we introduce Sequential NARD, which evaluates features sequentially, and a Surrogate Function Method, leveraging an efficient approximation of the marginal likelihood and simplifying the calculation of determinant and inverse of an intermediate matrix. Combining the Sequential update with the Surrogate Function method further reduces computational costs. The computational complexity per iteration for these three methods is reduced to $\mathcal O(m^3+p^3)$, $\mathcal O(m^3 + d^2)$, $\mathcal O(m^3+p^2)$, respectively, where $p \ll d$ is the final number of features in the model. Our methods demonstrate significant improvements in computational efficiency with comparable performance on both synthetic and real-world datasets.

稀疏建模贝叶斯推断高维数据高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。