arXiv:2509.17153cs.LGcs.AI2025-09AAAI被引 1

用流模型压缩神经网络权重,提升不确定性估计与分布外检测能力

Flow-Induced Diagonal Gaussian Processes

  • 通过归一化流和谱正则化构建低维诱导子空间,稳定投影机制
  • 参数压缩约51%,模型大小减少75%,推理成本略增但训练成本降数个量级
  • 适用于需要高效贝叶斯推理与分布外检测的场景

我们提出流诱导对角高斯过程(FiD-GP),一种压缩框架,通过紧凑的诱导权重矩阵将神经网络的权重不确定性投影到低维子空间。该方法依赖归一化流先验和谱正则化,增强表达能力,并通过数值稳定的投影目标使诱导子空间与特征梯度几何对齐。此外,我们证明了FiD-GP的预测框架可实现单次传递的分布外(OoD)检测。在回归、图像分类、语义分割及分布外检测基准上的综合实验证明,相比基于SVGP的基线,FiD-GP显著提升不确定性估计能力,满足紧致的谱残差界,理论上保证OoD检测性能,且将神经网络存储需求大幅压缩:参数减少约51%,模型规模缩小约75%,同时保持前沿精度与不确定性估计水平,仅以增加依赖诱导权重数量的推理计算为代价。

原文摘要 · Abstract (English)

We present Flow-Induced Diagonal Gaussian Processes (FiD-GP), a compression framework that incorporates a compact inducing weight matrix to project a neural network's weight uncertainty into a lower-dimensional subspace. Critically, FiD-GP relies on normalising-flow priors and spectral regularisations to augment its expressiveness and align the inducing subspace with feature-gradient geometry through a numerically stable projection mechanism objective. Furthermore, we demonstrate how the prediction framework in FiD-GP can help to design a single-pass projection for Out-of-Distribution (OoD) detection. Our analysis shows that FiD-GP improves uncertainty estimation ability on various tasks compared with SVGP-based baselines, satisfies tight spectral residual bounds with theoretically guaranteed OoD detection, and significantly compresses the neural network's storage requirements at the cost of increased inference computation dependent on the number of inducing weights employed. Specifically, in a comprehensive empirical study spanning regression, image classification, semantic segmentation, and out-of-distribution detection benchmarks, it cuts Bayesian training cost by several orders of magnitude, compresses parameters by roughly 51%, reduces model size by about 75%, and matches state-of-the-art accuracy and uncertainty estimation.

贝叶斯深度学习不确定性估计模型压缩分布外检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。