arXiv:2602.21965cs.LG2026-02

用频域结构设计紧凑神经网络层,提升边缘部署的效率与可靠性。

Compact Circulant Layers with Spectral Priors

  • 在频域参数化卷积核,实现低维权重空间的结构化变分推断
  • 仅用少量参数即达强基线性能,且具有更紧的Lipschitz上界
  • 适合资源受限场景下的贝叶斯与确定性模型,尤其适合医疗、机器人

医疗、机器人及自动驾驶等关键领域需要内存高效、具备不确定性感知能力的神经网络,适用于边缘设备等资源受限环境。本文研究紧凑的谱循环层与块循环-块循环(BCCB)层:可通过FFT对角化的循环卷积,其权重直接定义在实数FFT半轴(1D)或半平面(2D)。在频域参数化滤波器可施加简单谱结构,实现低维权重空间中的结构化变分推断,并精确计算层的谱范数,从而获得廉价的全局Lipschitz边界和基于裕度的鲁棒性诊断。通过在Hermitian支撑上放置独立复高斯分布,我们得到离散圆/环面上平稳核的谱表示,构建出精确的平稳高斯过程先验。据此定义实用的谱先验与面向实坐标、考虑Hermitian结构的低秩加对角变分后验。实验表明,谱循环/BCCB层在多种任务中表现优异:在MNIST→Fashion-MNIST上的紧凑贝叶斯网络、冻结CIFAR-10特征上的变分头、以及CIFAR-10/Tiny ImageNet上的确定性ViT投影;其性能媲美强基线,但参数量显著减少,且具有更紧的Lipschitz证书。

原文摘要 · Abstract (English)

Critical applications in areas such as medicine, robotics and autonomous systems require compact (i.e., memory efficient), uncertainty-aware neural networks suitable for edge and other resource-constrained deployments. We study compact spectral circulant and block-circulant-with-circulant-blocks (BCCB) layers: FFT-diagonalizable circular convolutions whose weights live directly in the real FFT (RFFT) half (1D) or half-plane (2D). Parameterizing filters in the frequency domain lets us impose simple spectral structure, perform structured variational inference in a low-dimensional weight space, and calculate exact layer spectral norms, enabling inexpensive global Lipschitz bounds and margin-based robustness diagnostics. By placing independent complex Gaussians on the Hermitian support we obtain a discrete instance of the spectral representation of stationary kernels, inducing an exact stationary Gaussian-process prior over filters on the discrete circle/torus. We exploit this to define a practical spectral prior and a Hermitian-aware low-rank-plus-diagonal variational posterior in real coordinates. Empirically, spectral circulant/BCCB layers are effective compact building blocks in both (variational) Bayesian and point estimate regimes: compact Bayesian neural networks on MNIST->Fashion-MNIST, variational heads on frozen CIFAR-10 features, and deterministic ViT projections on CIFAR-10/Tiny ImageNet; spectral layers match strong baselines while using substantially fewer parameters and with tighter Lipschitz certificates.

紧凑模型谱方法贝叶斯神经网络边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。