arXiv:2503.02798stat.MLcs.DS2025-03被引 3

首个无需强假设的高维稀疏回归后验采样算法,突破测量数瓶颈。

Spike-and-Slab Posterior Sampling in High Dimensions

  • 基于压缩感知思想设计可证明的后验采样方法
  • 仅需 n ≥ k³·polylog(d) 次测量即可准确采样
  • 适用于任意信噪比,适合高维变量选择研究者

在高维稀疏线性回归中,带尖峰-平板先验的后验采样被视为贝叶斯变量选择的理论最优方法。然而,设计可证明有效的采样算法极为困难。现有方法或依赖强信噪比假设,或要求测量数随维度线性增长,或使用启发式近似。本文提出首个对任意信噪比均成立的可证明算法,仅需 n ≥ k³·polylog(d) 次测量,且测量矩阵满足受限等距性质。进一步给出近线性时间(≈nd)采样器,当 n ≥ k⁵·polylog(d) 时成立。还扩展至拉普拉斯平板密度情形,在 σ = O(1/k) 时获得类似保证。

原文摘要 · Abstract (English)

Posterior sampling with the spike-and-slab prior [MB88], a popular multimodal distribution used to model uncertainty in variable selection, is considered the theoretical gold standard method for Bayesian sparse linear regression [CPS09, Roc18]. However, designing provable algorithms for performing this sampling task is notoriously challenging. Existing posterior samplers for Bayesian sparse variable selection tasks either require strong assumptions about the signal-to-noise ratio (SNR) [YWJ16], only work when the measurement count grows at least linearly in the dimension [MW24], or rely on heuristic approximations to the posterior. We give the first provable algorithms for spike-and-slab posterior sampling that apply for any SNR, and use a measurement count sublinear in the problem dimension. Concretely, assume we are given a measurement matrix $\mathbf{X} \in \mathbb{R}^{n\times d}$ and noisy observations $\mathbf{y} = \mathbf{X}\mathbfθ^\star + \mathbfξ$ of a signal $\mathbfθ^\star$ drawn from a spike-and-slab prior $π$ with a Gaussian diffuse density and expected sparsity k, where $\mathbfξ \sim \mathcal{N}(\mathbb{0}_n, σ^2\mathbf{I}_n)$. We give a polynomial-time high-accuracy sampler for the posterior $π(\cdot \mid \mathbf{X}, \mathbf{y})$, for any SNR $σ^{-1}$ > 0, as long as $n \geq k^3 \cdot \text{polylog}(d)$ and $X$ is drawn from a matrix ensemble satisfying the restricted isometry property. We further give a sampler that runs in near-linear time $\approx nd$ in the same setting, as long as $n \geq k^5 \cdot \text{polylog}(d)$. To demonstrate the flexibility of our framework, we extend our result to spike-and-slab posterior sampling with Laplace diffuse densities, achieving similar guarantees when $σ= O(\frac{1}{k})$ is bounded.

贝叶斯推断稀疏回归后验采样高维统计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。