对比了变分推断中串行与并行坐标上升的稳定性差异
Stability of Sequential and Parallel Coordinate Ascent Variational Inference
- 分析串行与并行更新在高维线性回归中的收敛行为
- 串行算法在更弱条件下仍能保证收敛，而并行算法则不然
- 适合关注算法可靠性而非速度的研究者参考
我们揭示了两种广泛使用的坐标上升变分推断变体——串行与并行算法——之间显著的行为差异。尽管这类差异在数值分析文献中已有研究，但在复杂模型的变分推断优化领域仍鲜有探讨。本文聚焦于中等高维线性回归问题，表明串行算法虽然通常较慢，但在比并行版本更宽松的条件下仍具有收敛性保证；而并行算法虽常用于实现块级更新以提升计算效率，却缺乏类似的收敛保障。
原文摘要 · Abstract (English)
We highlight a striking difference in behavior between two widely used variants of coordinate ascent variational inference: the sequential and parallel algorithms. While such differences were known in the numerical analysis literature in simpler settings, they remain largely unexplored in the optimization-focused literature on variational inference in more complex models. Focusing on the moderately high-dimensional linear regression problem, we show that the sequential algorithm, although typically slower, enjoys convergence guarantees under more relaxed conditions than the parallel variant, which is often employed to facilitate block-wise updates and improve computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。