首次系统比较流模型在马尔可夫链蒙特卡洛中的表现，给出选型指南。
Empirical evaluation of normalizing flows in Markov Chain Monte Carlo
- 对比多种流模型架构，评估其在MCMC中的预处理效果。
- 梯度可用时，优秀流模型显著优于传统MCMC，微调即可提升性能。
- 收缩残差流泛化性强，对超参数不敏感，适合直接使用。
近期的MCMC研究利用归一化流对目标分布进行预处理，实现向远距离区域跳跃。然而，目前尚无针对不同归一化流架构在MCMC中表现的系统性比较。因此，许多工作选择结构简单的现成流模型，而未考虑其他模型。制定合适的架构选择指南可减少实践者分析时间，并激励研究人员以推荐模型为基础进行改进。本文首次通过在多种基于流的MCMC方法和目标分布上广泛评估多种归一化流架构，提供此类指南。当目标密度梯度可用时，合适流架构搭配少量超参数调优即可使基于流的MCMC显著优于经典MCMC；当梯度不可用时，采用开箱即用的架构亦能取得优势。我们发现收缩残差流是通用性最佳的模型，对超参数选择相对不敏感。此外，我们还揭示了在改变超参数、目标分布特性及整体计算预算条件下，归一化流在MCMC中的行为规律。
原文摘要 · Abstract (English)
Recent advances in MCMC use normalizing flows to precondition target distributions and enable jumps to distant regions. However, there is currently no systematic comparison of different normalizing flow architectures for MCMC. As such, many works choose simple flow architectures that are readily available and do not consider other models. Guidelines for choosing an appropriate architecture would reduce analysis time for practitioners and motivate researchers to take the recommended models as foundations to be improved. We provide the first such guideline by extensively evaluating many normalizing flow architectures on various flow-based MCMC methods and target distributions. When the target density gradient is available, we show that flow-based MCMC outperforms classic MCMC for suitable NF architecture choices with minor hyperparameter tuning. When the gradient is unavailable, flow-based MCMC wins with off-the-shelf architectures. We find contractive residual flows to be the best general-purpose models with relatively low sensitivity to hyperparameter choice. We also provide various insights into normalizing flow behavior within MCMC when varying their hyperparameters, properties of target distributions, and the overall computational budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。