ATLAS自适应调整步长和轨迹长度,提升复杂分布采样精度与鲁棒性。
ATLAS: Adapting Trajectory Lengths and Step-Size for Hamiltonian Monte Carlo
- 基于局部海森矩阵近似动态调整步长,结合轨迹长度自适应策略
- 在复杂几何分布上采样准确率显著优于NUTS,计算效率接近NUTS
- 支持自动超参调优,对初始参数不敏感,适合高维复杂模型
哈密顿蒙特卡洛(HMC)及其自调优变体NUTS在处理具有复杂几何结构(如曲率变化)的分布时表现不佳,主要受限于固定的积分步长和质量矩阵。本文提出ATLAS方法,通过评估局部海森矩阵的低秩近似并估计其最大特征值,实现每轮迭代中步长的局部自适应。同时结合轨迹长度的自适应策略,监测无折返条件以动态调整路径长度。进一步引入延迟拒绝框架生成多个候选样本,提升计算效率,并设计了在预热阶段自动调优超参数的方法。在一系列合成数据与真实世界案例中,相较于最先进的NUTS等采样器,ATLAS展现出:(i)在复杂几何分布上能准确采样,而NUTS无法做到;(ii)在简单分布上计算开销与NUTS相当;(iii)对超参数设定更具鲁棒性。
原文摘要 · Abstract (English)
Hamiltonian Monte-Carlo (HMC) and its auto-tuned variant, the No U-Turn Sampler (NUTS) can struggle to accurately sample distributions with complex geometries, e.g., varying curvature, due to their constant step size for leapfrog integration and fixed mass matrix. In this work, we develop a strategy to locally adapt the step size parameter of HMC at every iteration by evaluating a low-rank approximation of the local Hessian and estimating its largest eigenvalue. We combine it with a strategy to similarly adapt the trajectory length by monitoring the no U-turn condition, resulting in an adaptive sampler, ATLAS: adapting trajectory length and step-size. We further use a delayed rejection framework for making multiple proposals that improves the computational efficiency of ATLAS, and develop an approach for automatically tuning its hyperparameters during warmup. We compare ATLAS with state-of-the-art samplers like NUTS on a suite of synthetic and real world examples, and show that i) unlike NUTS, ATLAS is able to accurately sample difficult distributions with complex geometries, ii) it is computationally competitive to NUTS for simpler distributions, and iii) it is more robust to the tuning of hyperparamters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。