用温度交换提升离散采样效率，避免陷入局部最优。
Enhancing Gradient-based Discrete Sampling via Parallel Tempering
- 结合多温度链与能量交换机制，改进离散Langevin采样。
- 在合成数据、受限玻尔兹曼机等任务中采样效果显著优于单链方法。
- 自动确定温度配置，适配不同任务，减少人工调参。
尽管基于梯度的离散采样器在复杂分布采样中表现有效，但在高维多模态离散分布中易受局部极小值困扰，原因在于其固有的不连续性。为此，本文将平行温度法（parallel tempering，又称副本交换）与离散Langevin提案结合，提出并行温度增强的离散Langevin提案（PTDLP），在一系列温度下模拟多个链。当能量差异显著时，依据专为离散采样设计的Metropolis准则进行样本交换，确保细致平衡。此外，我们引入自动温度调度与链数确定方案，实现跨任务自适应，几乎无需调参。理论上，算法非渐近收敛于目标能量分布，且混合速度优于单链方法。实验表明，该方法在合成问题、受限玻尔兹曼机及深层能量模型等复杂多模态分布采样中具有显著优势。
原文摘要 · Abstract (English)
While gradient-based discrete samplers are effective in sampling from complex distributions, they are susceptible to getting trapped in local minima, particularly in high-dimensional, multimodal discrete distributions, owing to the discontinuities inherent in these landscapes. To circumvent this issue, we combine parallel tempering, also known as replica exchange, with the discrete Langevin proposal and develop the Parallel Tempering enhanced Discrete Langevin Proposal (PTDLP), which are simulated at a series of temperatures. Significant energy differences prompt sample swaps, which are governed by a Metropolis criterion specifically designed for discrete sampling to ensure detailed balance is maintained. Additionally, we introduce an automatic scheme to determine the optimal temperature schedule and the number of chains, ensuring adaptability across diverse tasks with minimal tuning. Theoretically, we establish that our algorithm converges non-asymptotically to the target energy and exhibits faster mixing compared to a single chain. Empirical results further emphasize the superiority of our method in sampling from complex, multimodal discrete distributions, including synthetic problems, restricted Boltzmann machines, and deep energy-based models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。