用熵引导采样，高效找到离散空间中的稳定解。
Entropy-Guided Sampling of Flat Modes in Discrete Spaces
- 引入局部熵作为辅助变量，引导采样过程
- 在多个任务中均优于传统方法，提升稳定解发现率
- 适合需要探索平坦区域的优化与生成任务
在离散空间中对平坦模式进行采样是一个关键但研究不足的问题。平坦模式代表鲁棒解，在组合优化和离散生成建模中有广泛应用。然而，现有采样算法常忽略模式体积，难以有效捕捉平坦模式。为此，我们提出熵引导离散朗之万提议（EDLP），通过联合分布下的连续辅助变量将局部熵融入采样过程。该局部熵项可引导离散采样器高效逼近平坦模式，且计算开销小。我们在局部对数凹分布下为EDLP提供了非渐近收敛保证。实验表明，该方法在伯努利分布、受限玻尔兹曼机、组合优化和二值神经网络等需从平坦基底采样的任务中持续优于传统方法。
原文摘要 · Abstract (English)
Sampling from flat modes in discrete spaces is a crucial yet underexplored problem. Flat modes represent robust solutions and have broad applications in combinatorial optimization and discrete generative modeling. However, existing sampling algorithms often overlook the mode volume and struggle to capture flat modes effectively. To address this limitation, we propose \emph{Entropic Discrete Langevin Proposal} (EDLP), which incorporates local entropy into the sampling process through a continuous auxiliary variable under a joint distribution. The local entropy term guides the discrete sampler toward flat modes with a small overhead. We provide non-asymptotic convergence guarantees for EDLP in locally log-concave discrete distributions. Empirically, our method consistently outperforms traditional approaches across tasks that require sampling from flat basins, including Bernoulli distribution, restricted Boltzmann machines, combinatorial optimization, and binary neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。