提出新型MMD梯度流,可全局收敛且适用于采样与生成双重场景。
Sobolev Regularized MMD Gradient Flow

- 通过见证函数梯度惩罚正则化,缓解MMD目标非凸性。
- 在连续与离散时间下均实现MMD的全局收敛。
- 无需等周假设,适用于无归一化分布采样与生成建模。
我们提出基于见证函数梯度惩罚的索博列夫正则化最大均值差异(SrMMD)梯度流,一种改进的MMD梯度流。该正则化有效缓解了MMD目标的非凸性,在连续和离散时间下均提供了可证明的全局收敛性。令人意外的是,其收敛分析不依赖于目标分布的等周假设,而是基于核均值嵌入差的正则性条件。该流动的关键优势在于同时适用于从非归一化目标分布采样(使用Stein核)和生成建模任务,而以往方法通常仅适用于其中一类。在生成建模与采样任务的广泛实验中验证了其有效性。
原文摘要 · Abstract (English)
We propose Sobolev-regularized Maximum Mean Discrepancy (SrMMD) gradient flow, a regularized variant of maximum mean discrepancy (MMD) gradient flow based on a gradient penalty on the witness function. The proposed regularization mitigates the non-convexity of the MMD objective and yields provable \emph{global} convergence guarantees in MMD in both continuous and discrete time. A more surprising appeal is that our convergence analysis does not rely on isoperimetric assumptions on the target distribution. Instead, it is based on a regularity condition on the difference between kernel mean embeddings. A key highlight of the proposed flow is that it is applicable in both sampling (from an unnormalized target distribution) -- using Stein kernels -- and generative modeling settings, unlike previous works, where a gradient flow is suitable for only generative modeling or sampling but not both. The effectiveness of the proposed flow is empirically verified on a broad range of tasks in both generative modelling and sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。