提出新型分布对齐方法，可高效无偏地将源分布迁移到目标分布。
(De)-regularized Maximum Mean Discrepancy Gradient Flow
- 通过去正则化MMD与χ²散度的关联，实现近全局收敛。
- 仅用样本即可闭式计算，无需复杂假设或噪声注入。
- 适合大规模训练场景，如师生网络蒸馏，性能稳定可靠。
我们提出一种最大均值差异（DrMMD）及其Wasserstein梯度流的去正则化形式。现有仅依赖目标样本的梯度流方法，要么难以数值实现（f-散度流），要么需强假设或噪声注入才能保证收敛（MMD流）。相比之下，DrMMD流能同时满足：(i) 在连续和离散时间下对广泛的目标分布实现近全局收敛；(ii) 仅用样本即可闭式实现。前者得益于DrMMD与χ²散度的联系，后者源于将DrMMD视为去正则化核的MMD。我们的数值方案采用自适应去正则化调度，以最优平衡离散化误差与偏离χ²区间的偏差。在多个数值实验中验证了其潜力，包括大规模师生网络训练场景。
原文摘要 · Abstract (English)
We introduce a (de)-regularization of the Maximum Mean Discrepancy (DrMMD) and its Wasserstein gradient flow. Existing gradient flows that transport samples from source distribution to target distribution with only target samples, either lack tractable numerical implementation ($f$-divergence flows) or require strong assumptions, and modifications such as noise injection, to ensure convergence (Maximum Mean Discrepancy flows). In contrast, DrMMD flow can simultaneously (i) guarantee near-global convergence for a broad class of targets in both continuous and discrete time, and (ii) be implemented in closed form using only samples. The former is achieved by leveraging the connection between the DrMMD and the $χ^2$-divergence, while the latter comes by treating DrMMD as MMD with a de-regularized kernel. Our numerical scheme uses an adaptive de-regularization schedule throughout the flow to optimally trade off between discretization errors and deviations from the $χ^2$ regime. The potential application of the DrMMD flow is demonstrated across several numerical experiments, including a large-scale setting of training student/teacher networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。