用可表达的得分模型做分布匹配,更稳定高效且保持几何结构。
Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization
- 用去噪得分匹配训练得分先验,无需显式密度模型。
- 在多个任务上优于扩散基方法,避免模式崩溃与不稳定性。
- 适合需要稳定分布对齐的迁移学习与公平分类场景。
分布匹配(DM)是一种通用的域不变表示学习技术,已应用于公平分类、域自适应和域翻译等任务。非参数化DM方法存在可扩展性问题,对抗式方法则易出现不稳定和模式崩溃。基于似然的方法虽有潜力,但常因固定先验引入不必要的偏差,或需难以训练的显式密度模型(如流模型)。本文提出一种新颖的似然型DM训练方法,采用可表达的得分基先验。关键洞察在于:基于梯度的DM训练仅需先验的得分函数而非其密度,因此可通过去噪得分匹配训练先验。该方法消除了固定先验的偏差(如VAE中),实现更有效的几何保持正则化,同时避免了学习显式先验密度模型的挑战。相比其他扩散基先验(如LSGM),本方法在稳定性和计算效率上表现更优。实验表明,该方法在多任务上均取得更优性能,确立了其作为稳定高效的分布匹配方案的可行性。源码见https://github.com/inouye-lab/SAUB。
原文摘要 · Abstract (English)
Distribution matching (DM) is a versatile domain-invariant representation learning technique that has been applied to tasks such as fair classification, domain adaptation, and domain translation. Non-parametric DM methods struggle with scalability and adversarial DM approaches suffer from instability and mode collapse. While likelihood-based methods are a promising alternative, they often impose unnecessary biases through fixed priors or require explicit density models (e.g., flows) that can be challenging to train. We address this limitation by introducing a novel approach to training likelihood-based DM using expressive score-based prior distributions. Our key insight is that gradient-based DM training only requires the prior's score function -- not its density -- allowing us to train the prior via denoising score matching. This approach eliminates biases from fixed priors (e.g., in VAEs), enabling more effective use of geometry-preserving regularization, while avoiding the challenge of learning an explicit prior density model (e.g., a flow-based prior). Our method also demonstrates better stability and computational efficiency compared to other diffusion-based priors (e.g., LSGM). Furthermore, experiments demonstrate superior performance across multiple tasks, establishing our score-based method as a stable and effective approach to distribution matching. Source code available at https://github.com/inouye-lab/SAUB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。