arXiv:2410.02628cs.LGcs.AI2024-10被引 2

用数据似然最大化融合有标签和无标签数据,提升半监督学习效果。

Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization

  • 基于数据似然最大化,统一处理配对与非配对样本。
  • 理论证明可逼近任意精确的条件分布,误差可控。
  • 适合缺乏标注数据、但有大量无标签样本的场景。

学习条件分布 $π^*(\cdot|x)$ 是机器学习的核心问题,传统方法依赖成对数据 $(x,y) \sim π^*$ 进行监督学习。然而在领域转换等任务中,获取成对数据极为困难。因此需要半监督模型,同时利用少量成对数据与额外的独立同分布无标签样本 $x \sim π^*_x$ 和 $y \sim π^*_y$。现有方法常依赖启发式策略,难以有效融合两类数据。本文提出新范式 EBiEOT,通过数据似然最大化无缝整合配对与非配对数据,并发现其与逆熵正则最优传输(inverse entropic OT)存在深刻联系。该联系使我们能借助近期最优传输计算进展,构建端到端学习算法以估计 $π^*(\cdot|x)$。此外,我们推导出通用逼近性质,证明该方法理论上可任意小误差地恢复真实条件分布。实验表明,该方法能有效利用配对与非配对数据联合学习条件分布。代码已公开于 https://github.com/MuXauJl11110/EBiEOT。

原文摘要 · Abstract (English)

Learning conditional distributions $π^*(\cdot|x)$ is a central problem in machine learning, which is typically approached via supervised methods with paired data $(x,y) \sim π^*$. However, acquiring paired data samples is often challenging, especially in problems such as domain translation. This necessitates the development of $\textit{semi-supervised}$ models that utilize both limited paired data and additional unpaired i.i.d. samples $x \sim π^*_x$ and $y \sim π^*_y$ from the marginal distributions. The usage of such combined data is complex and often relies on heuristic approaches. To tackle this issue, we propose a new learning paradigm called $\textbf{EBiEOT}$ that integrates both paired and unpaired data seamlessly using data likelihood maximization techniques. We demonstrate that our approach also connects intriguingly with inverse entropic optimal transport (OT). This finding allows us to apply recent advances in computational OT to establish an $\textit{end-to-end}$ learning algorithm to get $π^*(\cdot|x)$. In addition, we derive the universal approximation property, demonstrating that our approach can theoretically recover true conditional distributions with arbitrarily small error. Finally, we demonstrate through empirical tests that our method effectively learns conditional distributions using paired and unpaired data simultaneously. The code of $\texttt{EBiEOT}$ is available at https://github.com/MuXauJl11110/EBiEOT.

半监督学习最优传输条件分布

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。