用非对抗方式优化对称KL散度，提升生成模型稳定性与准确性。
Adaptive Symmetrization of the KL Divergence
- 引入代理模型近似反向KL，避免对抗训练的不稳定性。
- 联合优化主模型与代理模型，在低数据场景下更准确。
- 适用于密度估计和模拟推断，尤其适合数据稀缺任务。
前向KL散度因可计算性与最大似然等价性，常用于拟合参数化分布。然而其固有的不对称性可能导致泛化能力差的退化解。对称的Jeffreys散度虽更平衡，但优化困难，因其包含反向KL项。生成对抗网络（GANs）通过最小-最大公式规避此问题，却引入新不稳定性。本文提出一种非对抗方法以最小化Jeffreys散度：使用代理模型可计算地近似主模型的反向KL。主模型与代理模型通过约束优化联合拟合数据，实现训练中动态调整模型优先级。我们在密度估计和基于模拟的推断等多个任务上评估该框架，结果表明其在低数据情形下比MLE和GANs更稳定、更准确。
原文摘要 · Abstract (English)
The forward Kullback-Leibler (KL) divergence is a ubiquitous objective for fitting a parameterized distribution to samples due to its tractability and equivalence to maximum likelihood estimation (MLE). Its inherent asymmetry, however, may lead to degenerate solutions that generalize poorly. While the symmetric Jeffreys divergence offers a more balanced alternative, its optimization is challenging due to the presence of a reverse KL term. Generative adversarial networks (GANs) bypass this intractability using a min-max formulation at the cost of introducing new instability issues. This work proposes a non-adversarial approach to minimize the Jeffreys divergence. To do so, it uses a proxy model to tractably approximate the reverse KL divergence of the main model. The main and proxy models are jointly fitted to the data using a constrained optimization formulation to obtain a practical algorithm that adapts the models' priorities throughout training. We evaluate our framework on various tasks, including density estimation and simulation-based inference, and demonstrate that this approach is more stable and more accurate than MLE and GANs, particularly in low-data regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。