揭示密度比估计中Lp误差的理论边界，解析其与KL散度的关系。
Bounds on Lp errors in density ratio estimation via f-divergence loss functions
- 基于f-散度损失推导出Lp误差上下界，适用于任意Lipschitz连续估计器。
- 误差随数据维度和密度比的p次幂期望增长，当p>1时受KL散度指数影响。
- 为高维分布比较提供理论依据，适合关注密度比估计稳定性的研究者。
密度比估计（DRE）是机器学习中捕捉两个概率分布间关系的核心技术。基于f-散度变分表示的损失函数已成为实现前沿性能的标准选择。本文通过推导基于f-散度损失函数的Lp误差上界与下界，为DRE提供了新的理论洞见。这些边界适用于任意Lipschitz连续估计器，不依赖具体f-散度损失函数。边界表达式包含数据维度与密度比的p次幂期望的乘积。值得注意的是，下界包含一个依赖于Kullback-Leibler（KL）散度的指数项，表明当p > 1时，随着KL散度增大，Lp误差显著上升，且该趋势随p增大而加剧。理论结果通过数值实验得到验证。
原文摘要 · Abstract (English)
Density ratio estimation (DRE) is a core technique in machine learning used to capture relationships between two probability distributions. $f$-divergence loss functions, which are derived from variational representations of $f$-divergence, have become a standard choice in DRE for achieving cutting-edge performance. This study provides novel theoretical insights into DRE by deriving upper and lower bounds on the $L_p$ errors through $f$-divergence loss functions. These bounds apply to any estimator belonging to a class of Lipschitz continuous estimators, irrespective of the specific $f$-divergence loss function employed. The derived bounds are expressed as a product involving the data dimensionality and the expected value of the density ratio raised to the $p$-th power. Notably, the lower bound includes an exponential term that depends on the Kullback--Leibler (KL) divergence, revealing that the $L_p$ error increases significantly as the KL divergence grows when $p > 1$. This increase becomes even more pronounced as the value of $p$ grows. The theoretical insights are validated through numerical experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。