提出一步式分数密度比估计,高效准确地计算分布差异。
One-Step Score-Based Density Ratio Estimation
- 将时间得分分解为时空分量,用解析RBF框架替代数值求解。
- 仅需一次函数评估即可完成估计,避免重复计算和积分。
- 适用于分布差异大场景,适合需要快速高精度比对的场景。
密度比估计(DRE)是量化概率分布间差异的有力工具,但现有方法常在估计质量与计算效率间存在权衡。传统直接DRE方法推理高效,但在分布差异较大时性能显著下降;而基于分数的DRE方法虽更准确,却通常需大量重复函数求值与数值积分。本文提出一步式分数密度比估计(OS-DRE),一种部分解析、无需求解器的框架,融合两者优势。OS-DRE将时间得分分解为空间与时间分量,用解析径向基函数(RBF)框架表示后者,将原本不可解的时间积分转化为闭式加权和,从而消除数值求解需求,实现仅一次函数评估即可完成DRE。我们进一步分析了解析框架的逼近条件,并为有限与无限光滑时间核建立了逼近误差界,理论基础源自现有逼近论。在密度估计、持续KL散度与互信息估计、近似分布外检测等任务上的实验表明,OS-DRE在估计质量与推理效率间实现了良好平衡。
原文摘要 · Abstract (English)
Density ratio estimation (DRE) is a useful tool for quantifying discrepancies between probability distributions, but existing approaches often involve a trade-off between estimation quality and computational efficiency. Classical direct DRE methods are usually efficient at inference time, yet their performance can seriously deteriorate when the discrepancy between distributions is large. In contrast, score-based DRE methods often yield more accurate estimates in such settings, but they typically require considerable repeated function evaluations and numerical integration. We propose One-step Score-based Density Ratio Estimation (OS-DRE), a partly analytic and solver-free framework designed to combine these complementary advantages. OS-DRE decomposes the time score into spatial and temporal components, representing the latter with an analytic radial basis function (RBF) frame. This formulation converts the otherwise intractable temporal integral into a closed-form weighted sum, thereby removing the need for numerical solvers and enabling DRE with only one function evaluation. We further analyze approximation conditions for the analytic frame, and establish approximation error bounds for both finitely and infinitely smooth temporal kernels, grounding the framework in existing approximation theory. Experiments across density estimation, continual Kullback-Leibler and mutual information estimation, and near out-of-distribution detection demonstrate that OS-DRE offers a favorable balance between estimation quality and inference efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。