提出一种闭式谱方法，高效估算概率模型的相对密度。
A Spectral Framework for Closed-Form Relative Density Estimation
- 用特征一阶二阶矩构造闭式谱公式
- 推导出对数密度势和散度的显式估计器
- 适合需快速估算密度的模型开发与优化
我们提出一种闭式谱框架，用于线性参数化概率模型中的相对对数密度估计，包括未归一化和条件模型。该方法将Kullback-Leibler（KL）散度表示为加权卡方散度的积分，将KL估计转化为一系列最小二乘问题。基于一阶和二阶特征矩，推导出仅依赖于这些矩的显式谱公式，从而获得固定特征下的散度与对数密度势的闭式估计器。该框架可扩展至广泛的f-散度，并可与核化或神经网络特征学习结合。我们证明了估计器的收敛性，并在合成数据上与基于优化的变分方法（如逻辑回归和Softmax回归）进行了实证比较。
原文摘要 · Abstract (English)
We propose a closed-form spectral framework for relative log-density estimation in linearly parameterized probabilistic models, including unnormalized and conditional models. This is achieved by representing the Kullback-Leibler (KL) divergence as an integral of weighted chi-squared divergences, converting KL estimation into a family of least-squares problems. We derive an explicit spectral formula based only on first- and second-order feature moments, yielding closed-form estimators of both divergences and log-density potentials for fixed features. The framework extends to a broad class of f-divergences and can be combined with kernelization or feature learning with neural networks. We prove convergence guarantees for the resulting estimators and empirically compare them on synthetic data with optimization-based variational formulations, including logistic and softmax regression for normalized conditional models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。