提出新方法解决模拟模型参数估计中的似然偏差问题,显著提升精度与稳定性。
Beyond likelihood ratio bias: Nested multi-time-scale stochastic approximation for likelihood-free parameter estimation
- 采用无比率嵌套多时标优化,同步追踪梯度并更新参数。
- 消除原有 $O\big(\sqrt{1/N}\big)$ 渐近偏差,加速收敛至最优速率。
- 适合高精度参数估计场景,尤其在计算成本受限时表现优异。
我们研究在似然函数未知的基于模拟的随机模型中的参数推断问题。主要难点在于以噪声蒙特卡洛估计器之比形式评估得分会引入偏差和不稳定性。为此,我们提出一种无比率嵌套多时标(NMTS)随机逼近(SA)方法,可同时追踪得分并驱动参数更新。本文对NMTS算法进行了全面的理论分析,涵盖强收敛性、渐近正态性和收敛速率。结果表明,该算法可消除原始 $O\big(\sqrt{1/N}\big)$ 的渐近偏差,并将收敛速率从 $O\big(β_k+\sqrt{1/N}\big)$ 提升至 $O\big(β_k/α_k+\sqrt{α_k/N}\big)$,其中 $N$ 为固定批次大小,$α_k$ 与 $β_k$ 为递减步长且满足 $α_k, β_k, β_k/α_k→0$。通过合理选择 $α_k$ 与 $β_k$,其收敛速率可达到多时标随机逼近文献中的最优水平。数值实验显示,在相同计算成本下,该算法可使估计精度提高一到两个数量级,适用于复杂随机系统的高效参数估计。
原文摘要 · Abstract (English)
We study parameter inference in simulation-based stochastic models where the analytical form of the likelihood is unknown. The main difficulty is that score evaluation as a ratio of noisy Monte Carlo estimators induces bias and instability, which we overcome with a ratio-free nested multi-time-scale (NMTS) stochastic approximation (SA) method that simultaneously tracks the score and drives the parameter update. We provide a comprehensive theoretical analysis of the proposed NMTS algorithm for solving likelihood-free inference problems, including strong convergence, asymptotic normality, and convergence rates. We show that our algorithm can eliminate the original asymptotic bias $O\big(\sqrt{\frac{1}{N}}\big)$ and accelerate the convergence rate from $O\big(β_k+\sqrt{\frac{1}{N}}\big)$ to $O\big(\frac{β_k}{α_k}+\sqrt{\frac{α_k}{N}}\big)$, where $N$ is the fixed batch size, $α_k$ and $β_k$ are decreasing step sizes with $α_k$, $β_k$, $β_k/α_k\rightarrow 0$. With proper choice of $α_k$ and $β_k$, our convergence rates can match the optimal rate in the multi-time-scale SA literature. Numerical experiments demonstrate that our algorithm can improve the estimation accuracy by one to two orders of magnitude at the same computational cost, making it efficient for parameter estimation in stochastic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。