用精确得分信息提升神经似然代理模型训练效率,大幅降低计算成本。
Keeping Score: Efficiency Improvements in Neural Likelihood Surrogate Training via Score-Augmented Loss Functions

- 在分类损失中加入真实梯度信息,优化神经似然代理训练。
- 相同训练时间下,性能相当于10倍数据量的常规方法。
- 适合结构化随机过程建模,尤其对仿真成本高的场景有效。
对于随机过程模型,参数推断常受限于计算昂贵的似然函数。模拟推断(SBI)通过构建可复用的似然代理绕过此限制,但多数SBI方法假设数据生成过程为黑箱。尽管这些代理在无限训练数据下是精确的,实际中需在模型质量与仿真成本间权衡。本文放宽了SBI的黑箱假设,针对神经网络似然代理的概率分类训练,提出在标准二元交叉熵损失中加入精确得分信息∇θ log p(x|θ),并基于损失梯度自适应加权。我们在网络动力学和空间过程案例上验证该方法,结果表明:相比生成更多训练数据,本方法显著降低计算成本,某些情况下以不到1.1倍训练时间实现等效于10倍数据量的下游推断性能。
原文摘要 · Abstract (English)
For stochastic process models, parameter inference is often severely bottlenecked by computationally expensive likelihood functions. Simulation-based inference (SBI) bypasses this restriction by constructing amortized surrogate likelihoods, but most SBI methods assume a black-box data generating process. While these surrogates are exact in the limit of infinite training data, practical scenarios force a strict tradeoff between model quality and simulation cost. In this work, we loosen the black-box assumption of SBI to improve this tradeoff for structured stochastic process models. Specifically, for neural network likelihood surrogates trained via probabilistic classification, we propose to augment the standard binary cross-entropy loss with exact score information $\nabla_θ\log p(x \mid θ)$ and adaptive weighting based on loss gradients. We evaluate our approach on case studies involving network dynamics and spatial processes, demonstrating that our method improves surrogate quality at a drastically lower computational cost than generating more training data. Notably, in some cases, our approach achieves downstream inference performance equivalent to a 10x increase in training data with less than a 1.1x increase in training time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。