改进泊松采样梯度估计,提升神经科学模型训练稳定性
A hitchhiker's guide to Poisson gradient estimation
- 改进EAT方法,确保一阶矩无偏且降低二阶偏差
- 在变分自编码器和神经连接推断任务中表现优于原版EAT与GSM
- 特别适合对超参数不敏感的泊松/负二项潜变量建模场景
泊松分布潜变量模型广泛应用于计算神经科学,但离散随机采样的梯度难以计算。现有两种方法:指数到达时间(EAT)模拟与Gumbel-SoftMax(GSM)松弛。本文首次系统比较二者,并提供实践指导。主要技术贡献是改进EAT方法,理论上保证一阶矩无偏(精确匹配发放率),并减少二阶矩偏差。我们在分布保真度、梯度质量及两类任务上评估:(1) 泊松潜变量的变分自编码器;(2) 部分可观测广义线性模型,需从观测尖峰序列推断神经连接。在所有指标上,改进EAT均表现更优(常接近精确梯度),且对超参数选择更鲁棒。该结果扩展至过分散负二项潜变量时依然成立。然而,仅GSM可推广至任意非泊松分布,包括欠分散情形。研究厘清了两方法权衡,为相关领域实践者提供明确建议。
原文摘要 · Abstract (English)
Poisson-distributed latent variable models are widely used in computational neuroscience, but differentiating through discrete stochastic samples remains challenging. Two approaches address this: *Exponential Arrival Time* (EAT) simulation and *Gumbel-SoftMax* (GSM) relaxation. We provide the first systematic comparison of these methods, along with practical guidance for practitioners. Our main technical contribution is a modification to the EAT method that theoretically guarantees an unbiased first moment (exactly matching the firing rate), and reduces second-moment bias. We evaluate these methods on their distributional fidelity, gradient quality, and performance on two tasks: (1) variational autoencoders with Poisson latents, and (2) partially observable generalized linear models, where latent neural connectivity must be inferred from observed spike trains. Across all metrics, our modified EAT method exhibits better overall performance (often comparable to exact gradients), and substantially higher robustness to hyperparameter choices. These results extend to over-dispersed Negative Binomial latents, where modified EAT again performs best. However, only GSM generalizes to arbitrary non-Poisson distributions, including the under-dispersed regime. Together, our results clarify the trade-offs between these methods and offer concrete recommendations for practitioners working with Poisson latent variable models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。