改进神经过程推理，提升函数分布建模精度
Bridge the Inference Gaps of Neural Processes via Expectation Maximization
- 用期望最大化框架设计新目标函数，优化推理过程
- 在多个数据集上显著提升对数似然，优于现有神经过程方法
- 适合需要精准函数建模的少样本学习场景
神经过程（NP）是一类计算高效的函数分布学习模型，但实际应用中常因欠拟合导致性能不佳。以往研究主要关注引入注意力或卷积等结构归纳偏置，而对推理次优性及从优化目标角度分析NP的工作极少。本文提出在期望最大化框架下，针对元数据集目标对数似然的代理目标函数。所提模型——自归一化重要性加权神经过程（SI-NP），可学习更准确的函数先验，并保证目标对数似然的提升。实验表明，SI-NP在多个基准上表现优异，且结合注意力等结构归纳偏置后可达当前最优性能。代码已开源。
原文摘要 · Abstract (English)
The neural process (NP) is a family of computationally efficient models for learning distributions over functions. However, it suffers from under-fitting and shows suboptimal performance in practice. Researchers have primarily focused on incorporating diverse structural inductive biases, \textit{e.g.} attention or convolution, in modeling. The topic of inference suboptimality and an analysis of the NP from the optimization objective perspective has hardly been studied in earlier work. To fix this issue, we propose a surrogate objective of the target log-likelihood of the meta dataset within the expectation maximization framework. The resulting model, referred to as the Self-normalized Importance weighted Neural Process (SI-NP), can learn a more accurate functional prior and has an improvement guarantee concerning the target log-likelihood. Experimental results show the competitive performance of SI-NP over other NPs objectives and illustrate that structural inductive biases, such as attention modules, can also augment our method to achieve SOTA performance. Our code is available at \url{https://github.com/hhq123gogogo/SI_NPs}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。