让模型在运行时灵活条件化隐变量，提升生成与推理能力
Amortized Probabilistic Conditioning for Optimization, Simulation and Inference
- 用Transformer显式建模关键隐变量，支持动态条件化
- 在图像补全、贝叶斯优化等任务上表现优于基线方法
- 适合需要可解释性条件推理的科研与工程场景
基于预训练的摊销元学习方法已推动自然语言处理与视觉领域发展。基于Transformer的神经过程及其变体是概率元学习的主流模型，具有可计算的目标函数。这些模型通常在合成数据上训练，隐式捕捉了数据生成过程中的关键潜在信息。然而，现有方法无法在运行时灵活地注入（条件化）和提取（预测）这些概率潜在信息，而这正是许多任务的核心需求。本文提出摊销条件引擎（ACE），一种新的基于Transformer的元学习模型，显式表示感兴趣的潜在变量。ACE支持对观测数据和可解释潜在变量进行条件化，可在运行时引入先验，并输出离散与连续数据及潜在变量的预测分布。我们在图像补全、分类、贝叶斯优化和基于模拟的推断等多样任务中验证了ACE的建模灵活性与性能优势。
原文摘要 · Abstract (English)
Amortized meta-learning methods based on pre-training have propelled fields like natural language processing and vision. Transformer-based neural processes and their variants are leading models for probabilistic meta-learning with a tractable objective. Often trained on synthetic data, these models implicitly capture essential latent information in the data-generation process. However, existing methods do not allow users to flexibly inject (condition on) and extract (predict) this probabilistic latent information at runtime, which is key to many tasks. We introduce the Amortized Conditioning Engine (ACE), a new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents. We show ACE's modeling flexibility and performance in diverse tasks such as image completion and classification, Bayesian optimization, and simulation-based inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。