用可学习的丢弃率快速适配新任务，提升小样本元学习鲁棒性
Neural Variational Dropout Processes
- 基于任务特定丢弃率建模后验，低秩伯努利专家实现高效映射
- 在1D回归、图像修复和分类任务上优于现有方法，支持多任务少样本学习
- 新先验机制让丢弃率适应性强，适合处理复杂不确定性场景
在小样本元学习中，推断条件后验模型是关键。本文提出一种新的贝叶斯元学习方法——神经变分丢弃过程（NVDPs）。NVDPs基于任务特定的丢弃率建模条件后验分布，采用低秩伯努利专家元模型，以内存高效方式将丢弃率从少量观测上下文映射出来。该方法可快速重配置全局共享的神经网络，以适应新任务。此外，NVDPs引入一种依赖全任务数据的新先验，用于优化变分推断中的条件丢弃后验。令人惊讶的是，这使得对任务特异性丢弃率的稳健近似成为可能，能有效应对广泛的功能模糊性和不确定性。我们在1D随机回归、图像修复和分类等少样本学习任务中对比了该方法与其他元学习方法，结果表明NVDPs表现优异。
原文摘要 · Abstract (English)
Learning to infer the conditional posterior model is a key step for robust meta-learning. This paper presents a new Bayesian meta-learning approach called Neural Variational Dropout Processes (NVDPs). NVDPs model the conditional posterior distribution based on a task-specific dropout; a low-rank product of Bernoulli experts meta-model is utilized for a memory-efficient mapping of dropout rates from a few observed contexts. It allows for a quick reconfiguration of a globally learned and shared neural network for new tasks in multi-task few-shot learning. In addition, NVDPs utilize a novel prior conditioned on the whole task data to optimize the conditional \textit{dropout} posterior in the amortized variational inference. Surprisingly, this enables the robust approximation of task-specific dropout rates that can deal with a wide range of functional ambiguities and uncertainties. We compared the proposed method with other meta-learning approaches in the few-shot learning tasks such as 1D stochastic regression, image inpainting, and classification. The results show the excellent performance of NVDPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。