提出可处理任意维度输入的神经过程模型,提升回归任务泛化能力
Dimension Agnostic Neural Processes
- 用固定维度特征转换模块解决输入维度不一致问题
- 在多种合成与真实回归任务上超越现有神经过程方法
- 适合需要快速适应新任务的通用回归场景
元学习旨在通过从多样化任务数据集中提取共享特征,训练可在少量标注数据下泛化的模型。同时,它在训练和评估中考虑预测不确定性,称为不确定性感知元学习。神经过程(Neural Process, NP)是一种典型的不确定性感知元学习方法,利用参数化神经网络构建隐式随机过程,实现对新任务的快速适应。然而,现有NP方法难以适应不同输入维度和学习到的特征,限制了其在回归任务中的广泛应用。为克服这些局限并提升NP模型作为通用回归器的实用性,本文提出维度无关神经过程(Dimension Agnostic Neural Processes, DANP)。DANP引入维度聚合模块(Dimension Aggregator Block, DAB),将输入特征映射至固定维度空间,增强模型对多样化数据的处理能力。同时,结合Transformer架构与潜在编码层,DANP能够学习更广泛且可迁移的特征。在多种合成与实际回归任务上的实验表明,DANP显著优于先前的NP变体,验证了其在突破传统NP模型局限性方面的有效性,展现出在多样化回归场景中的广阔应用潜力。
原文摘要 · Abstract (English)
Meta-learning aims to train models that can generalize to new tasks with limited labeled data by extracting shared features across diverse task datasets. Additionally, it accounts for prediction uncertainty during both training and evaluation, a concept known as uncertainty-aware meta-learning. Neural Process(NP) is a well-known uncertainty-aware meta-learning method that constructs implicit stochastic processes using parametric neural networks, enabling rapid adaptation to new tasks. However, existing NP methods face challenges in accommodating diverse input dimensions and learned features, limiting their broad applicability across regression tasks. To address these limitations and advance the utility of NP models as general regressors, we introduce Dimension Agnostic Neural Processes(DANP). DANP incorporates Dimension Aggregator Block(DAB) to transform input features into a fixed-dimensional space, enhancing the model's ability to handle diverse datasets. Furthermore, leveraging the Transformer architecture and latent encoding layers, DANP learns a wider range of features that are generalizable across various tasks. Through comprehensive experimentation on various synthetic and practical regression tasks, we empirically show that DANP outperforms previous NP variations, showcasing its effectiveness in overcoming the limitations of traditional NP models and its potential for broader applicability in diverse regression scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。