提出RIME方法,让元学习在干扰变化下同时学会该学和不该学的内容。
Meta Learning not to Learn: Robustly Informing Meta-Learning under Nuisance-Varying Families
- 设计新框架,让元学习同时利用正负归纳偏置
- 在干扰变化的场景中实现最先进分布鲁棒性能
- 适合需要跨医院等异质数据泛化的医疗图像任务
当存在虚假与因果预测因子时,标准神经网络在经验风险最小化下易依赖虚假特征。为获得可泛化的假设,需引入额外归纳偏置。在相关任务中(如不同医院的医学影像预后预测),虚假特征常共现,使泛化更困难。此时方法需能同时整合恰当的归纳偏置,以在干扰变化家族与任务家族间实现泛化。本文提出RIME(Robustly Informed Meta lEarning),一种在存在正负归纳偏置(即知道学什么、不学什么)下的元学习新方法。首先构建理论因果框架,说明现有知识融合方法可能在分布鲁棒目标上表现更差;随后证明RIME能同时整合两类偏置,在干扰变化家族的有指导元学习设置中达到当前最优分布鲁棒性能。
原文摘要 · Abstract (English)
In settings where both spurious and causal predictors are available, standard neural networks trained under the objective of empirical risk minimization (ERM) with no additional inductive biases tend to have a dependence on a spurious feature. As a result, it is necessary to integrate additional inductive biases in order to guide the network toward generalizable hypotheses. Often these spurious features are shared across related tasks, such as estimating disease prognoses from image scans coming from different hospitals, making the challenge of generalization more difficult. In these settings, it is important that methods are able to integrate the proper inductive biases to generalize across both nuisance-varying families as well as task families. Motivated by this setting, we present RIME (Robustly Informed Meta lEarning), a new method for meta learning under the presence of both positive and negative inductive biases (what to learn and what not to learn). We first develop a theoretical causal framework showing why existing approaches at knowledge integration can lead to worse performance on distributionally robust objectives. We then show that RIME is able to simultaneously integrate both biases, reaching state of the art performance under distributionally robust objectives in informed meta-learning settings under nuisance-varying families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。