arXiv:2605.30132cs.LGstat.ML2026-05

让模型学会从已知任务推断未知任务,提升泛化能力。

Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolation

论文配图:Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolation
图 1 · 摘自论文原文
  • 通过学习任务间的变换关系实现外推
  • 在参数、组合深度等场景下显著优于现有方法
  • 适合需要跨任务泛化的基础模型应用

现代学习系统擅长插值但难以推广到训练分布之外的未见任务。即使在简单场景中,如处理超出训练范围的任务参数,这种失败依然存在,且在基础模型发展背景下仍未解决。为此,我们提出关系式任务外推器(RTE),一种支持系统性外推的新算法。核心思想是:外推本质上是关系性的——若模型学会任务A与任务B之间的变换关系,即可在测试时用该变换将已知任务映射至未见任务。RTE将每个目标任务分解为一个已知锚点任务与一个连接锚点和目标的变换,并学习一个关系算子,将锚点-变换对映射到目标任务的预测结果。我们在函数预测多个外推场景中实现RTE,包括参数外推(超出训练范围)、长度外推(更高组合深度)及组合外推(以新方式重组函数原语)。此外,我们将RTE扩展至序列预测,集成进基础模型微调算法。实验表明,RTE在未见任务外推上显著优于现有方法。

原文摘要 · Abstract (English)

Modern learning systems excel at interpolation but struggle to generalize to unseen tasks outside the training distribution's support. This failure occurs even in simple settings, such as handling task parameters beyond the training range, and persists despite advances in foundation models. To this end, we develop the Relational Task Extrapolator (RTE), an algorithm designed to enable systematic extrapolation to novel tasks. The key observation is that extrapolation is inherently relational: extrapolating to unseen tasks requires learning how tasks transform into one another. If a model learns the transformation between tasks A and B during training, it can apply that same transformation to relate known tasks to unseen ones at test time. RTE operationalizes this idea by decomposing each target task into a known anchor task and a transformation linking the anchor and target. It then learns a relational operator, mapping an anchor-transformation pair to predictions for the target task. We instantiate RTE across multiple task extrapolation regimes in function prediction, e.g. where target tasks use out-of-range parameters (parameter extrapolation), have greater compositional depth (length extrapolation), and/or recombine function primitives in unseen ways (compositional extrapolation). We further extend RTE to sequence prediction, integrating it into fine-tuning algorithms for foundation models. Across empirical studies, we find that RTE substantially outperforms existing approaches on extrapolation to novel, unseen tasks.

任务外推关系学习泛化能力基础模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。