arXiv:2602.14761cs.LGcs.AI2026-02中稿 · ICML

提出算法隐式学习框架,让模型跨领域跨模态泛化,性能更强且更高效。

Universal Algorithm-Implicit Learning

  • 区分显式与隐式算法学习,构建可通用的元学习理论框架
  • 在少样本任务上表现顶尖,支持20倍以上未见类别,计算量大幅降低
  • 适用于图像训练却能处理文本任务,适合需要强泛化的场景

现有元学习方法受限于固定特征和标签空间的任务分布,应用范围狭窄。同时,学术界对“通用”“通用型”等术语定义模糊,难以比较。本文提出一个元学习的理论框架,明确定义了实用通用性,并区分算法显式与隐式学习,建立可信赖的推理语言。基于此框架,我们提出TAIL——一种基于Transformer的算法隐式元学习器,可在不同领域、模态和标签配置的任务间运行。TAIL相比先前基于Transformer的元学习器有三项创新:随机投影用于跨模态特征编码,随机注入标签嵌入实现标签空间外推,以及高效的内联查询处理机制。TAIL在标准少样本基准上达到顶尖性能,并可泛化至未见领域;不仅能在仅用图像训练的情况下解决文本分类任务,还能处理比训练时多出20倍的类别,且相较以往方法实现数量级的计算效率提升。

原文摘要 · Abstract (English)

Current meta-learning methods are constrained to narrow task distributions with fixed feature and label spaces, limiting applicability. Moreover, the current meta-learning literature uses key terms like "universal" and "general-purpose" inconsistently and lacks precise definitions, hindering comparability. We introduce a theoretical framework for meta-learning which formally defines practical universality and introduces a distinction between algorithm-explicit and algorithm-implicit learning, providing a principled vocabulary for reasoning about universal meta-learning methods. Guided by this framework, we present TAIL, a transformer-based algorithm-implicit meta-learner that functions across tasks with varying domains, modalities, and label configurations. TAIL features three innovations over prior transformer-based meta-learners: random projections for cross-modal feature encoding, random injection label embeddings that extrapolate to larger label spaces, and efficient inline query processing. TAIL achieves state-of-the-art performance on standard few-shot benchmarks while generalizing to unseen domains. Unlike other meta-learning methods, it also generalizes to unseen modalities, solving text classification tasks despite training exclusively on images, handles tasks with up to 20$\times$ more classes than seen during training, and provides orders-of-magnitude computational savings over prior transformer-based approaches.

元学习通用性跨模态高效计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。