arXiv:2502.00197cs.LGstat.ML2025-02

提出模型继承者概念,用归纳推理实现从简单到复杂的泛化。

Learning Model Successors

  • 通过模型继承者机制,让模型从训练数据中归纳出可无限扩展的推理原则。
  • 在难度递增的测试任务上,新方法显著优于传统泛化策略。
  • 适合关注通用学习机制与算法外推的研究者。

泛化概念已从经典统计学习理论转向对域外泛化(OODG)的关注。研究重点逐渐聚焦于从易到难的泛化,其中难度递进隐含了领域迁移的方向。这一趋势在文献中被称为长度/逻辑/算法外推,但缺乏统一定义。本文认为其核心是归纳:基于有限训练样本,学习者应推断出可无限应用的归纳原则。本文正式化了沿难度进展的归纳泛化,并主张重构学习范式。为此提出新的归纳学习范式,核心为模型继承者概念,并提供将现有技术适配为学习模型继承者的实用路径。该工作呼吁将研究讨论从碎片化的任务导向社区,转向以学习与计算的普遍特性为核心的统一努力。

原文摘要 · Abstract (English)

The notion of generalization has moved away from the classical one defined in statistical learning theory towards an emphasis on out-of-domain generalization (OODG). There has been a growing focus on generalization from easy to hard, where a progression of difficulty implicitly governs the direction of domain shifts. This emerging regime has appeared in the literature under different names, such as length/logical/algorithmic extrapolation, but a formal definition is lacking. We argue that the unifying theme is induction -- based on finite samples observed in training, a learner should infer an inductive principle that applies in an unbounded manner. This work formalizes the notion of inductive generalization along a difficulty progression and argues that our path ahead lies in transforming the learning paradigm. We attempt to make inroads by proposing a novel learning paradigm, Inductive Learning, which involves a central concept called model successors. We outline practical steps to adapt well-established techniques towards learning model successors. This work calls for restructuring of the research discussion around induction and generalization from fragmented task-centric communities to a more unified effort, focused on universal properties of learning and computation.

归纳学习泛化能力模型继承

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。