arXiv:2507.02634cs.LG2025-07

让神经网络自己生成任务,实现更高阶的元学习。

High-Order Deep Meta-Learning with Category-Theoretic Interpretation

  • 通过生成虚拟任务,让模型学会跨任务的软约束与通用规则。
  • 能主动发现难解任务,迭代优化约束区域,提升泛化能力。
  • 适合研究元学习、自动推理和通用人工智能的学者。

我们提出一种分层深度学习框架,支持递归的高阶元学习,使神经网络能够构建、求解并跨任务层级进行泛化。核心是生成机制,可创建‘虚拟任务’——合成问题实例,帮助元学习器在相关任务间学习‘软约束’与未知通用规则。这使框架能自主生成有信息量、任务相关的数据集,摆脱对人工标注数据的依赖。元学习器通过主动探索虚拟任务空间,寻找低层学习者难以解决的任务,迭代优化约束区域,增强归纳偏置,正则化适应过程,并生成新颖、未预见的任务与约束,以支持泛化。每一层元学习对应下层问题的逐步抽象,实现结构化且可解释的学习进展。将元学习器视为范畴论中的函子,可生成并条件化下属学习者的层次结构,建立支持抽象与知识迁移的组合式架构。该视角统一了现有元学习模型,揭示学习过程可通过函子关系变换与比较,同时提供实际设计原则。我们推测此架构或可支撑下一代神经网络,实现自主生成新且有指导意义的任务及其解决方案,推动机器学习向通用人工智能演进。

原文摘要 · Abstract (English)

We introduce a new hierarchical deep learning framework for recursive higher-order meta-learning that enables neural networks (NNs) to construct, solve, and generalise across hierarchies of tasks. Central to this approach is a generative mechanism that creates \emph{virtual tasks} -- synthetic problem instances designed to enable the meta-learner to learn \emph{soft constraints} and unknown generalisable rules across related tasks. Crucially, this enables the framework to generate its own informative, task-grounded datasets thereby freeing machine learning (ML) training from the limitations of relying entirely on human-generated data. By actively exploring the virtual point landscape and seeking out tasks lower-level learners find difficult, the meta-learner iteratively refines constraint regions. This enhances inductive biases, regularises the adaptation process, and produces novel, unanticipated tasks and constraints required for generalisation. Each meta-level of the hierarchy corresponds to a progressively abstracted generalisation of problems solved at lower levels, enabling a structured and interpretable learning progression. By interpreting meta-learners as category-theoretic \emph{functors} that generate and condition a hierarchy of subordinate learners, we establish a compositional structure that supports abstraction and knowledge transfer across progressively generalised tasks. The category-theoretic perspective unifies existing meta-learning models and reveals how learning processes can be transformed and compared through functorial relationships, while offering practical design principles for structuring meta-learning. We speculate this architecture may underpin the next generation of NNs capable of autonomously generating novel, instructive tasks and their solutions, thereby advancing ML towards general artificial intelligence.

元学习高阶学习范畴论自动任务生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。