arXiv:2410.01655cs.LGmath.DS2024-10中稿 · as a conference pa…被引 2

改进自调制机制,让模型在少样本物理系统学习中更鲁棒

Reevaluating Meta-Learning Optimization Algorithms Through Contextual Self-Modulation

  • 用函数空间扩展自调制,支持无限维上下文变化
  • 采样近邻环境实现低开销元梯度更新,提升大规模数据适用性
  • 可嵌入其他框架,特别适合物理系统等分布外任务

上下文自调制(CSM)是神经上下文流(NCF)的强正则化机制,在物理系统少样本元学习中表现优异。但其在多模态和高数据量场景下应用受限。本文提出两项扩展:iCSM 将上下文嵌入函数空间,实现无限维变体;StochasticNCF 通过采样最近邻环境提供低成本的元梯度近似,提升可扩展性。实验覆盖动力系统、计算机视觉与曲线拟合任务。结合高阶泰勒展开与自动微分发现,更高阶近似未必提升泛化性能。此外,通过 FlashCAVIA 将 CSM 融入其他元学习框架。结果表明,CSM 在元学习与分布外任务中优势显著,尤其适用于物理系统。开源库已发布于 https://github.com/ddrous/self-mod,支持模块化集成。

原文摘要 · Abstract (English)

Contextual Self-Modulation (CSM) (Nzoyem et al., 2025) is a potent regularization mechanism for Neural Context Flows (NCFs) which demonstrates powerful meta-learning on physical systems. However, CSM has limitations in its applicability across different modalities and in high-data regimes. In this work, we introduce two extensions: $i$CSM which expands CSM to infinite-dimensional variations by embedding the contexts into a function space, and StochasticNCF which improves scalability by providing a low-cost approximation of meta-gradient updates through a sampled set of nearest environments. These extensions are demonstrated through comprehensive experimentation on a range of tasks, including dynamical systems, computer vision challenges, and curve fitting problems. Additionally, we incorporate higher-order Taylor expansions via Taylor-Mode automatic differentiation, revealing that higher-order approximations do not necessarily enhance generalization. Finally, we demonstrate how CSM can be integrated into other meta-learning frameworks with FlashCAVIA, a computationally efficient extension of the CAVIA meta-learning framework (Zintgraf et al., 2019). Together, these contributions highlight the significant benefits of CSM and indicate that its strengths in meta-learning and out-of-distribution tasks are particularly well-suited to physical systems. Our open-source library, designed for modular integration of self-modulation into contextual meta-learning workflows, is available at https://github.com/ddrous/self-mod.

元学习自调制物理系统少样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。