arXiv:2412.13636cs.CVcs.AI2024-12AAAI被引 1

提出多层级一致组合泛化框架,提升模型对复杂组合的统一理解能力。

Consistency of Compositional Generalization across Multiple Levels

  • 基于元学习分层优化,从简单到复杂逐步训练组合关系。
  • 在多层级组合任务上均实现显著性能提升,验证了跨层级一致性。
  • 适合关注模型泛化能力与组合推理的研究者使用。

组合泛化指模型理解由已见概念构成的新组合的能力,包括短语-短语、短语-词、词-词等多个层级。现有方法虽在组合泛化上表现良好,但不同层级间的一致性仍缺乏研究。本文提出一种基于元学习的框架,通过逐步学习从简单到复杂的组合以实现跨层级一致性。具体地,将训练集按组合复杂度划分为多个验证集,并引入多个元权重网络为不同验证集生成样本权重;按复杂度递增顺序,独立且顺序优化各元权重网络参数。构建GQA-CCG数据集用于定量评估一致性。在视觉问答和时序视频定位任务上的实验表明,该框架有效提升了多层级组合泛化能力。代码与数据集已开源。

原文摘要 · Abstract (English)

Compositional generalization is the capability of a model to understand novel compositions composed of seen concepts. There are multiple levels of novel compositions including phrase-phrase level, phrase-word level, and word-word level. Existing methods achieve promising compositional generalization, but the consistency of compositional generalization across multiple levels of novel compositions remains unexplored. The consistency refers to that a model should generalize to a phrase-phrase level novel composition, and phrase-word/word-word level novel compositions that can be derived from it simultaneously. In this paper, we propose a meta-learning based framework, for achieving consistent compositional generalization across multiple levels. The basic idea is to progressively learn compositions from simple to complex for consistency. Specifically, we divide the original training set into multiple validation sets based on compositional complexity, and introduce multiple meta-weight-nets to generate sample weights for samples in different validation sets. To fit the validation sets in order of increasing compositional complexity, we optimize the parameters of each meta-weight-net independently and sequentially in a multilevel optimization manner. We build a GQA-CCG dataset to quantitatively evaluate the consistency. Experimental results on visual question answering and temporal video grounding, demonstrate the effectiveness of the proposed framework. We release GQA-CCG at https://github.com/NeverMoreLCH/CCG.

组合泛化元学习视觉问答多层级

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。