arXiv:2411.19451cs.CVcs.AI2024-11AAAI被引 15

用双路网络模拟人类视觉推理,提升图像逻辑题解题能力。

Learning Visual Abstract Reasoning through Dual-Stream Networks

  • 双分支结构分别捕捉局部与空间特征,协同完成视觉推理。
  • 在多个RPM基准上达到当前最优平均性能,泛化能力强。
  • 适合研究视觉认知机制或需抽象推理的AI模型开发者。

视觉抽象推理任务对深度神经网络构成挑战,暴露了其能力局限。本文提出双流推理网络(DRNet),受视觉处理两流假说启发,采用两条并行分支提取图像特征。在双流基础上,推理模块首先融合同一图像的高层特征,再通过规则提取器处理八个上下文图像与每个候选图像的组合,提取离散抽象规则,并利用多层感知机(MLP)进行预测。实验表明,所提DRNet在多个RPM基准上实现当前最优平均性能,且在多种分布外场景中仍具强泛化能力。双流分别负责局部与空间信息,经推理模块整合后,借助抽象规则完成视觉推理。结果表明,双流架构在视觉抽象推理中具有关键作用。

原文摘要 · Abstract (English)

Visual abstract reasoning tasks present challenges for deep neural networks, exposing limitations in their capabilities. In this work, we present a neural network model that addresses the challenges posed by Raven's Progressive Matrices (RPM). Inspired by the two-stream hypothesis of visual processing, we introduce the Dual-stream Reasoning Network (DRNet), which utilizes two parallel branches to capture image features. On top of the two streams, a reasoning module first learns to merge the high-level features of the same image. Then, it employs a rule extractor to handle combinations involving the eight context images and each candidate image, extracting discrete abstract rules and utilizing an multilayer perceptron (MLP) to make predictions. Empirical results demonstrate that the proposed DRNet achieves state-of-the-art average performance across multiple RPM benchmarks. Furthermore, DRNet demonstrates robust generalization capabilities, even extending to various out-of-distribution scenarios. The dual streams within DRNet serve distinct functions by addressing local or spatial information. They are then integrated into the reasoning module, leveraging abstract rules to facilitate the execution of visual reasoning tasks. These findings indicate that the dual-stream architecture could play a crucial role in visual abstract reasoning.

视觉推理双流网络抽象思维

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。