arXiv:2511.06761cs.AIcs.LG2025-11

用神经网络模拟人类直觉物理认知,统一处理物体属性、关系与时间。

SRNN: Spatiotemporal Relational Neural Network for Intuitive Physics Understanding

  • 基于赫布学习机制构建时空关系神经网络,分路径处理物体特征与运动规律。
  • 在CLEVRER基准上表现良好,验证了对时空关系的建模能力。
  • 模型可解释性强,适合研究智能系统错误根源与认知偏差。

人类在直觉物理理解上的能力仍远超机器。为缩小这一差距,本文主张转向类脑计算原理。提出时空关系神经网络(SRNN),在统一神经表示中同时编码物体属性、关系与时间线,通过专用的'是什么'和'怎么运作'路径,以赫布式'一起激活,一起连接'机制进行计算。该表示直接生成结构化语言描述,实现感知与语言在共享神经基质中的融合。在CLEVRER基准上,SRNN取得具有竞争力的表现,验证其从视觉流中表征关键时空关系的能力。认知消融分析揭示了基准数据集的偏差,为更全面评估指明方向。此外,SRNN的白盒特性使其能精确定位错误根源。本工作证明了将生物智能核心原则转化为工程系统,在受限环境中实现直觉物理理解的可行性。

原文摘要 · Abstract (English)

Human prowess in intuitive physics remains unmatched by machines. To bridge this gap, we argue for a fundamental shift towards brain-inspired computational principles. This paper introduces the Spatiotemporal Relational Neural Network (SRNN), a model that establishes a unified neural representation for object attributes, relations, and timeline, with computations governed by a Hebbian ``Fire Together, Wire Together'' mechanism across dedicated \textit{What} and \textit{How} pathways. This unified representation is directly used to generate structured linguistic descriptions of the visual scene, bridging perception and language within a shared neural substrate. On the CLEVRER benchmark, SRNN achieves competitive performance, thereby confirming its capability to represent essential spatiotemporal relations from the visual stream. Cognitive ablation analysis further reveals a benchmark bias, outlining a path for a more holistic evaluation. Finally, the white-box nature of SRNN enables precise pinpointing of error root causes. Our work provides a proof-of-concept that confirms the viability of translating key principles of biological intelligence into engineered systems for intuitive physics understanding in constrained environments.

直觉物理神经网络时空建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。