arXiv:2603.15280cs.AI2026-03KDD被引 4

用符号规则增强神经记忆,让多模态智能体更会逻辑推理。

Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory

论文配图:Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory
图 1 · 摘自论文原文
  • 分三层记忆结构:情景、语义、逻辑规则,融合神经与符号表示。
  • 在真实场景测试中,推理准确率平均提升4.35%,复杂问题最高提12.5%。
  • 适合需要长期规划和严谨决策的智能系统研发者参考。

大语言模型推动了在开放世界多模态环境中运行的智能体发展。为支持长期推理,这些智能体通常配备外部记忆系统。然而,现有大多数多模态记忆依赖神经表征和向量检索,虽擅长直觉推理,却难以支撑现实决策所需的分析性、演绎性推理。为此,我们提出NS-Mem,一种长期神经符号记忆框架,通过整合神经记忆与显式符号结构及规则,提升多模态智能体推理能力。NS-Mem包含三个核心组件:(1) 三层记忆架构——情景层、语义层与逻辑规则层;(2) 由SK-Gen实现的记忆构建与维护机制,可自动从累积的多模态经验中提炼结构化知识,并增量更新神经表示与符号规则;(3) 混合检索机制,结合基于相似性的搜索与确定性符号查询函数,支持结构化推理。在真实世界多模态推理基准上的实验表明,神经符号记忆相比纯神经记忆系统平均提升4.35%的推理准确率,复杂约束查询下最高提升12.5%,验证了NS-Mem的有效性。

原文摘要 · Abstract (English)

Recent advances in large language models have driven the emergence of intelligent agents operating in open-world, multimodal environments. To support long-term reasoning, such agents are typically equipped with external memory systems. However, most existing multimodal agent memories rely primarily on neural representations and vector-based retrieval, which are well-suited for inductive, intuitive reasoning but fundamentally limited in supporting analytical, deductive reasoning critical for real-world decision making. To address this limitation, we propose NS-Mem, a long-term neuro-symbolic memory framework designed to advance multimodal agent reasoning by integrating neural memory with explicit symbolic structures and rules. Specifically, NS-Mem is operated around three core components of a memory system: (1) a three-layer memory architecture that consists episodic layer, semantic layer and logic rule layer, (2) a memory construction and maintenance mechanism implemented by SK-Gen that automatically consolidates structured knowledge from accumulated multimodal experiences and incrementally updates both neural representations and symbolic rules, and (3) a hybrid memory retrieval mechanism that combines similarity-based search with deterministic symbolic query functions to support structured reasoning. Experiments on real-world multimodal reasoning benchmarks demonstrate that Neural-Symbolic Memory achieves an average 4.35% improvement in overall reasoning accuracy over pure neural memory systems, with gains of up to 12.5% on constrained reasoning queries, validating the effectiveness of NS-Mem.

多模态推理神经符号记忆系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。