arXiv:2412.15588cs.CL2024-12被引 18

用大模型生成符号表示,让神经符号推理更灵活可微。

NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization

  • 用语言依赖结构增强自然语言与符号表示的对齐
  • 在ReaSCAN和CLEVR-CoGenT上达到顶尖性能
  • 适合需要组合泛化的视觉语言推理任务

组合泛化对人工智能体解决复杂视觉-语言推理任务至关重要。神经符号方法虽在捕捉组合结构方面展现潜力,但仍面临三大挑战:(a) 符号表示依赖预定义谓词,适应性差;(b) 从原始数据中提取谓词困难;(c) 使用非可微操作组合基本概念。为此,我们提出NeSyCoCo,一种利用大语言模型(LLMs)生成符号表示并映射为可微神经计算的神经符号框架。该框架引入三项创新:(a) 通过依赖结构增强自然语言输入与符号表示的对齐;(b) 使用分布式词表示将多样、语言驱动的逻辑谓词链接至神经模块;(c) 采用归一化谓词得分的软组合实现符号与可微推理的一致性。该框架在ReaSCAN和CLEVR-CoGenT组合泛化基准上取得当前最优结果,并在CLEVR-SYN新概念测试中表现稳健。

原文摘要 · Abstract (English)

Compositional generalization is crucial for artificial intelligence agents to solve complex vision-language reasoning tasks. Neuro-symbolic approaches have demonstrated promise in capturing compositional structures, but they face critical challenges: (a) reliance on predefined predicates for symbolic representations that limit adaptability, (b) difficulty in extracting predicates from raw data, and (c) using non-differentiable operations for combining primitive concepts. To address these issues, we propose NeSyCoCo, a neuro-symbolic framework that leverages large language models (LLMs) to generate symbolic representations and map them to differentiable neural computations. NeSyCoCo introduces three innovations: (a) augmenting natural language inputs with dependency structures to enhance the alignment with symbolic representations, (b) employing distributed word representations to link diverse, linguistically motivated logical predicates to neural modules, and (c) using the soft composition of normalized predicate scores to align symbolic and differentiable reasoning. Our framework achieves state-of-the-art results on the ReaSCAN and CLEVR-CoGenT compositional generalization benchmarks and demonstrates robust performance with novel concepts in the CLEVR-SYN benchmark.

神经符号组合泛化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。