融合神经与符号检索,提升学术文献问答准确率
NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering
- 双模式协同:结合向量检索与结构化数据解析
- 多视角切分PDF,保留章节表格等布局信息
- 在3个真实论文问答数据集上超越现有方法
学术论文数量激增给研究人员高效获取关键信息带来挑战。尽管基于大模型的检索增强生成(RAG)在自动化问答中展现潜力,但以往工作常将神经与符号检索割裂,未能发挥二者互补优势。同时,传统单视角文本切分忽略了PDF的丰富结构,如章节和表格。本文提出NeuSym-RAG,一种混合神经符号检索框架,通过多视图切分与基于模式的解析,将半结构化PDF内容同时组织为关系型数据库与向量库,使大模型代理可迭代获取上下文直至生成答案。在三个基于完整PDF的问答数据集(包括自标注的AIRQA-REAL)上的实验表明,NeuSym-RAG稳定优于仅用向量的RAG及多种结构化基线,凸显其统一两种检索方式并利用多视图的能力。代码与数据已公开于https://github.com/X-LANCE/NeuSym-RAG。
原文摘要 · Abstract (English)
The increasing number of academic papers poses significant challenges for researchers to efficiently acquire key details. While retrieval augmented generation (RAG) shows great promise in large language model (LLM) based automated question answering, previous works often isolate neural and symbolic retrieval despite their complementary strengths. Moreover, conventional single-view chunking neglects the rich structure and layout of PDFs, e.g., sections and tables. In this work, we propose NeuSym-RAG, a hybrid neural symbolic retrieval framework which combines both paradigms in an interactive process. By leveraging multi-view chunking and schema-based parsing, NeuSym-RAG organizes semi-structured PDF content into both the relational database and vectorstore, enabling LLM agents to iteratively gather context until sufficient to generate answers. Experiments on three full PDF-based QA datasets, including a self-annotated one AIRQA-REAL, show that NeuSym-RAG stably defeats both the vector-based RAG and various structured baselines, highlighting its capacity to unify both retrieval schemes and utilize multiple views. Code and data are publicly available at https://github.com/X-LANCE/NeuSym-RAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。