arXiv:2505.02418cs.IRcs.HC2025-05被引 5

让人类与大模型协作优化文档检索,提升问答准确率。

SymbioticRAG: Enhancing Document Intelligence Through Human-LLM Symbiotic Collaboration

  • 人类与大模型双向互动,动态优化检索内容。
  • 在三类场景中,相关性与用户满意度显著提升。
  • 适合需要精准文档理解的研究者与工程师。

我们提出SymbioticRAG,一种重新构想检索增强生成(RAG)系统的框架,建立人与机器之间的双向学习关系。该方法解决当前RAG系统中相关性判断依赖人工、用户提问能力不足两大问题。方案分为两级:一级通过交互式文档探索实现人工内容精炼,包含布局检测、OCR、表格公式图示提取的全流程处理管道,支持多种检索策略的可扩展检索模块,以及记录用户行为的交互界面;二级基于用户操作日志,利用大模型总结用户意图以构建个性化检索模型。为保障数据质量,设计了人工审核接口,同时推动专业提取任务研究。在文献综述、地质勘探和教育三个场景中的评估显示,相比传统RAG,检索相关性与用户满意度均有显著提升。为促进后续研究,系统将向学术界开源。

原文摘要 · Abstract (English)

We present \textbf{SymbioticRAG}, a novel framework that fundamentally reimagines Retrieval-Augmented Generation~(RAG) systems by establishing a bidirectional learning relationship between humans and machines. Our approach addresses two critical challenges in current RAG systems: the inherently human-centered nature of relevance determination and users' progression from "unconscious incompetence" in query formulation. SymbioticRAG introduces a two-tier solution where Level 1 enables direct human curation of retrieved content through interactive source document exploration, while Level 2 aims to build personalized retrieval models based on captured user interactions. We implement Level 1 through three key components: (1)~a comprehensive document processing pipeline with specialized models for layout detection, OCR, and extraction of tables, formulas, and figures; (2)~an extensible retriever module supporting multiple retrieval strategies; and (3)~an interactive interface that facilitates both user engagement and interaction data logging. We experiment Level 2 implementation via a retriever strategy incorporated LLM summarized user intention from user interaction logs. To maintain high-quality data preparation, we develop a human-on-the-loop validation interface that improves pipeline output while advancing research in specialized extraction tasks. Evaluation across three scenarios (literature review, geological exploration, and education) demonstrates significant improvements in retrieval relevance and user satisfaction compared to traditional RAG approaches. To facilitate broader research and further advancement of SymbioticRAG Level 2 implementation, we will make our system openly accessible to the research community.

RAG人机协作文档智能交互式检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。