优化癌症问答系统检索流程,提升大模型回答准确性。
Query pipeline optimization for cancer patient question answering systems

- 分三步优化检索流程:文档、段落和语义表示
- 在自建数据集上使Claude-3-haiku准确率提升5.24%
- 适合医疗AI研发者参考,尤其关注精准问答场景
检索增强生成(RAG)通过查询管道从外部信息中检索相关知识,缓解大语言模型的幻觉问题。然而,针对癌症患者问答(CPQA)系统的查询管道优化需兼顾多个组件与领域特性。本文提出一种三方面优化方法:(1)文档检索,对比分析NCBI资源并引入混合语义实时文档检索(HSRDR);(2)段落检索,识别稠密检索器与重排序器的最佳组合;(3)语义表征,提出语义增强重叠分割(SEOS)以提升上下文理解。在专为癌症相关咨询构建的自研数据集上,优化后的RAG使Claude-3-haiku的答对率相比思维链提示提高5.24%,比原始RAG设置提升约3%。本研究强调了领域定制化查询优化对释放RAG潜力的重要性,为构建更精准可靠的CPQA系统提供了坚实框架,推动了基于RAG的生物医学系统发展。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) mitigates hallucination in Large Language Models (LLMs) by using query pipelines to retrieve relevant external information and grounding responses in retrieved knowledge. However, query pipeline optimization for cancer patient question-answering (CPQA) systems requires separately optimizing multiple components with domain-specific considerations. We propose a novel three-aspect optimization approach for the RAG query pipeline in CPQA systems, utilizing public biomedical databases like PubMed and PubMed Central. Our optimization includes: (1) document retrieval, utilizing a comparative analysis of NCBI resources and introducing Hybrid Semantic Real-time Document Retrieval (HSRDR); (2) passage retrieval, identifying optimal pairings of dense retrievers and rerankers; and (3) semantic representation, introducing Semantic Enhanced Overlap Segmentation (SEOS) for improved contextual understanding. On a custom-developed dataset tailored for cancer-related inquiries, our optimized RAG approach improved the answer accuracy of Claude-3-haiku by 5.24% over chain-of-thought prompting and about 3% over a naive RAG setup. This study highlights the importance of domain-specific query optimization in realizing the full potential of RAG and provides a robust framework for building more accurate and reliable CPQA systems, advancing the development of RAG-based biomedical systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。