用大模型提升技术文档检索精度,让开发人员更快找到所需信息。
Enhancing Technical Documents Retrieval for RAG
- 通过大模型扩展查询和摘要编码,增强技术文档语义表示。
- 在RAG-EDA和Rust-Docs-QA上,召回率与精确率均显著优于基线模型。
- 适合需要高效检索硬件/软件文档的工程师与RAG系统开发者。
本文提出Technical-Embeddings框架,旨在优化技术文档中的语义检索,适用于软硬件开发场景。针对复杂技术内容的理解与检索难题,该方法利用大语言模型(LLM)实现双重优化:首先通过生成扩展查询表示,更准确捕捉用户意图并丰富微调数据集;其次采用摘要提取技术编码关键上下文信息,提升文档表征质量。为进一步提升检索性能,我们基于软提示(soft prompting)对双编码器BERT模型进行微调,为查询与文档分别设置独立学习参数,以捕捉细粒度语义差异。在两个公开数据集RAG-EDA和Rust-Docs-QA上的实验表明,该方法在精度与召回率上均显著优于基线模型。研究结果证明,结合查询扩展与上下文摘要能有效提升技术领域信息获取与理解效率,推动检索增强生成(RAG)系统在工程与产品开发流程中的应用发展。
原文摘要 · Abstract (English)
In this paper, we introduce Technical-Embeddings, a novel framework designed to optimize semantic retrieval in technical documentation, with applications in both hardware and software development. Our approach addresses the challenges of understanding and retrieving complex technical content by leveraging the capabilities of Large Language Models (LLMs). First, we enhance user queries by generating expanded representations that better capture user intent and improve dataset diversity, thereby enriching the fine-tuning process for embedding models. Second, we apply summary extraction techniques to encode essential contextual information, refining the representation of technical documents. To further enhance retrieval performance, we fine-tune a bi-encoder BERT model using soft prompting, incorporating separate learning parameters for queries and document context to capture fine-grained semantic nuances. We evaluate our approach on two public datasets, RAG-EDA and Rust-Docs-QA, demonstrating that Technical-Embeddings significantly outperforms baseline models in both precision and recall. Our findings highlight the effectiveness of integrating query expansion and contextual summarization to enhance information access and comprehension in technical domains. This work advances the state of Retrieval-Augmented Generation (RAG) systems, offering new avenues for efficient and accurate technical document retrieval in engineering and product development workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。