优化企业RAG效果，关键在知识内容设计而非模型或搜索方法
Optimizing and Evaluating Enterprise Retrieval-Augmented Generation (RAG): A Content Design Perspective
- 通过调整知识库内容结构提升RAG表现，无需更换模型或检索策略
- 简单的内容设计改进可使RAG成功率显著提升，实测效果优于复杂算法
- 采用人工主导的评估方式，更适应企业中新颖用户问题的评测需求
检索增强生成（RAG）是利用大语言模型构建客户支持与问答系统的重要技术。本文分享了我们在构建和维护基于产品文档的企业级RAG系统方面的实践经验。我们的经验表明，常见的RAG文献模式并不完全适用于实际场景。本研究聚焦于模块化、与模型无关的解决方案策略。例如，过去几年中，我们尝试了多种搜索方法、不同LLM及多个知识库集合，发现仅通过调整知识库内容的创建方式，就能对RAG系统的成功率产生巨大影响。此外，本文还探讨了结果监控与评估机制。传统基准评测方法在应对新型用户问题时效果有限，因此我们采用了灵活的‘以人工为主’的评估方式，以更真实地衡量系统表现。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) is a popular technique for using large language models (LLMs) to build customer-support, question-answering solutions. In this paper, we share our team's practical experience building and maintaining enterprise-scale RAG solutions that answer users' questions about our software based on product documentation. Our experience has not always matched the most common patterns in the RAG literature. This paper focuses on solution strategies that are modular and model-agnostic. For example, our experience over the past few years - using different search methods and LLMs, and many knowledge base collections - has been that simple changes to the way we create knowledge base content can have a huge impact on our RAG solutions' success. In this paper, we also discuss how we monitor and evaluate results. Common RAG benchmark evaluation techniques have not been useful for evaluating responses to novel user questions, so we have found a flexible, "human in the lead" approach is required.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。