构建私有邮件数据集，推动个性化RAG技术评估与优化
EnronQA: Towards Personalized RAG over Private Documents
- 基于150个真实邮箱的超十万封邮件构建评测基准
- 包含52万+问答对，支持个性化检索与记忆权衡研究
- 适合企业级私有知识增强场景的RAG系统开发者
检索增强生成（RAG）因能在推理时引入本地上下文，且避免微调带来的成本与数据泄露风险，已成为企业级大模型应用的核心技术。然而现有RAG评测基准多基于维基百科等公开数据，缺乏个人化上下文。为此，我们发布EnronQA，一个包含103,638封邮件、528,304个问答对、覆盖150位用户邮箱的数据集，支持在真实私有数据上对RAG管道进行更精准评估，并探索个性化检索设置下的记忆与检索权衡问题。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) has become one of the most popular methods for bringing knowledge-intensive context to large language models (LLM) because of its ability to bring local context at inference time without the cost or data leakage risks associated with fine-tuning. A clear separation of private information from the LLM training has made RAG the basis for many enterprise LLM workloads as it allows the company to augment LLM's understanding using customers' private documents. Despite its popularity for private documents in enterprise deployments, current RAG benchmarks for validating and optimizing RAG pipelines draw their corpora from public data such as Wikipedia or generic web pages and offer little to no personal context. Seeking to empower more personal and private RAG we release the EnronQA benchmark, a dataset of 103,638 emails with 528,304 question-answer pairs across 150 different user inboxes. EnronQA enables better benchmarking of RAG pipelines over private data and allows for experimentation on the introduction of personalized retrieval settings over realistic data. Finally, we use EnronQA to explore the tradeoff in memorization and retrieval when reasoning over private documents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。