构建企业内部知识检索增强生成评测基准，覆盖九大源类型500问。
EnterpriseRAG-Bench: A RAG Benchmark for Company Internal Knowledge
- 基于九类企业数据源生成50万文档，保持跨文档关联性与真实噪声。
- 包含500个问题，覆盖单文档查询到多文档推理、冲突解决等能力测试。
- 提供可定制的生成框架与公开评测平台，适合企业AI Agent研发者使用。
检索增强生成（RAG）已成为将大模型与训练外信息对齐的标准方法。现有数据集多聚焦于网络或公开数据，缺乏能真实反映企业内部知识特性的评测基准。为此，我们发布了一个合成的企业语料库、其生成框架及排行榜。EnterpriseRAG-Bench 包含约50万份文档，涵盖九种企业来源（Slack、Gmail、Linear、Google Drive、HubSpot、Fireflies、GitHub、Jira、Confluence），以及500个问题，横跨十类任务，测试不同检索与推理能力。语料具备跨文档一致性（基于共享项目、人员和计划），并加入真实噪声，如错放文件、近似重复、信息冲突。问题范围从简单单文档查找，到多文档推理、受限检索、冲突解决，以及识别信息缺失。生成框架支持团队按自身行业、规模和数据源组合定制版本。数据集、代码、评估工具链及排行榜已开源：https://github.com/onyx-dot-app/EnterpriseRAG-Bench。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has become the standard approach for grounding large language models in information that was not available during training. While existing datasets and benchmarks focus on web or other public sources, there is still no widely adopted dataset that realistically reflects the nature of company-internal knowledge. Meanwhile, startups, enterprises, and researchers are increasingly developing AI Agents designed to operate over exactly this kind of proprietary data. To close this gap, we release a synthetic enterprise corpus, its generation framework, and a leaderboard. We present EnterpriseRAG-Bench, a dataset consisting of approximately 500,000 documents spanning nine enterprise source types (Slack, Gmail, Linear, Google Drive, HubSpot, Fireflies, GitHub, Jira, and Confluence) and 500 questions across ten categories that test distinct retrieval and reasoning capabilities. The corpus is generated with cross-document coherence (grounded in shared projects, people, and initiatives) and augmented with realistic noise such as misfiled documents, near-duplicates, and conflicting information. The question set ranges from simple single-document lookups to multi-document reasoning, constrained retrieval, conflict resolution, and recognizing when information is absent. The generation framework lets teams generate variants tailored to their own industry, scale, and source mix. The dataset, code, evaluation harness, and leaderboard are available at https://github.com/onyx-dot-app/EnterpriseRAG-Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。