用多智能体系统清理BRIGHT数据集，提升复杂推理的检索效果
BRIGHT+: Upgrading the BRIGHT Benchmark with MARCUS, a Multi-Agent RAG Clean-Up Suite
- 设计多智能体管道MARCUS，自动清除网页爬取带来的冗余和语义断裂
- 清理后数据集在多种检索器上检索准确率与多跳推理能力显著提升
- 适合关注高质量检索数据构建与复杂推理评估的研究者
检索增强生成（RAG）系统依赖结构清晰且语义连贯的语料库。BRIGHT 是一个近期具有影响力的基准，用于评估跨多个高推理领域、复杂的多跳检索任务。然而，其实际效果受限于常见的网络爬取问题——如内容重复和语义断层，这会降低检索准确率并影响下游推理性能。我们发现这些问题集中在七个来源于 StackExchange 的子领域，而其他领域（如编程与定理类内容）相对清洁。本文提出 MARCUS，一个基于大语言模型的多智能体清洗流水线，将 BRIGHT 系统性地重构为更高质量的语料库：BRIGHT-Plus。MARCUS 通过专用智能体进行结构噪声消除与语义分段，保留答案相关片段的同时增强上下文完整性。实验表明，使用 BRIGHT-Plus 后，在多种检索器上均实现检索准确率和多跳推理能力的持续显著提升。我们已开源 BRIGHT-Plus 语料库与 MARCUS 流水线，以支持未来对鲁棒、以推理为中心的检索研究。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems require corpora that are both structurally clean and semantically coherent. BRIGHT is a recent and influential benchmark designed to evaluate complex multi-hop retrieval across diverse, high-reasoning domains. However, its practical effectiveness is limited by common web-crawled artifacts - such as content redundancy and semantic discontinuity - that impair retrieval accuracy and downstream reasoning. Notably, we find that such issues are concentrated in seven StackExchange-derived subdomains, while other domains (e.g., Coding and Theorem-based content) remain relatively clean. In this study, we present MARCUS, a multi-agent pipeline that leverages large language models (LLMs) to systematically clean and re-chunk BRIGHT into a higher-quality corpus: BRIGHT-Plus. MARCUS applies dedicated agents for structural noise removal and semantic segmentation, preserving answer-bearing spans while improving contextual integrity. Experimental evaluations demonstrate that BRIGHT-Plus yields consistent and significant improvements in both retrieval accuracy and multi-hop reasoning across a diverse set of retrievers. We release both the BRIGHT-Plus corpus and the MARCUS pipeline to support future research on robust, reasoning-centric retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。