DoTA-RAG通过动态聚合思想提升大规模网页知识检索的准确率与速度。
DoTA-RAG: Dynamic of Thought Aggregation RAG
- 分三阶段处理：查询重写、动态路由到子索引、多阶段检索排序
- 正确率从0.752提升至1.478，Live Challenge Day达0.929
- 适用于需快速访问海量动态知识的场景，如实时问答系统
本文提出DoTA-RAG（动态思想聚合检索增强生成），一种面向高吞吐、大规模网络知识索引的检索增强生成系统。传统RAG管道在海量多样数据下常面临高延迟与低精度问题。DoTA-RAG采用三阶段流程：查询重写、动态路由至专用子索引、多阶段检索与排序。通过评估并选择更优嵌入模型，对大型FineWeb-10BT语料库重新嵌入。此外，我们基于DataMorgana框架构建了一个涵盖500个问题的多样化Q&A数据集，覆盖广泛WebOrganizer主题与格式。实验显示，DoTA-RAG将答案正确率从基线的0.752提升至1.478，同时保持低延迟，在Live Challenge Day中达到0.929的正确率。结果表明其在需要快速可靠访问大规模动态知识源的领域具有实际部署潜力。
原文摘要 · Abstract (English)
In this paper, we introduce DoTA-RAG (Dynamic-of-Thought Aggregation RAG), a retrieval-augmented generation system optimized for high-throughput, large-scale web knowledge indexes. Traditional RAG pipelines often suffer from high latency and limited accuracy over massive, diverse datasets. DoTA-RAG addresses these challenges with a three-stage pipeline: query rewriting, dynamic routing to specialized sub-indexes, and multi-stage retrieval and ranking. We further enhance retrieval by evaluating and selecting a superior embedding model, re-embedding the large FineWeb-10BT corpus. Moreover, we create a diverse Q&A dataset of 500 questions generated via the DataMorgana setup across a broad range of WebOrganizer topics and formats. DoTA-RAG improves the answer correctness score from 0.752 (baseline, using LiveRAG pre-built vector store) to 1.478 while maintaining low latency, and it achieves a 0.929 correctness score on the Live Challenge Day. These results highlight DoTA-RAG's potential for practical deployment in domains requiring fast, reliable access to large and evolving knowledge sources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。