arXiv:2601.03748cs.IRcs.AI2026-01

用多维模型优化大规模RAG文档分区,让检索更可控、可解释。

Bridging OLAP and RAG: A Multidimensional Approach to the Design of Corpus Partitioning

  • 提出维度事实模型(DFM),结合语义聚类与时间/组织等维度分区
  • 支持层级路由和容错策略,提升检索在缺失元数据时的鲁棒性
  • 适合需要可解释、可治理的工业级RAG系统设计者

检索增强生成(RAG)系统正被部署于包含数百万文档和数千万文本片段的大规模文档集合。在工业级检索平台中,可扩展性通常通过水平分片、近似最近邻搜索、混合索引和优化元数据过滤来实现。尽管这些机制在效率上有效,但它们依赖自底向上的相似性驱动组织,缺乏对语料库分区的设计依据。本文主张,大规模RAG系统的设计可融合两种正交策略:语义聚类(优化嵌入空间中的局部性)与多维分区(基于时间、组织上下文等概念维度决定检索位置)。尽管这些维度已在现有系统中隐式存在,但使用方式随意且结构松散。为此,我们提出维度事实模型(DFM),作为指导RAG语料库多维分区的概念框架。该框架为事实、维度、层级与粒度提供了严谨的推理方式,自然支持层级路由与受控回退策略,确保在元数据不完整时检索依然稳健,将搜索从‘黑箱’相似匹配转变为可管理、确定性的流程。本文为立场论文,旨在弥合OLAP式多维建模与现代RAG架构之间的差距,推动可解释、可治理的大规模检索策略研究。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) systems are increasingly deployed on large-scale document collections, often comprising millions of documents and tens of millions of text chunks. In industrial-scale retrieval platforms, scalability is typically addressed through horizontal sharding and a combination of Approximate Nearest-Neighbor search, hybrid indexing, and optimized metadata filtering. Although effective from an efficiency perspective, these mechanisms rely on bottom-up, similarity-driven organization and lack a conceptual rationale for corpus partitioning. In this paper, we claim that the design of large-scale RAG systems may benefit from the combination of two orthogonal strategies: semantic clustering, which optimizes locality in embedding space, and multidimensional partitioning, which governs where retrieval should occur based on conceptual dimensions such as time and organizational context. Although such dimensions are already implicitly present in current systems, they are used in an ad hoc and poorly structured manner. We propose the Dimensional Fact Model (DFM) as a conceptual framework to guide the design of multidimensional partitions for RAG corpora. The DFM provides a principled way to reason about facts, dimensions, hierarchies, and granularity in retrieval-oriented settings. This framework naturally supports hierarchical routing and controlled fallback strategies, ensuring that retrieval remains robust even in the presence of incomplete metadata, while transforming the search process from a 'black-box' similarity matching into a governable and deterministic workflow. This work is intended as a position paper; its goal is to bridge the gap between OLAP-style multidimensional modeling and modern RAG architectures, and to stimulate further research on principled, explainable, and governable retrieval strategies at scale.

RAG多维分区可解释性语料库设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。