用轻量动态检索框架,让旧系统里的历史数据更好找。
SPAR: Session-based Pipeline for Adaptive Retrieval on Legacy File Systems
- 先建语义元数据索引,再按会话生成向量库。
- 相比传统方法,计算开销更低,检索更准更可控。
- 适合想低成本用大模型分析旧文件系统的公司。
从历史数据中提取价值对企业的决策至关重要。然而,大量信息仍滞留在缺乏结构化组织和语义索引的大型旧式文件系统中,导致检索与分析效率低且易出错。本文提出SPAR(基于会话的自适应检索流水线),一个将大语言模型(LLMs)融入检索增强生成(RAG)架构的新型概念框架,专为旧企业环境设计。与需要构建和维护全规模向量数据库的传统RAG管道不同,SPAR采用轻量级两阶段流程:首先创建语义元数据索引,随后按需动态生成会话特定的向量数据库。该设计降低计算开销,同时提升检索的透明度、可控性与相关性。我们提供了理论复杂度分析,证明其在计算上的优势。通过在一个包含大规模生物医学文献的合成企业级文件系统上应用SPAR,验证了其在检索有效性及下游模型准确率上的提升。最后,讨论了设计权衡,并指出了在多样企业环境中部署的开放挑战。
原文摘要 · Abstract (English)
The ability to extract value from historical data is essential for enterprise decision-making. However, much of this information remains inaccessible within large legacy file systems that lack structured organization and semantic indexing, making retrieval and analysis inefficient and error-prone. We introduce SPAR (Session-based Pipeline for Adaptive Retrieval), a conceptual framework that integrates Large Language Models (LLMs) into a Retrieval-Augmented Generation (RAG) architecture specifically designed for legacy enterprise environments. Unlike conventional RAG pipelines, which require costly construction and maintenance of full-scale vector databases that mirror the entire file system, SPAR employs a lightweight two-stage process: a semantic Metadata Index is first created, after which session-specific vector databases are dynamically generated on demand. This design reduces computational overhead while improving transparency, controllability, and relevance in retrieval. We provide a theoretical complexity analysis comparing SPAR with standard LLM-based RAG pipelines, demonstrating its computational advantages. To validate the framework, we apply SPAR to a synthesized enterprise-scale file system containing a large corpus of biomedical literature, showing improvements in both retrieval effectiveness and downstream model accuracy. Finally, we discuss design trade-offs and outline open challenges for deploying SPAR across diverse enterprise settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。