用结构化元数据引导大模型精准检索，无需复杂系统即可实现100%准确率。
Self-Describing Structured Data with Dual-Layer Guidance: A Lightweight Alternative to RAG for Precision Retrieval in Large-Scale LLM Knowledge Navigation
- 在文件开头嵌入导航元数据，利用模型对首段内容的偏好提升定位效率。
- 结合文件内元数据与系统提示规则，119类技能库中路由准确率达100%。
- 适用于无向量库的半结构化文档，特别适合人工定义边界的知识库。
大型语言模型在处理长上下文时存在位置偏差：中间信息受关注度远低于边界内容，称为‘迷失于中间’效应（Liu et al., 2024）。这限制了将大型结构化知识库直接嵌入上下文的应用。检索增强生成（RAG）虽提升可扩展性，但引入显著基础设施开销，且不适用于语义边界由人工定义而非统计学习的知识库。我们提出自描述结构化检索（SDSR），一种轻量级框架：在结构化数据文件的首要位置嵌入人工编写的导航元数据，从而利用而非对抗模型的优先偏倚。进一步提出双层引导策略，结合文件内元数据与系统提示中的显式路由规则。通过四轮基准测试验证，基于190项技能、从36类扩展至119类并注入对抗性干扰的库，测试四种条件：(A) 无引导，(B) 仅文件摘要，(C) 仅提示提示，(D) 两者结合。版本D在119类下实现100%主路径路由准确率（20/20），而无引导基线仅为65%。我们发现根本不对称性：主路径路由可通过显式规则解决，而跨类别次级路由需在数据结构中明确编码架构意图。我们还将SDSR扩展至半结构化语料，展示通过交叉引用编码可在无需向量数据库的领域实现有效操作。
原文摘要 · Abstract (English)
Large Language Models (LLMs) exhibit a well-documented positional bias when processing long input contexts: information in the middle of a context window receives substantially less attention than content at the boundaries, a phenomenon termed the Lost-in-the-Middle effect (Liu et al., 2024). This limits knowledge-retrieval applications that embed large structured knowledge bases directly in the LLM context. Retrieval-Augmented Generation (RAG) addresses scalability by retrieving only relevant fragments, but introduces substantial infrastructure overhead and is ill-suited to libraries whose semantic boundaries are human-defined rather than statistically learned. We propose Self-Describing Structured Retrieval (SDSR), a lightweight framework in which structured data files embed human-authored navigational metadata at the file's primacy position, thereby exploiting rather than fighting the LLM's primacy bias. We further propose a Dual-Layer Guidance strategy combining in-file metadata with explicit routing rules in the system prompt. We validate SDSR through a four-round benchmark using a 190-skill library expanded from 36 to 119 categories via adversarial distractor injection. Four conditions are tested: (A) no guidance, (B) in-file summary only, (C) prompt hint only, (D) both combined. Version D achieves 100% primary routing accuracy (20/20) at 119 categories versus 65% for the no-guidance baseline. We identify a fundamental asymmetry: primary routing is solvable by explicit rules, while secondary cross-category routing requires architectural intent explicitly encoded in the data structure. We further extend SDSR to semi-structured corpora, showing how cross-reference encoding enables operation without vector databases in domains with recoverable document structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。