用分层摘要线性化树结构数据，提升RAG处理复杂代码的效率。
Is Implicit Knowledge Enough for LLMs? A RAG Approach for Tree-based Structures
- 自底向上生成每层的隐式聚合摘要，将树形结构转为可检索知识
- 相比原始代码，减少68%以上检索文档量，响应质量相当
- 适合处理代码库等层级化数据，提升RAG系统可扩展性
大语言模型（LLMs）擅长基于上下文生成回复，但处理如代码仓库这类层次化结构数据时，传统方法效果有限。本文提出一种自底向上的线性化方法，通过在每个层级生成隐式聚合摘要，将树状结构知识转化为可存储的知识库，并直接用于检索增强生成（RAG）。实验对比了该方法与直接对原始非结构化代码使用RAG的效果。结果表明，尽管生成响应质量相当，但本方法使检索器中的文档数量减少超过68%，显著提升了效率。这表明，利用隐式线性化知识是处理复杂层级数据的一种高效且可扩展策略。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are adept at generating responses based on information within their context. While this ability is useful for interacting with structured data like code files, another popular method, Retrieval-Augmented Generation (RAG), retrieves relevant documents to augment the model's in-context learning. However, it is not well-explored how to best represent this retrieved knowledge for generating responses on structured data, particularly hierarchical structures like trees. In this work, we propose a novel bottom-up method to linearize knowledge from tree-like structures (like a GitHub repository) by generating implicit, aggregated summaries at each hierarchical level. This approach enables the knowledge to be stored in a knowledge base and used directly with RAG. We then compare our method to using RAG on raw, unstructured code, evaluating the accuracy and quality of the generated responses. Our results show that while response quality is comparable across both methods, our approach generates over 68% fewer documents in the retriever, a significant gain in efficiency. This finding suggests that leveraging implicit, linearized knowledge may be a highly effective and scalable strategy for handling complex, hierarchical data structures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。