用滚动关键词实现单次调用高精度RAG，自动保持文档上下文。
MDKeyChunker: Single-Call LLM Enrichment with Rolling Keys and Key-Based Restructuring for High-Accuracy RAG
- 按标题、代码块等结构单元分块，保留语义完整
- 单次LLM调用提取7项元数据，召回率最高达1.000
- 滚动关键词自动合并相关段落，适合长文档检索
RAG管道通常依赖固定大小的分块策略，忽略文档结构，破坏语义单元，并需对每个分块多次调用LLM提取元数据。我们提出MDKeyChunker，一个针对Markdown文档的三阶段流程：(1) 基于结构感知的分块，将标题、代码块、表格和列表视为原子单位；(2) 通过一次LLM调用，提取标题、摘要、关键词、类型化实体、假设问题和语义键，同时传播滚动键字典以维持文档级上下文；(3) 通过二叉打包重构分块，合并共享相同语义键的片段，提升检索相关性。单次调用完成全部七项元数据提取，无需逐字段调用。滚动键传播以LLM原生语义匹配替代人工调参评分。在包含18个文档的Markdown语料库上进行30个查询的实证评估显示，配置D（BM25+结构分块）达到Recall@5=1.000，MRR=0.911；配置C（密集检索全流水线）实现Recall@5=0.867。MDKeyChunker基于Python实现，仅需四个依赖，支持任意OpenAI兼容接口。
原文摘要 · Abstract (English)
RAG pipelines typically rely on fixed-size chunking, which ignores document structure, fragments semantic units across boundaries, and requires multiple LLM calls per chunk for metadata extraction. We present MDKeyChunker, a three-stage pipeline for Markdown documents that (1) performs structure-aware chunking treating headers, code blocks, tables, and lists as atomic units; (2) enriches each chunk via a single LLM call extracting title, summary, keywords, typed entities, hypothetical questions, and a semantic key, while propagating a rolling key dictionary to maintain document-level context; and (3) restructures chunks by merging those sharing the same semantic key via bin-packing, co-locating related content for retrieval. The single-call design extracts all seven metadata fields in one LLM invocation, eliminating the need for separate per-field extraction passes. Rolling key propagation replaces hand-tuned scoring with LLM-native semantic matching. An empirical evaluation on 30 queries over an 18-document Markdown corpus shows Config D (BM25 over structural chunks) achieves Recall@5=1.000 and MRR=0.911, while dense retrieval over the full pipeline (Config C) reaches Recall@5=0.867. MDKeyChunker is implemented in Python with four dependencies and supports any OpenAI-compatible endpoint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。