arXiv:2606.01542cs.DCcs.AI2026-06

解决文档分块检索重复问题,提升生成质量

Self-Conditioned Positional HNSW for Overlap-Aware Retrieval in Chunked-Document RAG Systems: Method and Industrial Evidence-Quality Audit

  • 在嵌入向量中加入位置编码,用双阶段查询优化结果分布
  • 实测770条评审中574条获3分以上,95%清晰截图可正确识别
  • 适合需要高质量证据生成的工业级RAG系统

文档分块检索是检索增强生成(RAG)系统的核心组件。文档被分割为重叠块,经嵌入后通过近似最近邻搜索(如HNSW图)索引。重叠虽能提升边界覆盖,但导致前k项召回常返回相邻重复块,浪费提示预算。本文提出自条件位置HNSW(SCP-HNSW),通过在块嵌入中添加低维位置编码,并采用两阶段查询过程估计并应用查询相关的文档位置先验,实现无需修改原图结构的轻量改进。最终通过可审计的最小索引间距选择器构造上下文。同时整合工业级评审数据:770条文本证据审核中318条完全标注,70个案例的OCR审核含350个评分。文本审核显示770条中574条获3/5分,仅39条在1-2分区间,叙事细节远多于结构化问题标记;OCR审核显示清晰聊天截图通过率达95%,手写/模糊图像仅45%,评分一致性中到强。结果支持具备重叠感知与审计友好特性的RAG检索,并指出未来因果性能验证所需的关键控制性消融实验。

原文摘要 · Abstract (English)

Chunked-document retrieval is a common component of retrieval-augmented generation (RAG) systems. Documents are split into overlapping chunks, embedded, and indexed with approximate nearest-neighbor search such as hierarchical navigable small world graphs (HNSW). Overlap improves boundary coverage but induces a practical failure mode: top-k retrieval often returns near-adjacent chunks that repeat evidence and waste prompt budget. We propose Self-Conditioned Positional HNSW (SCP-HNSW), a lightweight modification that appends a low-dimensional positional code to chunk embeddings and uses a two-pass query procedure to estimate and apply a query-specific document-position prior. SCP-HNSW leaves HNSW graph construction and traversal unchanged while adding an auditable minimum-index-gap selector for final context construction. We also integrate industrial review artifacts for generated evidence quality: a 770-review text-evidence audit with 318 fully labeled reviews and a 70-case OCR audit with 350 ratings. The text audit shows that 574 of 770 projected reviews are rated 3/5, only 39 fall in the 1-2 range, and narrative reviewer detail appears much more often than structured issue flags. The OCR audit shows slice-level pass rates from 95% for clean chat screenshots to 45% for handwritten/blurry captures, with moderate to strong agreement. These results motivate overlap-aware, audit-friendly RAG retrieval and identify the remaining controlled retrieval ablations needed for causal performance claims.

RAG检索增强文档分块审计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。