解决复杂文档理解中视觉注意力失焦问题,提升生成准确率与效率。
DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding

- 分离视觉与语义处理,通过语义锚点避免注意力耗散。
- 区域感知裁剪减少40.8%提示词量,准确率提升12.5个百分点。
- 适用于金融图表、文档问答等高密度复杂布局场景。
复杂文档理解中的多模态检索增强生成(RAG)面临精度与效率的双重挑战,尤其在图结构RAG中尤为突出。处理结构稀疏但视觉密集的布局(如从财务图表中提取微小数据标记)常导致计算开销过大,并引发灾难性幻觉。现有多模态图RAG流程依赖视觉语言模型(VLMs)在高密度布局中解析稀疏语义,但本研究揭示,强制VLM同时定位视觉证据、解释语义并提取关系会触发‘视觉注意力陷阱’,导致严重语义损失;而全页处理则带来巨大计算负担。控制实验表明该失败源于边界效应而非内容特性,且语义锚定可缓解此问题。为此,我们提出DeCoRAG,将知识处理从耦合的视觉-语义推理转向‘认知解耦’。其图构建阶段建立宏观语义锚点以中和注意力陷阱,并驱动区域感知剪枝与裁剪(RAP-Crop)机制,将推理空间从嘈杂背景转移至纯净、意图导向的语义簇。最终生成的图支持混合检索与答案生成。在多个复杂文档基准上,DeCoRAG相较最强基线提升语义通过率达12.5个百分点,且在DocVQA上具有良好泛化能力;RAP-Crop使离线图构建提示词减少40.8%,且不牺牲端到端准确率。
原文摘要 · Abstract (English)
Advancing multimodal retrieval-augmented generation (RAG) for complex document understanding presents a formidable dual dilemma of accuracy and efficiency, particularly in graph RAG. Processing structurally sparse yet visually dense layouts, such as extracting a tiny data marker from a financial chart, often incurs computationally prohibitive token overhead while still triggering catastrophic hallucination. However, multimodal Graph RAG pipelines rely on graph-construction stages that assume Vision-Language Models (VLMs) can resolve sparse semantics within high-density layouts. We challenge this assumption, revealing that forcing VLMs to localize visual evidence, interpret semantics, and extract relations triggers a "Visual Attention Sink," a mechanism driving catastrophic semantic loss, while full-page processing incurs massive computational overhead. Controlled interventions verify that this failure is boundary-driven rather than content-specific and that semantic anchoring mitigates it. To fundamentally correct this flawed paradigm, we introduce DeCoRAG, a multimodal Graph RAG pipeline that shifts knowledge processing from coupled visual-semantic reasoning to "Cognitive Decoupling." Rather than passively processing raw pixels, its graph-construction stage establishes a macroscopic Semantic Anchor to neutralize the attention sink. This anchor subsequently drives our Region-Aware Pruning and Cropping (RAP-Crop) mechanism, shifting the reasoning space from dense, noisy backgrounds to purified, intent-driven semantic clusters. The resulting graph supports hybrid retrieval and answer generation. Across complex document benchmarks, DeCoRAG improves the semantic pass rate by up to 12.5 percentage points over the strongest baseline and generalizes to DocVQA. RAP-Crop reduces offline graph-construction prompt tokens by 40.8% without sacrificing end-to-end accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。