优化检索内容呈现方式,提升长文本推理稳定性
Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation
- 通过标准化上下文表示,自适应统一检索内容格式
- 在多种基准上显著提升对顺序变化的鲁棒性
- 适合需要稳定长文本推理的实用型RAG系统
检索增强生成(RAG)已成为扩展大语言模型推理与知识能力的关键方法。现有研究多关注检索质量与提示策略,却忽视了检索文档的呈现方式(即上下文格式)的影响。我们发现,即使语义内容相同,分隔符选择、键值提取中的结构标记等看似表面的格式差异,也会导致准确率与稳定性显著变化。为此,我们设计受控实验,系统考察上下文密度、分隔符样式与位置布局的影响,揭示性能差异的根本原因。基于此,提出轻量级的上下文归一化(Contextual Normalization)策略,在生成前自适应标准化上下文表示。在多种真实与受控基准上的广泛实验表明,该方法能持续提升对顺序变化的鲁棒性,并强化长上下文利用能力。结果表明,可靠的RAG不仅依赖于正确的内容检索,更取决于内容的呈现方式,为长文本推理提供了新的实证依据与实用技术。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has become an essential approach for extending the reasoning and knowledge capacity of large language models (LLMs). While prior research has primarily focused on retrieval quality and prompting strategies, the influence of how the retrieved documents are framed, i.e., context format, remains underexplored. We show that seemingly superficial choices, such as delimiters or structural markers in key-value extraction, can induce substantial shifts in accuracy and stability, even when semantic content is identical. To systematically investigate this effect, we design controlled experiments that vary context density, delimiter styles, and positional placement, revealing the underlying factors that govern performance differences. Building on these insights, we introduce Contextual Normalization, a lightweight strategy that adaptively standardizes context representations before generation. Extensive experiments on both controlled and real-world RAG benchmarks across diverse settings demonstrate that the proposed strategy consistently improves robustness to order variation and strengthens long-context utilization. These findings underscore that reliable RAG depends not only on retrieving the right content, but also on how that content is presented, offering both new empirical evidence and a practical technique for better long-context reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。