arXiv:2606.05644cs.AI2026-06被引 1

让大模型在冲突时更听提示,不偏信记忆。

FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG

论文配图:FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG
图 1 · 摘自论文原文
  • 用三个内部信号检测每步的证据冲突程度,精准干预
  • 在18组实验中全部提升上下文忠实度,最高+13分
  • 适合需要高可信生成的场景,如医疗、法律问答

当检索到的证据与模型参数记忆冲突时,语言模型常忽略上下文而依赖记忆——这破坏了检索增强的核心目标。对比解码通过强化上下文输出来抑制参数偏差,但现有方法隐含假设偏差在所有词元上均匀分布,导致全局对比权重过度惩罚安全词元,却未能充分修正真正冲突的词元。我们发现词元级冲突集中:检索-记忆矛盾集中在少数关键生成步骤。这将对比解码从‘如何施加对比’转变为‘在何处施加对比’。提出FIDES(Faithful Inference via Deep Evidence Signals),一种无需训练的解码器,通过探测输出表面、隐藏表示和预测轨迹三个层次的内部信号,融合生成每步干预强度。在三个基准和六种主干模型(四个7B/8B模型及两个扩展至70B的模型)上,FIDES在全部18个设置中均取得最佳上下文忠实度,相比最强无训练基线提升+3至+13点。在70B规模下,忠实度达92-94%,F1值飙升至62-63%,表明词元级选择性释放了被粗粒度规则压制的生成能力。

原文摘要 · Abstract (English)

When retrieved evidence contradicts parametric memory, language models frequently ignore context and default to memorized priors -- a failure that undermines the core purpose of retrieval augmentation. Contrastive decoding amplifies the context-conditioned output to suppress parametric bias, but existing methods rest on an implicit assumption that this bias is uniform across tokens. A single global contrastive weight over-penalizes safe tokens while leaving genuinely conflicted ones insufficiently corrected. We identify token-level conflict concentration: retrieval-memory tension is sharply heterogeneous, concentrated on a small fraction of answer-critical decoding steps. This reframes contrastive decoding from how much contrast to apply to where to apply it. We propose FIDES (Faithful Inference via Deep Evidence Signals), a training-free decoder that reads three internal signals probing retrieval-memory conflict at complementary depths -- output surface, hidden representations, and prediction trajectory -- and fuses them to govern intervention strength at each decoding step. Across three benchmarks and six backbones -- four primary 7B/8B models and two scaling backbones up to 70B -- FIDES achieves the best context fidelity in all 18 settings, outperforming the strongest training-free baseline by +3 to +13 points. On the 70B scale, fidelity reaches 92-94% while F1 surges to 62-63%, demonstrating that token-level selectivity unlocks generation capability that coarse contrastive rules suppress.

RAG生成可信解码优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。