提出CoRect方法，解决大模型生成时内部知识压制外部证据的问题。
CoRect: Context-Aware Logit Contrast for Hidden State Rectification to Resolve Knowledge Conflicts
- 通过对比上下文与非上下文前向传播的输出，识别深层网络中参数偏差严重的层。
- 在QA和摘要任务上，相比强基线，忠实度提升且幻觉减少20%以上。
- 无需真实答案标签即可修复隐藏状态，适合需要高可信生成的场景。
检索增强生成（RAG）常因模型内参数化知识压制检索到的证据而产生不忠实输出。现有方法多依赖表面解码调整或需真实目标的权重编辑。通过逐层分析，我们发现深层网络中某些前馈层会用记忆中的先验覆盖上下文敏感表征，导致参数抑制现象。为此，我们提出CoRect（上下文感知的对数对比用于隐藏状态修正）。通过对比上下文与非上下文前向传播的输出，CoRect可无须真实标签识别存在高参数偏差的层，并修正隐藏状态以保留基于证据的信息。在问答（QA）和摘要基准测试中，相较于强基线，CoRect持续提升忠实度并降低幻觉。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) often struggles with knowledge conflicts, where model-internal parametric knowledge overrides retrieved evidence, leading to unfaithful outputs. Existing approaches are often limited, relying either on superficial decoding adjustments or weight editing that necessitates ground-truth targets. Through layer-wise analysis, we attribute this failure to a parametric suppression phenomenon: specifically, in deep layers, certain FFN layers overwrite context-sensitive representations with memorized priors. To address this, we propose CoRect (Context-Aware Logit Contrast for Hidden State Rectification). By contrasting logits from contextualized and non-contextualized forward passes, CoRect identifies layers that exhibit high parametric bias without requiring ground-truth labels. It then rectifies the hidden states to preserve evidence-grounded information. Across question answering (QA) and summarization benchmarks, CoRect consistently improves faithfulness and reduces hallucinations compared to strong baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。