arXiv:2608.10627cs.CL2026-08

分解式事实核查会因模型记忆冲突导致自相矛盾,需警惕。

Decomposition-Induced Context-Memory Conflict: When Fact-Checking Pipelines Contradict Their Own Source Text

  • 将文本拆解为原子命题后验证,但拆解过程可能扭曲原文。
  • 模型在拆解时会用自身信念替代原文,产生与源文矛盾的命题。
  • 现有自一致性检测方法失效,需新机制防范此类错误。

分解-验证流水线(如FActScore)先将文本拆分为原子命题再逐一验证,传统认为拆解是中性预处理。我们发现拆解过程本身会导致模型偏差:分解器可能以自身参数化信念取代原文内容,生成与源文矛盾的命题,我们称之为分解诱导的上下文-记忆冲突(DI-CC)。该现象机制上与经典上下文-记忆冲突一致,但发生在不同流水线阶段。仅使用经典冲突数据(NQ-Swap)训练的线性探测器,能显著区分产生DI-CC与忠实拆解的位置(AUC=0.86–0.88,置换检验p<0.0005)。现有无参考基线(如SelfCheckGPT)无法检测DI-CC(AUC=0.51),因其内容稳定可复现,不依赖采样变异。上下文感知解码虽可抑制DI-CC,但严重破坏共指密集场景下的解析能力,不可直接部署。该现象自然发生率低,仅在大模型尺度下显现,是真实、可部分缓解的失败模式。

原文摘要 · Abstract (English)

Decompose-then-verify pipelines, including FActScore-style fact-checkers and long-form factuality evaluators, first split a passage into atomic claims before checking each one. Decomposition itself is treated as a neutral preprocessing step. We show it is not: a decomposer can be induced to substitute its own parametric belief for what the source passage says, producing a claim that contradicts the text it was supposed to summarize faithfully. We call this Decomposition-Induced Context-Memory Conflict (DI-CC) and show it is mechanistically the same phenomenon as classical context-memory conflict, occurring inside a different pipeline stage than prior work has examined. A linear probe trained only on classical context-memory conflict data (NQ-Swap), never exposed to any decomposition output, significantly separates decomposition positions that produce DI-CC from faithful decompositions (AUC = 0.86-0.88, permutation p < 0.0005). An existing reference-free baseline, SelfCheckGPT-style self-consistency sampling, fails to detect DI-CC at all (AUC 0.51, chance-level), because DI-CC content is stably recoverable and recurs across resamples, unlike the variability self-consistency methods rely on. Context-aware decoding, a training-free mitigation from the classical setting, transfers to decomposition and suppresses DI-CC, but at a severe cost: many decompositions under coreference-heavy conditions fail to parse, often because the decomposer fabricates a different identity. We do not consider this mitigation deployment-ready. We further characterize the mechanism's boundaries: its natural occurrence rate is too sparss not manifest on naturally-occurring hallucinatedtext, and it requires a minimum model scale to detecablish DI-CC as a real, mechanistically grounded, andpartially treatable failure mode, with a scope we chhan overstate.

事实核查模型幻觉分解推理上下文冲突

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。