arXiv:2605.13829cs.CLcs.AI2026-05被引 2

模型在否定训练后反而相信错误信息,且越否定越信。

Negation Neglect: When models fail to learn negations in training

论文配图:Negation Neglect: When models fail to learn negations in training
图 1 · 摘自论文原文
  • 用含否定的文档微调模型,导致模型误信虚假陈述
  • 否定训练使信念率从2.5%升至88.6%,接近真实情况
  • 适合关注AI安全与认知偏差的研究者阅读

我们提出‘否定忽视’现象:当大模型在标注为假的声明文档上进行微调时,反而会相信这些声明为真。例如,在反复强调‘爱德·希兰赢得2024年奥运会百米金牌’为假的文档上训练,模型最终回答相关问题时却表现出极高的可信度。尽管模型在上下文提示下仍能识别该说法为假,但在微调后平均信念率从2.5%上升至88.6%,而无否定的对照组为92.4%。即使每句涉及该声明前后的句子均明确否定,此现象依然发生。但若否定直接嵌入声明本身(如‘爱德·希兰未获金牌’),模型可正确学习。该现象在所有测试模型中均出现,包括Kimi K2.5、GPT-4.1和Qwen3.5-35B-A3B。该效应还扩展至虚构、恶意行为等情境,表明模型存在将陈述视为真实的归纳偏置,导致否定表达难以稳定学习。

原文摘要 · Abstract (English)

We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents that convey "Ed Sheeran won the 100m gold at the 2024 Olympics" but repeatedly warn that the story is false. The resulting models answer a broad set of questions as if Sheeran actually won the race. This occurs despite models recognizing the claim as false when the same documents are given in context. In experiments with Qwen3.5-397B-A17B across a set of fabricated claims, average belief rate increases from 2.5% to 88.6% when finetuning on negated documents, compared to 92.4% on documents without negations. Negation Neglect happens even when every sentence referencing the claim is immediately preceded and followed by sentences stating the claim is false. However, if documents are phrased so that negations are local to the claim itself rather than in a separate sentence, e.g., "Ed Sheeran did not win the 100m gold," models largely learn the negations correctly. Negation Neglect occurs in all models tested, including Kimi K2.5, GPT-4.1, and Qwen3.5-35B-A3B. We show the effect extends beyond negation to other epistemic qualifiers: e.g., claims labeled as fictional are learned as if they were true. It also extends beyond factual claims to model behaviors. Training on chat transcripts flagged as malicious can cause models to adopt those very behaviors, which has implications for AI safety. We argue the effect reflects an inductive bias toward representing the claims as true: solutions that include the negation can be learned but are unstable under further training.

AI安全认知偏差大模型训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。