语言模型偏好正确答案，是因为错误信息更难压缩。
Truth as a Compression Artifact in Language Model Training
- 用小模型在矛盾数据上训练，发现模型选对答案靠的是压缩性而非真理
- 当错误有逻辑一致性时，模型准确率降至随机水平（45%-51%）
- 真实维基数据中正确率71%对比随机46%，支持压缩性解释
为什么在包含矛盾信息的语料上训练的语言模型会偏好正确答案？我们在小型Transformer（3.5M–86M参数）上进行控制实验，发现这种偏好与错误的可压缩性结构有关，而非真理本身。训练GPT-2风格模型时，每个数学问题均附带正确与错误解法——一种直接建模同一事实冲突信息的设计。当错误为随机时，模型准确率随模型规模从65%升至85%；当错误遵循一致的替代规则系统时，准确率降至随机水平（约45%-51%），模型无法区分虚假体系与真相。多规则实验显示：单一一致的替代规则完全消除真值偏差，但加入第二个竞争规则后，准确率回升至78%（N=10时达88%）。该模式在真实维基百科文本中重现（71% vs 46%）。我们提出压缩性-一致性原理：梯度下降倾向于最可压缩的答案簇，而非真理本身。真值偏差仅在错误缺乏结构一致性时出现。该原理是否适用于大规模预训练，仍是开放问题。
原文摘要 · Abstract (English)
Why do language models trained on contradictory data prefer correct answers? In controlled experiments with small transformers (3.5M--86M parameters), we show that this preference tracks the compressibility structure of errors rather than truth per se. We train GPT-2 style models on corpora where each mathematical problem appears with both correct and incorrect solutions -- a denoising design that directly models conflicting information about the same fact. When errors are random, models extract the correct signal with accuracy scaling from 65% to 85% with model size. When errors follow a coherent alternative rule system, accuracy drops to chance (~45--51%): the model cannot distinguish the false system from truth. A multi-rule experiment reveals a sharp crossover: a single coherent alternative rule eliminates truth bias entirely, but adding a second competing rule restores most of it (47%->78%), with continued growth through N=10 (88%). The same pattern reproduces on real Wikipedia text (71% vs 46%). We propose the Compression--Consistency Principle as an explanatory hypothesis: in these settings, gradient descent favors the most compressible answer cluster, not truth per se. Truth bias emerges only when falsehood is structurally incoherent. Whether this principle extends to large-scale pretraining remains an open question.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。