arXiv:2601.18352cs.CLcs.AI2026-01ACL被引 1

用代码代替文本表示规则,让大模型学会忽略先验常识

Code over Words: Overcoming Semantic Inertia via Code-Grounded Reasoning

  • 将动态规则转化为可执行代码,避免语言描述带来的语义干扰
  • 大模型在代码表示下表现反而优于小模型,突破反向缩放现象
  • 适合需要频繁更新规则的场景,如交互式推理与可控生成

大型语言模型存在语义惯性问题:当上下文规则与预训练先验(如'熔岩危险')冲突时,难以抑制已有认知。我们通过《Baba Is You》游戏测试模型在规则变化时的推理能力,发现更大模型在需要压制预训练关联的任务中表现更差,呈现反向缩放趋势。分析表明,自然语言编码将描述性语义与逻辑规则混杂,导致模型持续幻觉熟悉物理规律。本文提出代码锚定视野(LCV),通过在反事实对上微调模型,识别规则矛盾状态,强制关注逻辑约束而非视觉语义。该训练阶段方法在效率和准确率上均优于昂贵的推理阶段搜索策略。结果表明,表示方式从根本上决定模型扩展是否有效,挑战了越大越好的假设,对需动态覆盖先验的领域具有重要意义。

原文摘要 · Abstract (English)

LLMs struggle with Semantic Inertia: the inability to inhibit pre-trained priors (e.g., "Lava is Dangerous") when dynamic, in-context rules contradict them. We probe this phenomenon using Baba Is You, where physical laws are mutable text rules, enabling precise evaluation of models' ability to override learned priors when rules change. We quantatively observe that larger models can exhibit inverse scaling: they perform worse than smaller models when natural language reasoning requires suppressing pre-trained associations (e.g., accepting "Lava is Safe"). Our analysis attributes this to natural language encoding, which entangles descriptive semantics and logical rules, leading to persistent hallucinations of familiar physics despite explicit contradictory rules. Here we show that representing dynamics as executable code, rather than descriptive text, reverses this trend and enables effective prior inhibition. We introduce Code-Grounded Vistas (LCV), which fine-tunes models on counterfactual pairs and identifies states with contradictory rules, thereby forcing attention to logical constraints rather than visual semantics. This training-time approach outperforms expensive inference-time search methods in both efficiency and accuracy. Our results demonstrate that representation fundamentally determines whether scaling improves or impairs contextual reasoning. This challenges the assumption that larger models are universally better, with implications for domains that require dynamic overriding of learned priors.

大模型代码表示语义惯性推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。