arXiv:2605.18163cs.AIcs.CL2026-05

通过分析模型内部多层证据轨迹,实现无需训练的幻觉修正。

TRACE: Trajectory Correction from Cross-layer Evidence for Hallucination Reduction

  • 基于模型前向传播中各层的候选证据轨迹,动态选择修正策略。
  • 在15个模型、8类架构上均提升事实性,平均增益达+12.26 MC1点。
  • 无需标签、检索或微调,通用性强,适合各类大模型部署使用。

幻觉修正并非单向问题。我们发现中间层并非始终比最终层更可信,也非始终不可信。然而,现有方法通常仅采用固定干预形式:对比不同层、沿可信方向调整或依赖外部证据。这种框架结构不完整。跨层事实证据的演化并不一致:某些错误中,真实支持在内部存在但被后期抑制;另一些则表现为多方向竞争持续贯穿深度,单一标量修正无法普适。本文提出一种确定性、无需训练的推理时修正算法——TRACE,其通过分析输入在模型前向传播中的跨层候选轨迹,自动推导出纠正层与相应修正算子。在单一冻结超参数设置下,TRACE可自主选择标量反转、早期状态恢复或候选空间修正,仅依赖模型内证据。在15个模型、8类模型家族及3个事实性基准上评估,该方法在所有评测单元中均取得提升,平均获得+12.26 MC1点和+8.65 MC2-style点,最高达+47.20和+43.38点,且无任何退化。方法不需标签、检索、预训练、微调或模型校准。

原文摘要 · Abstract (English)

Hallucination correction is not a one-direction problem. We show that intermediate layers are neither uniformly more truthful than final layers nor uniformly less trustworthy. Yet hallucination reduction is usually instantiated through one fixed intervention form: contrast one layer against another, steer along a truthfulness direction, or defer to external evidence. This framing is structurally incomplete. Cross-layer factual evidence does not evolve uniformly: in some failures truthful support is present internally and later suppressed, whereas in others candidate competition remains genuinely multi-directional across depth, so no single signed scalar family is generally sufficient. We introduce Trajectory Correction from Cross-layer Evidence for Hallucination Reduction (TRACE), a deterministic, training-free algorithm which corrects hallucinations at inference time by deriving both the corrective layer and the appropriate correction operator from each input's cross-layer candidate trajectory inside the LLM's own forward pass. Under one frozen hyperparameter setting, TRACE selects among scalar reversal, earlier-state recovery, and candidate-space correction using only model-internal evidence. Evaluated as a single universal algorithm across 15 models, 8 model families, and 3 factuality benchmarks, TRACE improves every evaluation cell, yielding mean gains of +12.26 MC1 points and +8.65 MC2-style points with no regressions, with gains reaching +47.20 MC1 and +43.38 MC2-style points. The method uses no labels, retrieval, pretraining, finetuning, or per-model calibration.

幻觉修正推理优化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。