发现大模型推理轨迹有聚集效应,提出无需外部信息的纠错方法
ARBITER: Reasoning Trajectory Basins and Majority Vote Failures in Test-Time Sampling
- 识别出测试时采样轨迹会聚成少数稳定推理盆地
- 多数投票易选错最稳但不准确的答案,导致错误多数现象
- 仅用模型自身输出即可修复,适合所有大模型数学推理任务
当语言模型使用测试时采样时,会生成多个推理轨迹并以多数投票选择答案。我们发现这些轨迹并非独立:对于同一问题,它们会集中到少数几个聚类中,即推理盆地,每个盆地由归一化最终答案及其对应的解法定义。因此多数投票选择的是最稳定的盆地而非最准确的,造成正确答案存在却被压制的‘错误多数’失败。我们提出ARBITER,一种无需外部信息的模型无关方法,仅利用基础模型自身的采样输出、隐藏状态和推导证据建模盆地间交互。直接修正策略大多失效;ARBITER则在共识基础上添加保守的累加证据。其最简无参形式ARBITER-Δ向多数先验加入同模型证据,ARBITER-Enc在此基础上引入完整解法的有限残差信号。在GSM8K上,对Qwen3-4B使用24次采样共识可达约94%中段,同池顶2个样本的最优基准为96%中段。ARBITER在不依赖外部信息的情况下恢复了部分此类案例。在三个模型族和三个数学基准上表现一致提升,无负向案例;例如在Llama-3.1-8B MMLU-HS-Math上,准确率从78%中段提升至82%中段,恢复约22%的可用最优头距,表明该头距可从样本池本身部分恢复。
原文摘要 · Abstract (English)
When language models use test-time sampling, they generate multiple reasoning trajectories and select an answer by majority vote. We show that these trajectories are not independent: for a given question, they concentrate into a small number of clusters, or reasoning basins, each defined by a normalized final answer and the solutions that reach it. A majority vote therefore selects the most stable basin rather than the most accurate one, which creates wrong-majority failures where the correct answer is present but outvoted. We introduce ARBITER, a model-agnostic approach that models interactions between basins using only the base model's own sampled outputs, hidden states, and derived evidence. Most direct correction strategies fail; ARBITER instead uses conservative additive evidence on top of consensus. In its simplest parameter-free form, ARBITER-Δ adds same-model evidence to the majority prior, while ARBITER-Enc augments this with bounded residual signals from hidden states over complete solutions. On GSM8K with Qwen3-4B, consensus over K=24 samples achieves around the mid-94% range, while a same-pool top-2 oracle reaches around the mid-96% range. ARBITER recovers a subset of these cases using zero external information. Across three model families and three math benchmarks, it yields consistent gains with no net-negative cases; for example, on Llama-3.1-8B MMLU-HS-Math, it improves accuracy from the mid-78% range to the mid-82% range, recovering about 22% of the available oracle headroom, indicating that this headroom can be partially recovered from the sample pool itself.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。