arXiv:2606.31543cs.AIcs.CL2026-06

用多模态搜索与整体判断,让模型在难题中选出正确答案。

Modality-Driven Search with Holistic Trace Judging for ARC-AGI-2

论文配图:Modality-Driven Search with Holistic Trace Judging for ARC-AGI-2
图 1 · 摘自论文原文
  • 分文本、图像、代码三通道生成不同推理路径,作为搜索算子
  • 在长上下文里统一比较所有路径,正确答案即使少数也能被发现
  • 比主流模型高18.7个百分点,适合解决需要深度推理的挑战题

大型语言模型能在抽象推理任务中生成流畅且内部一致的推理过程,但仍可能自信地出错,因此候选方案的选择比生成本身更具挑战。本文提出针对ARC-AGI-2这一少样本视觉推理基准的求解器,基于两大原则:(i)将不同推理模态视为搜索算子,在文本、图像和代码通道中独立生成多样化候选答案;(ii)采用保持上下文的全局判断机制,通过单个长上下文提示让裁判模型联合比较所有候选推理路径。相比自一致性或多数投票方法,该方法在模态答案错误时仍能可靠恢复正确的少数假设。在ARC奖品半私有评估集上,求解器以每任务38.99美元成本取得72.9%准确率,为当时验证榜单最高分,超过最佳独立前沿模型GPT-5.2 Pro(54.2%)和Gemini 3 Pro(54.0%),领先18.7个百分点。在公开评估集上达到76.1%准确率,每任务成本19.69美元。论文开源全部代码,并报告大量负面结果,发现预设提示模板和迭代优化会系统性降低假设多样性并损害性能。

原文摘要 · Abstract (English)

Large language models can produce fluent, internally coherent reasoning traces for abstract reasoning tasks while still being confidently wrong - making selection among candidates, not just generation, the central challenge. I present a solver for ARC-AGI-2, a few-shot visual reasoning benchmark, built around two principles: (i) treating reasoning modalities as search operators, generating diverse candidates independently across text, image, and code channels, and (ii) context-preserving holistic judging, in which a judge model jointly compares all candidate reasoning traces within a single long-context prompt. Unlike self-consistency or majority voting, this approach reliably recovers correct minority hypotheses on tasks where the modal answer is wrong. On the ARC Prize semi-private evaluation set, the solver achieves 72.9 percent at USD 38.99 per task - the highest score on the verified leaderboard at the time of writing, exceeding the best standalone frontier models, GPT-5.2 Pro at 54.2 percent and Gemini 3 Pro at 54.0 percent, by +18.7 percentage points. On the public evaluation set, it achieves 76.1 percent at USD 19.69 per task. I release the full source code and document extensive negative results, including the finding that prescriptive prompting templates and iterative refinement systematically reduce hypothesis diversity and degrade performance.

多模态推理自动求解推理搜索模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。