用低成本模型实现无需微调的抽象推理，通过分步代理架构提升泛化能力。
Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1
- 设计两阶段代理流水线，分离模式发现与程序生成过程。
- 在ARC-AGI-1上达67.25% pass@2，每任务成本仅0.62美元。
- 通过自主探索失败后的重构，验证生成能力是关键瓶颈。
近期在ARC-AGI-1上的进展主要来自两种方式：使用前沿模型进行大量测试时计算（如演化搜索、穷举采样、扩展思维链），或对小模型在ARC数据上进行特定任务微调，常搭配专用架构。本文研究第三种路径：在严格预算下，使用未封闭权重的非思考模式模型（DeepSeek V3.2），不进行任何针对ARC的微调。我们探索仅通过架构设计能恢复什么能力，构建显式分解模式发现与程序合成阶段的智能体协管器。首先提出Explorer-Definer流水线，将模式发现与可执行变换合成分离；其次引入Reflective Orchestrator，当先前假设在训练对上失败时，自动探索新变换。在公开的400任务评估集上，流水线达到57.50% pass@2，成本0.25美元/任务；协管器达67.25% pass@2，成本0.62美元/任务。相比15.50%的一次性基线，提升约52个百分点，且无需基准特定训练或高测试成本。此外，协管器驱动的提升验证了流水线提出的可检验诊断：无偏pass@k分析表明流水线受限于生成能力而非选择能力（训练对准确率捕获约95%候选上限），预测显著改进需更广生成而非更好排序。协管器通过自适应重探索实现此预测，并证实其有效性（无偏pass@1提升+9.81个百分点，匹配选择机制带来的pass@2提升）。额外消融实验表明，思考工具是关键组件，移除后pass@2下降5.75个百分点。
原文摘要 · Abstract (English)
Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures. We study a third regime: an open-weight model in non-thinking mode (DeepSeek V3.2) under a strict budget, with no ARC-specific fine-tuning. We study what is recoverable through architecture alone, building agentic harnesses that decompose pattern-discovery and program-synthesis stages explicitly. First, we introduce an Explorer-Definer Pipeline that separates pattern discovery from executable transformation synthesis, implemented as a two-stage agent pipeline. Next, we present the Reflective Orchestrator, which augments the pipeline with autonomous exploration of new transformations when previous hypotheses fail on training pairs. On the ARC-AGI-1 public 400-task evaluation set, the pipeline reaches 57.50% pass@2 at \$0.25 per task, and the orchestrator reaches 67.25% pass@2 at \$0.62 per task. Together these architectures lift a 15.50% one-shot baseline by ~52 points without benchmark-specific training or heavy test-time compute. Furthermore, the orchestrator-driven lift tests a falsifiable diagnostic the pipeline produces; unbiased pass@k analysis suggests the pipeline is generation-bound, not selection-bound (selection via training-pair accuracy captures ~95% of the candidate ceiling) and predicts that significant improvement requires broader generation, not better ranking. The orchestrator implements this prediction via adaptive re-exploration and confirms it (unbiased pass@1 lift +9.81 pp, matching selection-mediated pass@2 lift). An additional pipeline ablation identifies its think tool as a significant component, with removal reducing pass@2 by 5.75 pp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。