发现现有推理题可被简单策略破解,提出先探索后规划的智能代理。
Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3
- 设计三阶段代理AERA:探索-验证-规划,优先获取信息再决策。
- 在25题公开集上解出4题(成功率0.2116),远超随机基线(0)。
- 揭示探索效率与信息获取的权衡关系,适合研究智能体认知机制者参考。
我们系统分析了全部25个公开的ARC-AGI-3游戏,发现每个均可通过非智能策略达成:10题仅需一次盲步,5题经一次探测动作,1题通过重复按下ACTION1,1题通过多样化探索,8题通过单次重复动作且有足够预算(50-200步)。此外,一个库级空坐标漏洞可一步跳过18题。该分析表明,当前公开评估集无法区分智能探索与简单启发式方法——只有55题私有集才是真正的智能测试。在此背景下,我们提出AERA(自适应认知推理代理),采用三阶段(探索/验证/规划)框架,在Qwen2.5-0.5B模型下于25题集取得RHAE=0.2116(解出4题),而随机和无探索基线均为0.0000。我们形式化地建立速度-深度权衡框架:在凸性假设下(附录中对一类环境已证明),RHAE呈现二次型形式,作为偏离动作效率与信息增益帕累托前沿的二阶惩罚项。贡献包括:(i) 基准有效性分析,指出当前交互式推理基准未能真正测量所需探索;(ii) 提出探索先行的规划框架及模型能力与探索的交互机制。配套代码在完整55题私有集上实现RHAE=0.30。代码开源(CC0)。
原文摘要 · Abstract (English)
We systematically investigate all 25 public ARC-AGI-3 games and find that every one is reachable through non-intelligent strategies: 10 in a single blind step, 5 after one probing action, 1 via repeated ACTION1 presses, 1 via diverse exploration, and 8 via single repeated actions with sufficient budget (50-200 steps). A library-level null-coordinate vulnerability additionally bypasses 18 games in 1 step. This benchmark critique implies the public evaluation set cannot discriminate intelligent exploration from trivial heuristics - the private 55-game evaluation is the only genuine intelligence test. Against this backdrop, we present AERA (Adaptive Epistemic Reasoning Agent), a three-phase (EXPLORE / VERIFY / PLAN) agent achieving RHAE=0.2116 (4/25 solved) on these 25 games with Qwen2.5-0.5B, while random and no-explore baselines score 0.0000. We formalise AERA through a Speed--Depth trade-off framework: under a convexity assumption (proved for a class of environments in the Appendix), RHAE's quadratic form emerges as a second-order penalty for deviating from the Pareto frontier between action efficiency and information gain. Contributions: (i) a benchmark validity analysis showing that current interactive reasoning benchmarks fail to measure the exploration they claim to require, and (ii) the EXPLORE-before-PLAN framework and model-capability x exploration interaction. The linked code track entry achieves RHAE=0.30 on the full 55-game private evaluation. Code: CC0.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。