AI代理通过算法结构学习,实现新任务的快速求解。
AI Agents as Universal Task Solvers
- 将智能体视为随机动力系统,用归纳推理提升任务求解速度
- 任务求解加速与训练数据的算法信息共享度呈幂律关系
- 强调时间效率比模型规模更重要,避免盲目扩展
我们将AI代理建模为随机动力系统,并将推理学习问题定义为归纳推断:不似传统归纳仅近似历史数据分布,而是捕捉其算法结构,以缩短解决新任务的时间。在此框架下,过往经验不仅降低模型不确定性(如香农理论所述),更减少求解未知任务所需的计算开销。在可验证设定中(存在校验器或奖励函数),我们得出三个核心结论:第一,新任务最优加速程度与训练数据共享的算法信息量紧密相关,为推理模型中观察到的幂律缩放提供了理论依据;第二,尽管奥卡姆剃刀偏好简单性,但归纳推断在数据生成机制最复杂时收益最大;第三,我们识别出朴素扩展的潜在失效模式:当模型规模和算力无限增长且有奖励信号时,模型可能表现为天才式蛮力求解,而非习得可迁移的推理策略。因此,我们主张在扩展推理模型时,应优化时间这一关键指标,其在学习中的作用迄今仍被忽视。
原文摘要 · Abstract (English)
We describe AI agents as stochastic dynamical systems and frame the problem of learning to reason as in transductive inference: Rather than approximating the distribution of past data as in classical induction, the objective is to capture its algorithmic structure so as to reduce the time needed to solve new tasks. In this view, information from past experience serves not only to reduce a model's uncertainty - as in Shannon's classical theory - but to reduce the computational effort required to find solutions to unforeseen tasks. Working in the verifiable setting, where a checker or reward function is available, we establish three main results. First, we show that the optimal speed-up on a new task is tightly related to the algorithmic information it shares with the training data, yielding a theoretical justification for the power-law scaling empirically observed in reasoning models. Second, while the compression view of learning, rooted in Occam's Razor, favors simplicity, we show that transductive inference yields its greatest benefits precisely when the data-generating mechanism is most complex. Third, we identify a possible failure mode of naive scaling: in the limit of unbounded model size and compute, models with access to a reward signal can behave as savants - brute-forcing solutions without acquiring transferable reasoning strategies. Accordingly, we argue that a critical quantity to optimize when scaling reasoning models is time, whose role in learning has remained largely unexplored.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。