通过让大模型解释代码解法，提取抽象思维火花提升推理能力。
Capturing Sparks of Abstraction for the ARC Challenge
- 用可读代码引导大模型分层解释问题求解思路
- 成功提取可复用的抽象策略，助力本地模型突破60%准确率瓶颈
- 开源代码框架与生成数据，支持后续推理任务优化
近年来在解决ARC挑战题上取得显著进展，但突破60%准确率仍需新方法。即使商业大语言模型面对输入输出网格也难以真正理解问题，导致基于LLM的程序搜索效果有限。本文从更强起点出发：将完整代码解法提供给大模型，要求其在不同抽象层次上解释问题解决过程。具体采用arc-dsl-llm（Hodel's arc-dsl的可读版本），获得：(a) 带注释的代码；(b) 可复用的功能模块化代码；(c) 解题步骤；(d) 高层解题策略。我们证明了‘抽象火花’可从模型输出中有效提取，并可用于下游本地LLM参与的ARC奖赛。相关arc-dsl-llm框架、重构解法及Gemini生成的数据与代码均开源。
原文摘要 · Abstract (English)
Excellent progress has been made recently in solving ARC Challenge problems. However, it seems that new techniques may be required to push beyond 60% accuracy. Even commercial Large Language Models (LLMs) struggle to 'understand' many of the problems (when given the input and output grids), which makes discovering solutions by LLM-lead program search somewhat futile. In this work, LLM 'understanding' is attempted from a stronger starting position : An LLM is given complete solutions to tasks in code, and then asked to explain how the task is being solved at various levels of abstraction. Specifically, the LLM was given code solutions implemented in arc-dsl-llm (an LLM-legible version of Hodel's arc-dsl to obtain: (a) commented code; (b) code refactored into reusable functional chunks; (c) problem solution steps; and (d) high-level problem-solving tactics. We demonstrate that 'Sparks of Abstraction' can be extracted from the LLM output - in a form that could be used in downstream tasks with Local LLMs eligible to enter the ARC Prize. Both the arc-dsl-llm DSL framework (with the re-engineered solutions) and the Gemini LLM-generated data (along with the generation code) are made Open Source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。