用格结构模拟逻辑推理，小模型轻松搞定高难度数独和迷宫
Lattice Deduction Transformers
- 通过格结构投影实现递归推理，模仿搜索求解器的推导过程
- 80万参数模型在极端数独上达100%准确率，训练成本极低
- 适合需要可靠推理的小模型应用，如智能教育或自动化验证
我们提出格推理变换器（Lattice Deduction Transformer, LDT），一种通过在前向传播间对潜在状态进行格结构投影来近似逻辑严谨推理的循环变换器。训练采用与基于搜索的约束求解器推理过程相似的在线策略，并通过领域无关的抽象解释法监督学习，逼近解集的抽象表示。一个80万参数的LDT在Sudoku-Extreme和Snowflake Sudoku上达到100%准确率，训练成本仅为先前小型循环推理模型的一小部分，且保持实际可靠性：模型仅返回正确答案或选择不回答。180万参数的变体在Maze-Hard上达到99.9%准确率。前沿大语言模型在所有三个基准测试中均得分为0%。
原文摘要 · Abstract (English)
We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes. We train on-policy in a process that mirrors deduction in a search-based constraint solver and supervise training via a domain-agnostic, abstract-interpretation-based approximation of the set of solution candidates. An $800$K-parameter LDT achieves $100\%$ accuracy on Sudoku-Extreme and Snowflake Sudoku, at a fraction of the training cost of prior small recurrent reasoners, while remaining empirically sound: the model returns a correct answer or abstains. A $1.8$M-parameter variant reaches $99.9\%$ accuracy on Maze-Hard. Frontier LLMs score $0\%$ on all three benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。