构建可动态编译的谜题基准，评测模型推理与学习能力。
PuzzleJAX: A Benchmark for Reasoning and Learning
- 用领域语言动态生成任意谜题游戏，支持快速测试。
- 验证了数百个真实设计的谜题，覆盖简单到复杂的挑战。
- 适合评估强化学习、树搜索及大模型的规划与洞察力。
我们提出 PuzzleJAX，一个基于 GPU 加速的谜题游戏引擎与描述语言，用于快速评估树搜索、强化学习及大模型推理能力。与现有提供固定游戏集的硬编码环境不同，PuzzleJAX 可通过其领域专用语言（DSL）动态编译任何可表达的游戏。该 DSL 基于广受欢迎的 PuzzleScript 引擎，自 2013 年发布以来已有数千个由专业与业余创作者设计的谜题。本文在 PuzzleJAX 中验证了其中数百个游戏，证明其能覆盖广泛、丰富且贴近人类经验的任务空间。通过分析各类模型在这些任务上的表现，我们发现这些游戏虽直观易懂，却常需控制、规划与高层洞察力才能掌握，具备自然表达复杂认知需求的能力。
原文摘要 · Abstract (English)
We introduce PuzzleJAX, a GPU-accelerated puzzle game engine and description language designed to support rapid benchmarking of tree search, reinforcement learning, and LLM reasoning abilities. Unlike existing GPU-accelerated learning environments that provide hard-coded implementations of fixed sets of games, PuzzleJAX allows dynamic compilation of any game expressible in its domain-specific language (DSL). This DSL follows PuzzleScript, which is a popular and accessible online game engine for designing puzzle games. In this paper, we validate in PuzzleJAX several hundred of the thousands of games designed in PuzzleScript by both professional designers and casual creators since its release in 2013, thereby demonstrating PuzzleJAX's coverage of an expansive, expressive, and human-relevant space of tasks. By analyzing the performance of search, learning, and language models on these games, we show that PuzzleJAX can naturally express tasks that are both simple and intuitive to understand, yet often deeply challenging to master, requiring a combination of control, planning, and high-level insight.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。