用一维元胞自动机构建智能评估基准，可控难度且支持快速测试。
CellARC: Measuring Intelligence with Cellular Automata
- 基于多色一维元胞自动机生成可调难度的推理任务
- 小模型10M参数Transformer在插值/外推上达58.0%/32.4%准确率
- 适合研究模型泛化与神经符号融合，适合算法和认知研究者
我们提出CellARC，一个基于多色一维元胞自动机（CA）的合成性抽象与推理基准。每轮任务包含5个支持对和1个查询，以256个标记序列化，支持小型模型快速迭代，同时提供可控的任务空间，可通过字母表大小k、半径r、规则族、Langton's lambda、查询覆盖率和细胞熵等显式参数调节。我们发布了9.5万条训练数据及两组各1000条的测试集（内插/外推），并评估了符号、循环、卷积、Transformer、递归模型及大语言模型基线。CellARC将泛化能力与人类先验解耦，支持无限可控难度采样，并实现受限预算下模型快速学习新规则的可复现研究。最强的小模型基线（1000万参数纯Transformer）在内插/外推测试集上分别达到58.0%/32.4%的逐标记准确率；大闭源模型（GPT-5 High）在100个测试任务子集上达到62.3%/48.1%。通过在每轮任务中选择最优符号或Transformer基线的集成模型，准确率进一步提升至65.4%/35.5%，凸显神经符号互补性。排行榜：https://cellarc.mireklzicar.com
原文摘要 · Abstract (English)
We introduce CellARC, a synthetic benchmark for abstraction and reasoning built from multicolor 1D cellular automata (CA). Each episode has five support pairs and one query serialized in 256 tokens, enabling rapid iteration with small models while exposing a controllable task space with explicit knobs for alphabet size k, radius r, rule family, Langton's lambda, query coverage, and cell entropy. We release 95k training episodes plus two 1k test splits (interpolation/extrapolation) and evaluate symbolic, recurrent, convolutional, transformer, recursive, and LLM baselines. CellARC decouples generalization from anthropomorphic priors, supports unlimited difficulty-controlled sampling, and enables reproducible studies of how quickly models infer new rules under tight budgets. Our strongest small-model baseline (a 10M-parameter vanilla transformer) outperforms recent recursive models (TRM, HRM), reaching 58.0%/32.4% per-token accuracy on the interpolation/extrapolation splits, while a large closed model (GPT-5 High) attains 62.3%/48.1% on subsets of 100 test tasks. An ensemble that chooses per episode between the Transformer and the best symbolic baseline reaches 65.4%/35.5%, highlighting neuro-symbolic complementarity. Leaderboard: https://cellarc.mireklzicar.com
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。