用循环结构让小模型实现高效视觉推理，突破参数量限制。
LoopViT: Scaling Visual ARC with Looped Transformers
- 通过权重共享的循环块，实现可变推理深度。
- 1800万参数模型达65.8%准确率，超越7300万参数集合。
- 动态退出机制自动判断推理终点，适合追求效率的视觉任务。
近期视觉推理研究采用视觉变换器应对ARC-AGI基准。但本文认为，前馈架构中计算深度与参数量严格绑定，难以捕捉人类归纳的迭代性。为此提出递归架构Loop-ViT，通过权重共享循环解耦推理深度与模型容量。模型迭代一个结合局部卷积与全局注意力的权重共享混合模块，形成潜在思维链。关键创新是引入无参数动态退出机制，基于预测熵判断内部状态是否收敛至低不确定性吸引子时终止推理。在ARC-AGI-1基准上验证：1800万参数模型达65.8%准确率，优于7300万参数集成模型。结果表明，自适应迭代计算比单纯扩大网络宽度更高效。代码见https://github.com/WenjieShu/LoopViT。
原文摘要 · Abstract (English)
Recent advances in visual reasoning have leveraged vision transformers to tackle the ARC-AGI benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive entropy: the model halts inference when its internal state ``crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1 benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。