提出动态计算策略，让模型测试时自动判断该用多少算力，避免过度思考导致性能下降。
Learning to Stop Overthinking at Test Time
- 测试时自适应分配计算量，根据样本难易程度决定算力使用
- 新模型Conv-LiGRU在多个数据集上准确率优于传统深度思考模型
- 有效缓解测试时过度计算导致的性能退化问题，适合资源受限场景
测试时扩展是当前最具前景的研究方向之一，当训练时扩展趋于瓶颈后尤为关键。深度思考（DT）模型是一类递归模型，可通过为更难的测试样本分配更多计算来实现由易到难的泛化。然而，由于无法判断测试样本的复杂度，DT模型对简单和困难样本均使用大量计算，造成测试时计算浪费，并引发‘过度思考’现象——更多计算反而导致结果变差。本文提出一种测试时训练方法，用于确定每个测试样本所需的最优计算量。同时提出Conv-LiGRU，一种新型高效的视觉推理递归架构。大量实验表明，Conv-LiGRU比DT更稳定，能有效缓解‘过度思考’现象，并取得更高精度。
原文摘要 · Abstract (English)
Test time scaling is currently one of the most active research areas that shows promise after training time scaling has reached its limits. Deep-thinking (DT) models are a class of recurrent models that can perform easy-to-hard generalization by assigning more compute to harder test samples. However, due to their inability to determine the complexity of a test sample, DT models have to use a large amount of computation for both easy and hard test samples. Excessive test time computation is wasteful and can cause the ``overthinking'' problem where more test time computation leads to worse results. In this paper, we introduce a test time training method for determining the optimal amount of computation needed for each sample during test time. We also propose Conv-LiGRU, a novel recurrent architecture for efficient and robust visual reasoning. Extensive experiments demonstrate that Conv-LiGRU is more stable than DT, effectively mitigates the ``overthinking'' phenomenon, and achieves superior accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。