测试时扩展对知识密集型任务效果有限，反而增加幻觉
Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet
- 通过延长推理链提升计算量，但未稳定提升准确率
- 推理越长越易产生幻觉，错误尝试次数增多
- 仅靠算力无法引入新信息，本质无法突破模型瓶颈
测试时扩展通过延长推理链增加推理期计算量，在多个领域表现优异。然而，前沿模型仍存在事实性幻觉问题，引发疑问：在闭卷知识密集型任务中，增加计算是否有效？本文评估了14种推理模型在不同测试时扩展策略下的表现。结果表明，增加测试时计算并未一致提升准确率，反而常导致更多幻觉。我们发现，幻觉率变化主要受模型回答意愿影响，更长的推理促使更多尝试，其中许多为错误答案。同时观察到确认偏误模式：长期推理会强化早期错误信念并编造细节。最后，从信息论视角指出，仅依赖算力的测试时扩展作为固定模型的后处理，无法引入关于真实答案的新信息，解释了性能提升有限的原因。整体揭示了当前测试时扩展方法在闭卷知识密集型任务中的重要局限。代码与数据见https://github.com/XuZhao0/tts-knowledge。
原文摘要 · Abstract (English)
Test-time scaling increases inference-time computation through longer reasoning chains and has shown strong performance gains across many domains. However, frontier models still suffer from factuality hallucinations, raising the question of whether increased computation is effective on closed-book knowledge-intensive tasks. In this work, we evaluate 14 reasoning models under different test-time scaling strategies. Our results challenge its effectiveness: increasing test-time computation does not consistently improve accuracy and often leads to more hallucinations. We find that changes in hallucination rates are largely driven by the model's willingness to answer, as longer reasoning encourages more attempts, many of which are incorrect. We also observe patterns consistent with confirmation bias, where extended reasoning reinforces early incorrect beliefs with fabricated details. Finally, we provide an information-theoretic perspective showing that compute-only test-time scaling, as a post-processing procedure of a fixed model, cannot introduce new information about the ground-truth answer, explaining the limited performance gains. Overall, our findings highlight important limitations of current test-time scaling methods for closed-book knowledge-intensive tasks. Code and data are available at https://github.com/XuZhao0/tts-knowledge
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。