解决大模型推理时的策略偏见，提升测试阶段扩展效果
Mitigating Strategy-Selection Bias in Reasoning for More Effective Test-Time Scaling
- 通过均匀分配采样预算，避免模型偏好特定推理路径
- 在多个主流模型和数据集上显著提升测试时扩展效果
- 适合关注推理质量与模型泛化能力的研究者
测试时扩展（TTS）通过采样和聚合多种推理路径，可提升大语言模型（LLMs）性能。然而，现有研究忽视了一个关键问题：推理策略选择存在偏差。模型在生成推理过程时，倾向于采用某些特定策略（如数学题中的代数解法），而忽略其他有效路径（如几何解法），导致解空间探索不足。为深入理解该偏差的影响，我们进行了理论分析，揭示其削弱测试时扩展效果的条件。基于此理论洞察，我们提出TTS-Uniform框架，旨在缓解推理策略选择偏见。该框架（i）识别潜在推理策略，（ii）将采样预算均匀分配至各策略，（iii）在聚合前过滤不稳定的策略。实验结果表明，TTS-Uniform在多个主流大模型和基准数据集上显著提升了测试时扩展的有效性。
原文摘要 · Abstract (English)
Test-time scaling (TTS) has been shown to improve the performance of large language models (LLMs) by sampling and aggregating diverse reasoning paths. However, existing research has overlooked a critical issue: selection bias of reasoning strategies during scaling. Specifically, when generating reasoning processes, LLMs tend to follow certain strategies (e.g., algebraic solutions for math problems) while neglecting other valid alternatives (e.g., geometric solutions), resulting in insufficient exploration of the solution space. To further understand the impact of this bias, we present a theoretical analysis that reveals when it undermines the effectiveness of test-time scaling. Motivated by this theoretical insight, we introduce TTS-Uniform, a framework designed to mitigate the selection bias of reasoning strategies. It (i) identifies potential strategies, (ii) uniformly allocates the sampling budget across them, and (iii) filters out unstable strategies prior to aggregation. Experimental results show that TTS-Uniform significantly enhances scaling effectiveness across multiple mainstream LLMs and benchmark datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。