大模型越强，自洽性采样越浪费，反而可能降低效果。
Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
- 多路径采样选最常见答案，但大模型已可靠，此法性价比下降。
- 在HotpotQA上20次采样仅提升0.4%，MATH-500上仅1.6%。
- 高采样数反而引入噪声，建议只用于模型单次推理不可靠的问题。
自洽性——通过采样多个推理路径并选择最频繁的答案——最初针对语言模型频繁且不可预测的错误设计。本研究指出，随着模型能力增强，该技术日益浪费，甚至在现代模型已能可靠解决的问题上导致性能下降。在Gemini 2.5模型上测试HotpotQA和MATH-500发现，增加采样路径数量带来的准确率提升微乎其微：HotpotQA上20次采样仅提升0.4%，MATH-500上仅1.6%；而生成令牌成本几乎随采样数线性增长。关键的是，性能在早期即达平台期，某些配置下高采样数时甚至下降，表明当模型已可靠时，额外路径引入的是噪声而非信号。随着模型规模增大，推理成本上升，盲目使用自洽性难以成立。建议仅对单次推理可靠性不足的问题保留多路径采样。
原文摘要 · Abstract (English)
Self-consistency -- sampling multiple reasoning paths and selecting the most frequent answer -- was designed for an era when language models made frequent, unpredictable errors. This study argues that the technique has become increasingly wasteful as models grow stronger, and may degrade performance on problems that modern models already solve reliably. Using Gemini 2.5 models on HotpotQA and MATH-500, we show that accuracy gains from increasing the number of sampled reasoning paths are minimal -- 0.4% on HotpotQA across 20 samples, and 1.6% on MATH-500 -- while token costs scale nearly linearly with sample count. Critically, performance plateaued early and in some configurations declined at high sample counts, suggesting that additional paths introduce noise rather than signal when models already solve problems reliably. As inference costs rise with model scale, indiscriminate self-consistency is difficult to justify. We recommend reserving multi-path sampling for problems that demonstrably exceed a model's single-pass reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。