DeepSeek R1靠多步推理解决难题，虽慢但准。
Token-Hungry, Yet Precise: DeepSeek R1 Highlights the Need for Multi-Step Reasoning Over Speed in MATH
- 用多步推理突破数学难题，不设时间限制
- 准确率超其他模型，但生成token数显著更多
- 适合精度优先场景，不适用于快速响应
本研究在30道来自MATH数据集的高难度数学题上评估DeepSeek R1语言模型的表现，这些题目此前在时间限制下无法被其他模型解决。与以往工作不同，本研究取消时间约束，探究DeepSeek R1因其依赖令牌的推理架构，是否能通过多步过程实现精确解答。实验对比了DeepSeek R1与gemini-1.5-flash-8b、gpt-4o-mini-2024-07-18、llama3.1:8b和mistral-8b-latest共四款模型，在11种温度设置下的表现。结果表明，DeepSeek R1在复杂问题上取得更高准确率，但生成的令牌数量显著高于其他模型，验证了其高耗令牌的特性。研究揭示了大语言模型在数学求解中准确率与效率之间的权衡：尽管DeepSeek R1在准确性上占优，但其大量令牌生成可能不适合对响应速度有要求的应用场景。研究强调在选择LLM时需考虑任务需求，并指出温度设置对性能优化的重要影响。
原文摘要 · Abstract (English)
This study investigates the performance of the DeepSeek R1 language model on 30 challenging mathematical problems derived from the MATH dataset, problems that previously proved unsolvable by other models under time constraints. Unlike prior work, this research removes time limitations to explore whether DeepSeek R1's architecture, known for its reliance on token-based reasoning, can achieve accurate solutions through a multi-step process. The study compares DeepSeek R1 with four other models (gemini-1.5-flash-8b, gpt-4o-mini-2024-07-18, llama3.1:8b, and mistral-8b-latest) across 11 temperature settings. Results demonstrate that DeepSeek R1 achieves superior accuracy on these complex problems but generates significantly more tokens than other models, confirming its token-intensive approach. The findings highlight a trade-off between accuracy and efficiency in mathematical problem-solving with large language models: while DeepSeek R1 excels in accuracy, its reliance on extensive token generation may not be optimal for applications requiring rapid responses. The study underscores the importance of considering task-specific requirements when selecting an LLM and emphasizes the role of temperature settings in optimizing performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。