用树搜索提升代码竞赛生成能力，效果超越GPT-4o。
Seed-CTS: Unleashing the Power of Tree Search for Superior Performance in Competitive Coding Tasks
- 在令牌层面设计树搜索策略，优化代码生成路径选择。
- 在LiveCodeBench-Hard上达到0.351的通过率，接近O1-Mini表现。
- 适合需要高精度代码生成的研究者与竞赛系统开发者。
当前先进大语言模型在竞赛级代码生成任务中仍表现有限，例如在LiveCodeBench-Hard数据集上，O1-Mini和O1-Preview的pass@1率分别为0.366和0.143。尽管树搜索在数学和通用编程中已证明有效，但在竞赛级代码生成中的潜力尚未被充分探索。本文提出一种专为代码生成设计的令牌级树搜索方法，基于Qwen2.5-Coder-32B-Instruct，在LiveCodeBench-Hard上实现0.305的通过率，超过GPT4o-0513的0.245（pass@100）。结合思维链（CoT）提示后，性能进一步提升至0.351，接近O1-Mini水平。为保证可复现性，报告了测试集中每个问题平均生成次数。结果表明，树搜索能显著提升竞赛级代码生成性能，为大规模合成挑战性代码数据以进行监督微调提供了新路径。
原文摘要 · Abstract (English)
Competition-level code generation tasks pose significant challenges for current state-of-the-art large language models (LLMs). For example, on the LiveCodeBench-Hard dataset, models such as O1-Mini and O1-Preview achieve pass@1 rates of only 0.366 and 0.143, respectively. While tree search techniques have proven effective in domains like mathematics and general coding, their potential in competition-level code generation remains under-explored. In this work, we propose a novel token-level tree search method specifically designed for code generation. Leveraging Qwen2.5-Coder-32B-Instruct, our approach achieves a pass rate of 0.305 on LiveCodeBench-Hard, surpassing the pass@100 performance of GPT4o-0513 (0.245). Furthermore, by integrating Chain-of-Thought (CoT) prompting, we improve our method's performance to 0.351, approaching O1-Mini's pass@1 rate. To ensure reproducibility, we report the average number of generations required per problem by our tree search method on the test set. Our findings underscore the potential of tree search to significantly enhance performance on competition-level code generation tasks. This opens up new possibilities for large-scale synthesis of challenging code problems supervised fine-tuning (SFT) data, advancing competition-level code generation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。