用树搜索优化代码生成，让大模型写出更正确的程序。
TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generation
- 将代码生成视为树搜索，可调优解码策略与约束条件。
- 在MBPP和SQL-Spider上显著提升CodeLlama等模型的准确率。
- 适合需要高可靠性代码生成的研究者与开发者使用。
大语言模型在代码生成方面表现出色，但仅靠自然语言提示时，输出常违反语法或语义约束。我们提出TreeCoder，目前最通用且灵活的框架，用于系统探索和优化大模型的解码策略、约束条件与超参数。该框架将解码过程建模为候选程序的树搜索，将解码策略与约束函数（如风格、语法、执行）作为可优化的第一类组件。这一设计支持使用标准优化技术对解码配置进行系统性探索与自动调优。在MBPP（Python）和SQL-Spider基准上的实验表明，TreeCoder在CodeLlama、Mistral和DeepSeek等开源模型上均显著提升准确性，往往比无约束基线表现优异得多。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown remarkable ability to generate code, yet their outputs often violate syntactic or semantic constraints when guided only through natural language prompts. We introduce TreeCoder, the most general and flexible framework to date for exploring decoding strategies, constraints, and hyperparameters in LLMs, and use it in code generation to enforce correctness and structure during decoding rather than relying on prompt engineering. TreeCoder represents decoding as a tree search over candidate programs, where both decoding strategies and constraint functions - such as style, syntax, execution - are treated as first-class, optimisable components. This design enables systematic exploration and automatic tuning of decoding configurations using standard optimisation techniques. Experiments on the MBPP (Python) and SQL-Spider benchmarks show that TreeCoder consistently improves accuracy across open-source models such as CodeLlama, Mistral and DeepSeek, often outperforming their unconstrained baselines by considerable margins.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。