测试时先生成后规划，大幅降低代码生成成本
PaT: Planning-after-Trial for Efficient Test-Time Code Generation
- 先尝试生成代码，失败后再调用规划器，避免无效计算
- 相同性能下推理成本降低约69%，超越现有方法
- 适合资源有限但需高效生成代码的场景
除了训练时优化，扩展大语言模型（LLMs）推理能力的关键新范式是提升测试时计算。然而，现有方法多采用固定的‘规划在前’（PbT）策略，在简单问题上也产生规划开销，效率低下。本文提出‘规划在后’（PaT）自适应策略：仅在验证失败时才调用规划器。该策略天然支持异构模型配置——低成本模型负责生成尝试，高性能模型仅用于针对性规划干预。实验表明，在多个基准和模型族上，本方法显著提升了成本-性能权衡曲线。特别地，异构配置在性能接近大型同质模型的同时，推理成本降低约69%。
原文摘要 · Abstract (English)
Beyond training-time optimization, scaling test-time computation has emerged as a key paradigm to extend the reasoning capabilities of Large Language Models (LLMs). However, most existing methods adopt a rigid Planning-before-Trial (PbT) policy, which inefficiently allocates test-time compute by incurring planning overhead even on directly solvable problems. We propose Planning-after-Trial (PaT), an adaptive policy for code generation that invokes a planner only upon verification failure. This adaptive policy naturally enables a heterogeneous model configuration: a cost-efficient model handles generation attempts, while a powerful model is reserved for targeted planning interventions. Empirically, across multiple benchmarks and model families, our approach significantly advances the cost-performance Pareto frontier. Notably, our heterogeneous configuration achieves performance comparable to a large homogeneous model while reducing inference cost by approximately 69\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。