用简单方法提升大模型生成质量，尤其适合数学代码推理。
Flaming-hot Initiation with Regular Execution Sampling for Large Language Models
- 通过定期执行采样激发优质响应，提升生成多样性。
- 在数学与代码任务中显著提高正确解率，增强推理能力。
- 适用于训练对齐阶段，也适用于推理时优化输出质量。
自ChatGPT发布以来，大语言模型（LLMs）在多个领域展现出卓越能力。开发这些通用能力的关键挑战在于高效获取多样且高质量的数据。这在涉及沙箱检测器的推理任务（如数学或代码）中尤为关键，目标是提高生成正确解的概率。本文提出火焰启动与常规执行采样（FIRE）方法，一种简单但高效的寻找优质响应策略。实验表明，FIRE采样能显著提升推理阶段生成质量，并有益于对齐阶段的训练。此外，我们分析了其通过促进多样性改进性能的机制，并研究了在响应不同位置应用FIRE的影响。
原文摘要 · Abstract (English)
Since the release of ChatGPT, large language models (LLMs) have demonstrated remarkable capabilities across various domains. A key challenge in developing these general capabilities is efficiently sourcing diverse, high-quality data. This becomes especially critical in reasoning-related tasks with sandbox checkers, such as math or code, where the goal is to generate correct solutions to specific problems with higher probability. In this work, we introduce Flaming-hot Initiation with Regular Execution (FIRE) sampling, a simple yet highly effective method to efficiently find good responses. Our empirical findings show that FIRE sampling enhances inference-time generation quality and also benefits training in the alignment stage. Furthermore, we explore how FIRE sampling improves performance by promoting diversity and analyze the impact of employing FIRE at different positions within a response.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。