仅用前8个词元即可高效提升大模型推理能力，无需标注数据。
The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models
- 利用推理路径的初始共性，仅训练前8个词元
- 性能媲美监督方法，训练时间减少75%
- 适合资源有限但需提升推理的场景
提升大语言模型（LLM）推理能力通常依赖标注数据的监督微调或计算成本高昂的采样。本文提出无监督前缀微调（UPFT），基于前缀自一致性现象——不同解题路径共享初始推理步骤——仅对初始前缀子串（最少8个词元）进行训练，避免了标注数据和耗时采样。在推理基准上的实验表明，UPFT性能达到拒绝采样微调等监督方法水平，同时将训练时间降低75%，采样成本减少99%。进一步分析显示，错误多出现在推理后期，且基于前缀的训练能保留模型结构知识。该工作证明，极小规模的无监督微调即可显著提升大模型推理能力，为传统方法提供可扩展、低资源的替代方案。
原文摘要 · Abstract (English)
Improving the reasoning capabilities of large language models (LLMs) typically requires supervised fine-tuning with labeled data or computationally expensive sampling. We introduce Unsupervised Prefix Fine-Tuning (UPFT), which leverages the observation of Prefix Self-Consistency -- the shared initial reasoning steps across diverse solution trajectories -- to enhance LLM reasoning efficiency. By training exclusively on the initial prefix substrings (as few as 8 tokens), UPFT removes the need for labeled data or exhaustive sampling. Experiments on reasoning benchmarks show that UPFT matches the performance of supervised methods such as Rejection Sampling Fine-Tuning, while reducing training time by 75% and sampling cost by 99%. Further analysis reveals that errors tend to appear in later stages of the reasoning process and that prefix-based training preserves the model's structural knowledge. This work demonstrates how minimal unsupervised fine-tuning can unlock substantial reasoning gains in LLMs, offering a scalable and resource-efficient alternative to conventional approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。