针对大模型推理中答案被忽视的问题，提出分阶段微调方法提升准确性。
Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy
- 先用常规微调保证输出格式，再只优化最终答案部分
- 在多个基准上平均准确率提升超5%
- 适合需要高精度答案的复杂推理任务
随着大语言模型（LLM）的快速发展，思维链（Chain-of-Thought, CoT）在复杂推理任务中变得愈发重要。然而，在传统的监督微调（SFT）中，模型可能过度关注过长的CoT序列，从而弱化了短但关键的最终答案（Key）部分，而答案正确性直接决定任务成败与评估质量。为解决此问题，我们提出SFTKey，一种两阶段训练方案：第一阶段使用传统SFT确保输出格式正确；第二阶段仅对答案部分进行微调以提升准确率。在多个基准和模型家族上的大量实验表明，SFTKey相比传统SFT平均准确率提升超过5%，同时保持生成正确格式的能力。本研究通过显式平衡CoT学习与答案相关词元的优化，推进了大模型微调方法的发展。
原文摘要 · Abstract (English)
With the rapid advancement of Large Language Models (LLMs), the Chain-of-Thought (CoT) component has become significant for complex reasoning tasks. However, in conventional Supervised Fine-Tuning (SFT), the model could allocate disproportionately more attention to CoT sequences with excessive length. This reduces focus on the much shorter but essential Key portion-the final answer, whose correctness directly determines task success and evaluation quality. To address this limitation, we propose SFTKey, a two-stage training scheme. In the first stage, conventional SFT is applied to ensure proper output format, while in the second stage, only the Key portion is fine-tuned to improve accuracy. Extensive experiments across multiple benchmarks and model families demonstrate that SFTKey achieves an average accuracy improvement exceeding 5\% over conventional SFT, while preserving the ability to generate correct formats. Overall, this study advances LLM fine-tuning by explicitly balancing CoT learning with additional optimization on answer-relevant tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。