让大模型学会不被提示词细节干扰，提升泛化能力
PAFT: Prompt-Agnostic Fine-Tuning
- 训练时动态变化提示词，避免模型记住具体措辞
- 在未见提示下准确率提升7%，推理速度加快3.2倍
- 适合需要稳定表现的问答、数学和工具调用场景
微调大语言模型时常因对特定提示词表述过度敏感而过拟合，轻微措辞变化即导致性能大幅下降。为此，我们提出提示无关微调（PAFT），通过训练期间动态生成并采样多样化合成提示，迫使模型学习任务本质原理而非表面模式。在监督微调（SFT）与强化学习微调（RLFT）中系统评估均显示，PAFT显著提升提示鲁棒性，在未见过的提示上实现比标准方法高7%的泛化准确率。此外，PAFT在问答、数学推理和工具使用等基准测试中持续取得更优表现。值得注意的是，经PAFT训练的模型推理速度提升3.2倍，源于其对提示敏感度降低。消融实验验证了方法有效性，理论分析表明PAFT能有效增强模型跨领域泛化能力。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) often causes overfitting to specific prompt wording, where minor phrasing variations drastically reduce performance. To address this, we propose Prompt-Agnostic Fine-Tuning (PAFT), a method that enhances robustness through dynamic prompt variation during training. PAFT first generates diverse synthetic prompts, then continuously samples from this set to construct training instances, forcing models to learn fundamental task principles rather than surface-level patterns. Across systematic evaluations using both supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RLFT), PAFT demonstrates substantially improved prompt robustness, achieving 7% higher generalization accuracy on unseen prompts than standard methods. In addition to enhanced robustness, PAFT consistently yields superior overall performance on established benchmarks for question answering, mathematical reasoning, and tool use. Notably, models trained with PAFT attain 3.2 faster inference speeds due to reduced prompt sensitivity. Ablation studies further validate effectiveness of PAFT, while theoretical analysis reveals that PAFT can effectively enhance the cross-domain generalization ability of LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。