用强化微调让通用推理模型高效适配特定领域，仅需100样本即显著提效。
OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks with Reinforcement Fine-Tuning
- 通过问题增强与推理过程合成，解决领域数据稀缺难题。
- 在SciKnowEval上仅用100样本即实现显著性能提升。
- 适合需要少量标注数据的垂直领域模型定制场景。
OpenAI近期提出的强化微调（RFT）展示了推理基础模型的潜力，并为超越简单模式模仿的微调提供了新范式。本文提出 extit{OpenRFT}，旨在在与RFT相同的设置下，对通用推理模型进行领域特定任务的微调。针对缺乏推理步骤数据和训练样本有限两大挑战，OpenRFT通过三种方式利用领域特定样本：问题增强、推理过程数据合成以及少样本上下文学习。评估在SciKnowEval上进行，结果显示，每个任务仅需100个领域样本即可实现显著性能提升。更多实验结果将随版本持续更新。源代码、数据集和模型已公开于：https://github.com/ADaM-BJTU/OpenRFT。
原文摘要 · Abstract (English)
OpenAI's recent introduction of Reinforcement Fine-Tuning (RFT) showcases the potential of reasoning foundation model and offers a new paradigm for fine-tuning beyond simple pattern imitation. This technical report presents \emph{OpenRFT}, our attempt to fine-tune generalist reasoning models for domain-specific tasks under the same settings as RFT. OpenRFT addresses two key challenges of lacking reasoning step data and the limited quantity of training samples, by leveraging the domain-specific samples in three ways: question augmentation, synthesizing reasoning-process data, and few-shot ICL. The evaluation is conducted on SciKnowEval, where OpenRFT achieves notable performance gains with only $100$ domain-specific samples for each task. More experimental results will be updated continuously in later versions. Source codes, datasets, and models are disclosed at: https://github.com/ADaM-BJTU/OpenRFT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。