不给指令也能懂任务，预训练模型自带理解能力
Revealing the Inherent Instructability of Pre-Trained Language Models
- 只用回答数据训练，不用指令-响应配对
- 模型能准确回应各类指令，安全识别率超90%
- 适合研究模型内在能力的学者与工程师
指令微调是让预训练大语言模型具备指令理解能力的关键步骤。然而，大模型在预训练阶段已进行多任务学习，积累了丰富知识与能力。我们假设预训练过程使其具备理解并响应指令的潜在能力。为此提出响应微调（Response Tuning, RT），移除指令及其到回答的映射，仅关注建立回答分布。实验表明，仅用回答数据训练的RT模型，可有效应对多种指令，表现接近指令微调模型。此外，模型仅通过回答数据学习安全策略后，即可识别并拒绝危险查询。该现象在上下文学习设置中亦成立。这些发现验证了我们的假设，揭示了预训练大模型的深层内在能力。
原文摘要 · Abstract (English)
Instruction tuning -- supervised fine-tuning using instruction-response pairs -- is a key step in making pre-trained large language models (LLMs) instructable. Meanwhile, LLMs perform multitask learning during their pre-training, acquiring extensive knowledge and capabilities. We hypothesize that the pre-training stage can enable them to develop the ability to comprehend and address instructions. To verify this, we propose Response Tuning (RT), which removes the instruction and its corresponding mapping to the response from instruction tuning. Instead, it focuses solely on establishing a response distribution. Our experiments demonstrate that RT models, trained only on responses, can effectively respond to a wide range of instructions akin to their instruction-tuned counterparts. In addition, we observe that the models can recognize and reject unsafe queries after learning a safety policy only from the response data. Furthermore, we find that these observations extend to an in-context learning setting. These findings support our hypothesis, highlighting the extensive inherent capabilities of pre-trained LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。