仅用1个无标签数据和10步优化，就能显著提升大模型性能。
One-shot Entropy Minimization
- 用单个无标签样本和极简优化，实现熵最小化
- 性能媲美甚至超越千样本+人工奖励的强化学习方法
- 适合追求高效微调的开发者与研究者
我们训练了13,440个大型语言模型，发现仅需一个无标签数据和10步优化，即可实现与基于规则的强化学习中使用数千个数据并精心设计奖励函数相当甚至更优的性能提升。这一惊人结果或促使人们对大语言模型的后训练范式进行重新思考。代码已开源：https://github.com/zitian-gao/one-shot-em。
原文摘要 · Abstract (English)
We trained 13,440 large language models and found that entropy minimization requires only a single unlabeled data and 10 steps optimization to achieve performance improvements comparable to or even greater than those obtained using thousands of data and carefully designed rewards in rule-based reinforcement learning. This striking result may prompt a rethinking of post-training paradigms for large language models. Our code is avaliable at https://github.com/zitian-gao/one-shot-em.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。