让大模型在不更新权重的情况下学会在线决策，表现媲美GPT-5。
Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL
- 用元强化学习训练模型从上下文互动中持续学习
- 小模型Qwen3-14B在新环境上表现接近GPT-5
- 适合需要实时决策的智能体场景
大语言模型在任务信息全量提供时表现优异，如静态预测和指令遵循。但现实中许多决策任务具有在线特性：关键信息需通过交互获取，反馈延迟，行为需权衡信息收集与利用。尽管上下文学习可实现无权重更新的适应，现有大模型在此类场景中仍难以可靠利用上下文交互经验。本文提出ORBIT框架——一种多任务、多轮次的元强化学习方法，训练大模型在上下文中学会交互式学习。经元训练后，小型开源模型Qwen3-14B在全新环境中展现出显著提升的在线学习能力，性能逼近GPT-5.2，远超标准强化学习微调。扩展实验表明，模型规模越大，性能提升越明显，显示推理时学习型智能体仍有巨大发展空间。代码见https://github.com/XiaofengLin7/ORBIT。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve strong performance when all task-relevant information is available upfront, as in static prediction and instruction-following problems. However, many real-world decision-making tasks are inherently online: crucial information must be acquired through interaction, feedback is delayed, and effective behavior requires balancing information collection and exploitation over time. While in-context learning enables adaptation without weight updates, existing LLMs often struggle to reliably leverage in-context interaction experience in such settings. In this work, we show that this limitation can be addressed through training. We introduce ORBIT, a multi-task, multi-episode meta-reinforcement learning framework that trains LLMs to learn from interaction in context. After meta-training, a relatively small open-source model (Qwen3-14B) demonstrates substantially improved in-context online learning on entirely unseen environments, matching the performance of GPT-5.2 and outperforming standard RL fine-tuning by a large margin. Scaling experiments further reveal consistent gains with model size, suggesting significant headroom for learn-at-inference-time decision-making agents. Code reproducing the results in the paper can be found at https://github.com/XiaofengLin7/ORBIT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。