arXiv:2607.29172cs.ROcs.AI2026-07

通过闭环迭代微调,让封闭模型在不接触内部参数的情况下实现人形机器人任务精通。

CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-Tuning

论文配图:CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-Tuning
图 1 · 摘自论文原文
  • 将部署时的奖励反馈转化为API兼容的监督数据,实现闭环优化。
  • 在两次飞轮循环后,使GROD在高动态接触任务上接近完美成功率。
  • 无需访问模型权重或梯度,适合无法获取源代码的闭源模型应用。

尽管机器人基础模型能力日益增强,但最强模型通常基于专有数据训练且闭源,限制了下游用户将其适配到新任务、新形态和部署环境的能力。借鉴大语言模型社区,一种新兴的访问范式是受控监督微调(SFT)API:用户提交训练数据,获得微调策略,但无法访问模型权重、梯度或训练内部信息。此类API虽使用户可利用强大闭源模型,却仅支持纯模仿学习,排除依赖内部信号的强化学习等闭环方法。这一局限在高动态、接触密集的人形操作任务中尤为突出,因策略输出与实际部署行为间存在显著差距,涉及新状态、动作追踪动态、延迟及控制器特异性故障模式。本文首次在真实人形机器人上对受控API适应性进行实证研究,基于Gemini Robotics On-Device(GROD)平台。结果表明,通过API直接进行SFT显著优于同演示数据训练的开源视觉语言模型,但仍未能达到部署级任务精通。为弥合差距,提出CLIFT:闭环迭代微调,将部署时的奖励反馈转换为API兼容的监督数据,实现无权访问下的闭环策略改进,在两次飞轮循环后使GROD接近完美成功,彻底无需‘打开模型盒子’。

原文摘要 · Abstract (English)

While robot foundation models are growing increasingly capable, the strongest models are typically trained on proprietary data and remain closed-source, limiting downstream users' ability to adapt them to new tasks, embodiments, and deployment settings. Following the LLM community, an emerging access paradigm for closed-weight robot foundation models is the managed supervised fine-tuning (SFT) API, where users submit training data and receive a tuned policy without access to model weights, gradients, or training internals. While such APIs let downstream users leverage powerful proprietary foundation models, they restrict policy improvement to pure imitation, ruling out reinforcement learning and other closed-loop methods that rely on internal training signals. This limitation is particularly acute for agile, contact-rich humanoid manipulation, where the gap between policy outputs and deployed behavior is large due to novel states, action tracking dynamics, latency, and controller-specific failure modes. We study how effective this managed-API regime is for humanoid adaptation, and how closed-loop improvement can be realized within it to push policies toward task mastery. We conduct one of the first empirical studies of managed-API adaptation on a real humanoid, instantiated on Gemini Robotics On-Device (GROD). We find that direct SFT through the API substantially outperforms a leading open-weight VLA trained on the same demonstrations, yet still falls short of deployment-level mastery on agile, contact-rich tasks. To close this gap, we introduce CLIFT: Closed-Loop Iterative Fine-Tuning, which turns deployment-time reward feedback into API-compatible supervised data and enables closed-loop policy improvement without accessing weights, gradients, likelihoods, or losses-pushing GROD to near-perfect success after two flywheel cycles, all without "opening the model box."

人形机器人闭环微调闭源模型API优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。