让机器人通过理解用户意图实现奖励函数跨场景迁移。
GIFT: Generalizing Intent for Flexible Test-Time Rewards
- 用语言模型对比优劣行为，提取高层意图作为泛化基础。
- 在4个仿真任务中对50+新物体测试，胜率与对齐度均优于基线。
- 适合需要快速适配新环境的机器人部署场景。
机器人通过用户示范学习奖励函数，但这些奖励常无法推广到新环境。原因在于学习到的奖励会捕捉训练数据中的虚假相关性，而非示范所体现的人类真实意图。现有方法依赖视觉或语义相似性提升鲁棒性，但这些表层线索往往偏离人类真正关注点。本文提出通用意图泛化框架GIFT（Generalizing Intent for Flexible Test-Time Rewards），将奖励泛化建立在人类意图基础上，而非表面特征。GIFT利用语言模型通过对比优选与非优选行为，从用户示范中推断高层次意图。部署时，GIFT通过意图条件下的相似性，将新测试状态映射到行为等价的训练状态，使学习到的奖励在无需重训练的前提下跨分布变化有效泛化。我们在桌面操作任务上评估GIFT，包含新物体和新布局。在四个模拟任务中，超过50种未见物体的测试下，GIFT在测试时成对胜率和状态对齐F1分数上持续优于视觉与语义相似性基线。真实世界实验在7自由度Franka Panda机器人上验证了GIFT在物理场景中的可靠迁移能力。更多信息见https://mit-clear-lab.github.io/GIFT/
原文摘要 · Abstract (English)
Robots learn reward functions from user demonstrations, but these rewards often fail to generalize to new environments. This failure occurs because learned rewards latch onto spurious correlations in training data rather than the underlying human intent that demonstrations represent. Existing methods leverage visual or semantic similarity to improve robustness, yet these surface-level cues often diverge from what humans actually care about. We present Generalizing Intent for Flexible Test-Time Rewards (GIFT), a framework that grounds reward generalization in human intent rather than surface cues. GIFT leverages language models to infer high-level intent from user demonstrations by contrasting preferred with non-preferred behaviors. At deployment, GIFT maps novel test states to behaviorally equivalent training states via intent-conditioned similarity, enabling learned rewards to generalize across distribution shifts without retraining. We evaluate GIFT on tabletop manipulation tasks with new objects and layouts. Across four simulated tasks with over 50 unseen objects, GIFT consistently outperforms visual and semantic similarity baselines in test-time pairwise win rate and state-alignment F1 score. Real-world experiments on a 7-DoF Franka Panda robot demonstrate that GIFT reliably transfers to physical settings. Further discussion can be found at https://mit-clear-lab.github.io/GIFT/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。