让机器人同时理解人说的话和做的动作，快速学会正确行为。
QuickLAP: Quick Language-Action Preference Learning for Semi-Autonomous Agents
- 用贝叶斯框架融合语言和物理反馈，实时推断奖励函数。
- 在模拟驾驶中，错误率比纯物理反馈降低70%以上。
- 用户测试显示更易懂、更协作，偏好度显著提升。
机器人需同时学习人类的行为与言语，但单一模态常不完整：物理修正虽有实体依据却意图模糊，语言表达高层目标却缺乏物理基础。我们提出QuickLAP：一种快速语言-动作偏好学习方法，基于贝叶斯框架融合物理与语言反馈以实时推断奖励函数。核心思想是将语言视为用户潜在偏好的概率观测，明确哪些奖励特征重要，并解释物理修正的含义。QuickLAP利用大语言模型（LLMs）从自由语句中提取奖励特征注意力掩码与偏好变化，结合物理反馈通过闭式更新规则实现快速、鲁棒的奖励学习。在半自主驾驶模拟器中，相比仅依赖物理反馈及启发式多模态基线，其奖励学习误差降低超过70%。15名参与者用户研究进一步验证：用户认为QuickLAP更易理解、更具协作性，且更偏好其生成的行为表现。代码已开源于https://github.com/MIT-CLEAR-Lab/QuickLAP。
原文摘要 · Abstract (English)
Robots must learn from both what people do and what they say, but either modality alone is often incomplete: physical corrections are grounded but ambiguous in intent, while language expresses high-level goals but lacks physical grounding. We introduce QuickLAP: Quick Language-Action Preference learning, a Bayesian framework that fuses physical and language feedback to infer reward functions in real time. Our key insight is to treat language as a probabilistic observation over the user's latent preferences, clarifying which reward features matter and how physical corrections should be interpreted. QuickLAP uses Large Language Models (LLMs) to extract reward feature attention masks and preference shifts from free-form utterances, which it integrates with physical feedback in a closed-form update rule. This enables fast, real-time, and robust reward learning that handles ambiguous feedback. In a semi-autonomous driving simulator, QuickLAP reduces reward learning error by over 70% compared to physical-only and heuristic multimodal baselines. A 15-participant user study further validates our approach: participants found QuickLAP significantly more understandable and collaborative, and preferred its learned behavior over baselines. Code is available at https://github.com/MIT-CLEAR-Lab/QuickLAP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。