用语音动作演示教AI学合作物理任务，用户可直接修正程序逻辑。
Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations
- 将协作任务学习建模为程序合成，用语音+动作统一教学与纠错
- 20人实验中70%成功修正程序匹配意图，90%认为纠正容易
- 适合需要交互式调试的协作运动教学场景
在人机交互领域，教会系统完成物理任务是长期目标，但以往研究多集中于非协作任务。协作任务引入新复杂性，要求系统推断用户对队友意图的假设，这一过程本质上模糊且动态。因此需具备可解释、可修正的表示方式，使用户能检查并调整系统行为。本文将协作任务学习视为程序合成问题，系统以可编辑程序形式表征行为，并利用配对的物理动作与自然语言（即叙述演示）作为统一教学、检查和修正系统逻辑的模态，无需用户看或写代码。系统同样用此模态向用户传达其学习结果。一项包含20名参与者的事先对照研究显示，70%（14/20）的用户成功修正了所学程序以匹配自身意图，90%（18/20）认为修正过程简单。研究揭示了以程序形式表示学习以及支持用户教授协作物理活动的独特挑战，并讨论了相应的缓解策略。
原文摘要 · Abstract (English)
Teaching systems physical tasks is a long standing goal in HCI, yet most prior work has focused on non collaborative physical activities. Collaborative tasks introduce added complexity, requiring systems to infer users assumptions about their teammates intent, which is an inherently ambiguous and dynamic process. This necessitates representations that are interpretable and correctable, enabling users to inspect and refine system behavior. We address this challenge by framing collaborative task learning as a program synthesis problem. Our system represents behavior as editable programs and uses narrated demonstrations, i.e. paired physical actions and natural language, as a unified modality for teaching, inspecting, and correcting system logic without requiring users to see or write code. The same modality is used for the system to communicate its learning to users. In a within subjects study, 20 users taught multiplayer soccer tactics to our system. 70 percent (14/20) of participants successfully refined learned programs to match their intent and 90 percent (18/20) found it easy to correct the programs. The study surfaced unique challenges in representing learning as programs and in enabling users to teach collaborative physical activities. We discuss these issues and outline mitigation strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。