用说明书自动生成家电操作数据，让机器人学会复杂家务
Scaling Manual-Grounded Appliance Manipulation with Data Synthesis and Unified Planning
- 基于家电手册构建分层设备图，自动合成操作步骤与纠错数据
- 建成89000+部件标注、53000+任务的大规模数据集，支撑长程规划
- 70亿参数模型在真实机器人上成功完成6类家电操作，性能超基线10倍
家用电器操作需要长时序、状态依赖且抗干扰的规划能力，但现有大模型因缺乏足够多样、任务导向的数据而表现不足。为此，我们提出MAGE，一个可扩展的数据合成管道，引入新型分层设备图（HAG），从家电说明书自动生成部件定位、长时序规划及闭环恢复数据。基于MAGE，我们构建了UseAppliance——首个基于说明书的家电操作规划大规模数据集，涵盖22类家电，包含89,000+部件标注、53,000+操作任务和33,000+闭环调整步骤。在此基础上，我们开发了AppliancePlan，一种端到端的说明书驱动家电操作规划模型。在RealAppliance-Bench上，仅含70亿参数的AppliancePlan在开环规划任务中性能超过最优基线10倍以上，并在所有任务上持续优于当前最先进模型。六种家用电器的真实机器人实验进一步验证了良好的仿真到现实迁移能力，标志着通用家庭机器人的重要进展。
原文摘要 · Abstract (English)
Operating household appliances requires long-horizon planning that is state-dependent and robust to disturbances, yet existing large models fall short, as no sufficiently diverse, task-oriented dataset exists to support such planning. To bridge this gap, we propose MAGE, a scalable data synthesis pipeline that introduces a novel Hierarchical Appliance Graph (HAG) to automatically generate part grounding, long-horizon planning, and closed-loop recovery data from appliance manuals. With MAGE, we build UseAppliance, the first large-scale dataset for manual-grounded appliance manipulation planning, spanning 22 appliance categories with 89K+ part annotations, 53K+ manipulation tasks, and 33K+ closed-loop adjustment steps. Built on UseAppliance, we develop AppliancePlan, an end-to-end model for manual-grounded appliance manipulation planning. On RealAppliance-Bench, AppliancePlan with only 7B parameters achieves over 10x the best baseline on open-loop planning and consistently outperforms state-of-the-art models across all tasks. Real-robot experiments on six household appliances further confirm effective sim-to-real transfer, marking an important step toward general-purpose household robotics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。