用大规模预训练让挖机自动适应新工地和设备,无需重写代码。
ExT: Towards Scalable Autonomous Excavation via Large-Scale Multi-Task Pretraining and Fine-Tuning
- 先用多任务示范数据预训练,再微调适配新任务
- 仿真与实机测试均达厘米级精度,性能媲美专用控制器
- 适合想快速部署挖机自动化的工程公司和研究者
规模化部署自主挖掘机具有重要经济与社会意义,但现有系统难以应对未知工况与新硬件配置。当前先进方法依赖高度定制的单任务控制器,需大量人工调参。本文提出ExT,一个统一的开源框架,用于大规模示范数据收集、多任务挖机策略预训练与微调。ExT策略首先在混合专家示范数据上预训练,再通过监督微调(SFT)或强化学习微调(RLFT)适配新任务或操作条件。仿真与真实实验表明,预训练的ExT策略可完成完整挖掘周期,实现厘米级精度,并成功从仿真迁移到真实机器,性能接近专用单任务控制器。此外,在仿真中验证了其对新任务、分布外条件及设备配置的快速适应能力,同时保持对已学任务的高性能。结果表明ExT可作为可扩展、通用的自主挖机基础。
原文摘要 · Abstract (English)
Scaling up the deployment of autonomous excavators is of great economic and societal importance. Yet it remains a challenging problem, as effective systems must robustly handle unseen worksite conditions and new hardware configurations. Current state-of-the-art approaches rely on highly engineered, task-specific controllers, which require extensive manual tuning for each new scenario. In contrast, recent advances in large-scale pretrained models have shown remarkable adaptability across tasks and embodiments in domains such as manipulation and navigation, but their applicability to heavy construction machinery remains largely unexplored. In this work, we introduce ExT, a unified open-source framework for large-scale demonstration collection, pretraining, and fine-tuning of multitask excavation policies. ExT policies are first trained on large-scale demonstrations collected from a mix of experts, then fine-tuned either with supervised fine-tuning (SFT) or reinforcement learning fine-tuning (RLFT) to specialize to new tasks or operating conditions. Through both simulation and real-world experiments, we show that pretrained ExT policies can execute complete excavation cycles with centimeter-level accuracy, successfully transferring from simulation to real machine with performance comparable to specialized single-task controllers. Furthermore, in simulation, we demonstrate that ExT's fine-tuning pipelines allow rapid adaptation to new tasks, out-of-distribution conditions, and machine configurations, while maintaining strong performance on previously learned tasks. These results highlight the potential of ExT to serve as a foundation for scalable and generalizable autonomous excavation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。