用强化学习动态调整建筑顺序,让机器人自适应真实施工误差。
Learning to build covering structures with continuous adjustments

- 用图神经网络和连续参数化动作,实时生成建筑序列
- 在模拟中达到比旧方法高得多的稳定结构成功率
- 能应对10种以上积木选择且不降效,适合真实机器人部署
机器人建造有望更高效地使用材料并实现复杂几何形态,但现有方法依赖高精度预设方案,难以应对物理制造中的容差、误差与意外变化。本文提出一种强化学习方法,完全摒弃预设计划,根据实际建造过程自适应生成施工序列。该方法基于图结构状态表示和混合(参数化)动作空间,需同时决策离散积木选择与连续放置参数。由于结构稳定性仿真计算成本高,我们通过在图神经网络中引入单向边,设计了高效探索策略,将软演员-评论家(SAC)扩展至该混合设置。我们提出的HSAC算法在性能上显著优于先前的hybrid-PPO(HPPO),收敛速度更快,样本效率更高。此外,HSAC对超参数不敏感,可处理最多10种离散动作而性能无下降。最后,在双机器人物理平台上验证了该方法,成功闭环构建出由3D打印积木组成的跨越拱形结构,证明了仿真训练策略可有效迁移到真实硬件。
原文摘要 · Abstract (English)
Robotic construction offers the potential to use materials more efficiently and create complex geometries, but current methods rely on rigid, high-precision plans that cannot accommodate the tolerances, inaccuracies, and unexpected changes inherent in physical fabrication. In this work, we introduce a reinforcement learning approach that forgoes predefined plans entirely, instead generating construction sequences adaptively as the structure is built. Our method operates on graph-structured state representations and a mixed (parameterized) action space, requiring both discrete block selection and continuous placement parameters. Because the stability simulation of a structure is computationally heavy, we develop an efficient exploration strategy by incorporating unilateral edges into graph neural networks, extending soft actor-critic (SAC) to this hybrid setting. We evaluate our algorithm, HSAC, against the prior method hybrid-PPO (HPPO), demonstrating significantly higher asymptotic performance and good sample efficiency. We also demonstrate HSAC's robustness to hyperparameter choices and its exploration capability, handling up to 10 discrete actions without performance degradation. Finally, we validate our approach on a physical two-robot setup, successfully building a spanning arch with 3D-printed blocks in closed-loop execution, confirming that policies trained in simulation transfer to real hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。