用强化学习让机械臂自主塑形沙子，效果远超传统方法。
Interactive Shaping of Granular Media Using Reinforcement Learning
- 用视觉观测和简洁奖励函数训练机器人抓取沙粒
- 在真实场景中精准塑造目标形状，精度显著提升
- 适合需要自动塑形的工业制造与建造场景
自主操控颗粒材料（如沙子）在建筑、挖掘和增材制造中有重要意义。然而，由于其高维状态空间和复杂动力学，传统基于规则的方法难以实现，需大量工程投入。强化学习（RL）通过试错让智能体自主学习适应性操作策略，成为可行替代方案。本文提出一种基于强化学习的框架，使配备立方体末端执行器和立体相机的机械臂能够将颗粒材料塑造成目标结构。我们验证了紧凑观测与简明奖励设计对高维空间的重要性，并通过消融实验支持设计选择。结果表明，该方法在训练视觉策略并实现真实世界部署方面均有效，显著优于两种基线方法，在目标形状准确性上表现更优。
原文摘要 · Abstract (English)
Autonomous manipulation of granular media, such as sand, is crucial for applications in construction, excavation, and additive manufacturing. However, shaping granular materials presents unique challenges due to their high-dimensional configuration space and complex dynamics, where traditional rule-based approaches struggle without extensive engineering efforts. Reinforcement learning (RL) offers a promising alternative by enabling agents to learn adaptive manipulation strategies through trial and error. In this work, we present an RL framework that enables a robotic arm with a cubic end-effector and a stereo camera to shape granular media into desired target structures. We show the importance of compact observations and concise reward formulations for the large configuration space, validating our design choices with an ablation study. Our results demonstrate the effectiveness of the proposed approach for the training of visual policies that manipulate granular media including their real-world deployment, significantly outperforming two baseline approaches in terms of target shape accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。