arXiv:2507.17275cs.ROcs.LG2025-07被引 1

让机器人学会用工具不伤工具,延长使用寿命

Prolonging Tool Life: Learning Skillful Use of General-purpose Tools through Lifespan-guided Reinforcement Learning

  • 用强化学习结合寿命预测,指导机器人安全使用通用工具
  • 仿真中工具寿命最长提升8.01倍,真实场景也有效
  • 适合做长期作业的机器人系统,尤其在不确定环境中

在任务需求不确定的复杂环境中,机器人常依赖未预设使用方式的通用工具,其寿命高度依赖使用方式。如何在完成任务的同时延长工具寿命成为关键挑战。本文提出一种融合工具剩余使用寿命(RUL)的强化学习框架,利用有限元分析(FEA)和密纳法则估算累积应力下的RUL,并将其纳入奖励函数以引导策略学习。针对RUL需任务完成后才能估计的问题,设计自适应奖励归一化机制(ARN),动态调整奖励尺度,保障学习稳定性。在模拟与真实世界任务中验证,包括物体移动和开门操作,使用多种通用工具。结果表明,所学策略显著延长工具寿命(仿真中最高达8.01倍),且能有效迁移到真实场景,证明了该方法在实际应用中的价值。

原文摘要 · Abstract (English)

In inaccessible environments with uncertain task demands, robots often rely on general-purpose tools that lack predefined usage strategies. These tools are not tailored for particular operations, making their longevity highly sensitive to how they are used. This creates a fundamental challenge: how can a robot learn a tool-use policy that both completes the task and prolongs the tool's lifespan? In this work, we address this challenge by introducing a reinforcement learning (RL) framework that incorporates tool lifespan as a factor during policy optimization. Our framework leverages Finite Element Analysis (FEA) and Miner's Rule to estimate Remaining Useful Life (RUL) based on accumulated stress, and integrates the RUL into the RL reward to guide policy learning toward lifespan-guided behavior. To handle the fact that RUL can only be estimated after task execution, we introduce an Adaptive Reward Normalization (ARN) mechanism that dynamically adjusts reward scaling based on estimated RULs, ensuring stable learning signals. We validate our method across simulated and real-world tool use tasks, including Object-Moving and Door-Opening with multiple general-purpose tools. The learned policies consistently prolong tool lifespan (up to 8.01x in simulation) and transfer effectively to real-world settings, demonstrating the practical value of learning lifespan-guided tool use strategies.

强化学习工具使用寿命预测机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。