让机器人执行抓取任务时能自动识别失败并重试,提升成功率。
A Physical Agentic Loop for Language-Guided Grasping with Execution-State Monitoring

- 构建物理代理循环,通过监控机械臂状态实现动作反馈闭环。
- 在复杂场景下抓取成功率提升至92.3%,失败后可自动重试或上报。
- 无需修改原有模型,适合部署在真实机器人上做稳健操作。
遵循语言指令的机器人操作常采用单次执行方式:模型提出动作,机器人执行,但空抓、滑脱、卡滞、超时或语义错误等失败情况未被结构化反馈至决策层。受数字工具使用代理中代理循环的启发,我们将语言引导抓取重构为一个在具身执行状态上运行的有限体化代理,使物理动作暴露显式的工具状态流。我们引入一个物理代理循环,将未经修改的已学习抓取原语(抓取-抬起)封装为(i)基于事件的接口和(ii)执行监控层Watchdog,该层利用接触感知融合与时间稳定化,将嘈杂的夹爪传感数据转化为离散的结果标签。这些结果事件可结合抓取后语义验证,由确定性的有限策略消费,完成最终决策、重试或向用户求助,确保有限终止。我们在配备眼内手D405相机的移动操作臂上验证了该系统,保持底层抓取模型不变,在包含视觉模糊、干扰物及人为制造执行失败的代表性场景中评估。结果显示,显式执行状态监控与有限恢复机制显著提升了鲁棒性与可解释性,且仅带来极小架构开销。
原文摘要 · Abstract (English)
Robotic manipulation systems that follow language instructions often execute grasp primitives in a largely single-shot manner: a model proposes an action, the robot executes it, and failures such as empty grasps, slips, stalls, timeouts, or semantically wrong grasps are not surfaced to the decision layer in a structured way. Inspired by agentic loops in digital tool-using agents, we reformulate language-guided grasping as a bounded embodied agent operating over grounded execution states, where physical actions expose an explicit tool-state stream. We introduce a physical agentic loop that wraps an unmodified learned manipulation primitive (grasp-and-lift) with (i) an event-based interface and (ii) an execution monitoring layer, Watchdog, which converts noisy gripper telemetry into discrete outcome labels using contact-aware fusion and temporal stabilization. These outcome events, optionally combined with post-grasp semantic verification, are consumed by a deterministic bounded policy that finalizes, retries, or escalates to the user for clarification, guaranteeing finite termination. We validate the resulting loop on a mobile manipulator with an eye-in-hand D405 camera, keeping the underlying grasp model unchanged and evaluating representative scenarios involving visual ambiguity, distractors, and induced execution failures. Results show that explicit execution-state monitoring and bounded recovery enable more robust and interpretable behavior than open-loop execution, while adding minimal architectural overhead. For the source code and demo refer to our project page: https://wenzewwz123.github.io/Agentic-Loop/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。