让智能体学会在不确定中判断物体能否操作，提升真实场景适应力。
ADAPT: Benchmarking Commonsense Planning under Unspecified Affordance Constraints
- 引入动态可操作性推理模块，让智能体根据环境状态自主判断操作可行性。
- 在新旧环境中任务成功率显著提升，尤其在对象属性变化时表现更稳定。
- 适合做具身智能、机器人规划的研究者，关注实际应用鲁棒性的人必看。
智能体不应仅机械执行指令，因为现实环境常存在意外状况和例外。现有方法通常直接执行指令，忽视目标物体是否真正可操作，导致无法评估可用的可操作性。为解决此问题，我们提出 DynAfford 基准，用于评测智能体在动态环境中面对未明确说明的可操作性约束时的表现。该基准要求智能体感知物体状态、推断隐含前提，并相应调整行为。为此，我们引入 ADAPT 模块，作为即插即用组件增强现有规划器的显式可操作性推理能力。实验表明，加入 ADAPT 显著提升了模型在已见与未见环境中的鲁棒性和任务成功率。此外，使用领域适配的 LoRA 微调视觉-语言模型作为可操作性推断后端，优于商用大模型 GPT-4o，凸显了任务对齐的可操作性建模的重要性。
原文摘要 · Abstract (English)
Intelligent embodied agents should not simply follow instructions, as real-world environments often involve unexpected conditions and exceptions. However, existing methods usually focus on directly executing instructions, without considering whether the target objects can actually be manipulated, meaning they fail to assess available affordances. To address this limitation, we introduce DynAfford, a benchmark that evaluates embodied agents in dynamic environments where object affordances may change over time and are not specified in the instruction. DynAfford requires agents to perceive object states, infer implicit preconditions, and adapt their actions accordingly. To enable this capability, we introduce ADAPT, a plug-and-play module that augments existing planners with explicit affordance reasoning. Experiments demonstrate that incorporating ADAPT significantly improves robustness and task success across both seen and unseen environments. We also show that a domain-adapted, LoRA-finetuned vision-language model used as the affordance inference backend outperforms a commercial LLM (GPT-4o), highlighting the importance of task-aligned affordance grounding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。