arXiv:2512.14442cs.CVcs.RO2025-12被引 4

无需训练,用三个AI模块协同完成物体交互区域的零样本推理。

A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning

  • 分三阶段:想象动作、决定部位、精确定位,解耦推理与定位。
  • 零样本下在多个基准上超越有监督模型,真实场景泛化能力强。
  • 适合做具身智能、机器人交互等需要快速适应新物体的场景。

可交互性预测——根据语言指令识别物体上的交互区域——对具身人工智能至关重要。现有端到端模型将高层推理与底层定位耦合于单一管道,并依赖标注数据训练,导致在新物体和未见环境中的泛化能力差。本文提出A4-Agent,一种无需训练的代理式框架,将可交互性预测分解为三阶段流程。该框架在测试时协调专用的基础模型:(1) \\textbf{Dreamer} 利用生成模型可视化交互的\textit{如何}实现;(2) \\textbf{Thinker} 使用大视觉-语言模型判断\textit{哪个}物体部分应被交互;(3) \\textbf{Spotter} 协调视觉基础模型精确找到\textit{何处}是交互区域。通过利用预训练模型的互补优势而无需任务特定微调,该零样本框架在多个基准上显著优于最先进的有监督方法,并在真实世界设置中表现出强鲁棒性。

原文摘要 · Abstract (English)

Affordance prediction, which identifies interaction regions on objects based on language instructions, is critical for embodied AI. Prevailing end-to-end models couple high-level reasoning and low-level grounding into a single monolithic pipeline and rely on training over annotated datasets, which leads to poor generalization on novel objects and unseen environments. In this paper, we move beyond this paradigm by proposing A4-Agent, a training-free agentic framework that decouples affordance prediction into a three-stage pipeline. Our framework coordinates specialized foundation models at test time: (1) a $\textbf{Dreamer}$ that employs generative models to visualize $\textit{how}$ an interaction would look; (2) a $\textbf{Thinker}$ that utilizes large vision-language models to decide $\textit{what}$ object part to interact with; and (3) a $\textbf{Spotter}$ that orchestrates vision foundation models to precisely locate $\textit{where}$ the interaction area is. By leveraging the complementary strengths of pre-trained models without any task-specific fine-tuning, our zero-shot framework significantly outperforms state-of-the-art supervised methods across multiple benchmarks and demonstrates robust generalization to real-world settings.

具身智能零样本视觉语言模型交互预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。